2018 - Collecting and Storing Data from Internet-based Sources

Course summary:
Many websites allow researchers and developers to download data using their Application Programming Interface (API). This data is often in formats that social scientists are unfamiliar with (e.g. JSON). Downloaded data can be processed immediately or stored in a database for later processing in a package like R or Stata. Data can be collected at regular intervals over a period of time, using the built-in functionality of the Windows or Linux operating systems.
Course content:
Course participants will be introduced to the following:
- Understand the JSON data format
- Understand how to use APIs to collect data
- Data storage and retrieval using a database (SQLite)
- Ability to set up automated procedures to collect data