Web-scraping for Social Science Research
Vast swathes of our social interactions and personal behaviours are now conducted online and/or captured digitally. In addition to common sources such as social media/network platforms and text corpora, websites and online databases contain rich information of relevance to social science research. Thus, computational methods for collecting data from the web are an increasingly important component of a social scientist’s toolkit.
The following topics are covered under this training series:
- Case Study - understand the research potential of web-scraping through examining a published piece of social science research.
- Websites as a Source of Data - learn how to collect data from websites using Python.
- APIs as a Source of Data - learn how to download data from online databases using Python and Application Programming Interfaces (APIs).
The training materials - including webinar recordings, slides, and sample Python code - can be found in the following folders:
- code - run and/or download web-scraping code using our Jupyter notebook resources.
- faq - read through some of the frequently asked questions that are posed during our webinars.
- installation - view instructions for how to download and install Python and other packages necessary for working with new forms of data.
- reading-list - explore further resources including articles, books, online resources and more.
- webinars - watch recordings of our webinars and download the underpinning slides.
We are grateful to UKRI through the Economic and Social Research Council for their generous funding of this training series.
- To access learning materials from the wider New Forms of Data for Social Science Research training series: [Training Materials]
- To keep up to date with upcoming and past training events: [Events]
- To get in contact with feedback, ideas or to seek assistance: [Help]
Thank you and good luck on your journey exploring new forms of data!