Skip to content
Scrapes the Daily Wail Online for articles containing specific words and phrases
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
README.md
mailscraper4.py
output.csv Refactored code Feb 18, 2018

README.md

Transparency-Scraper

Scrapes the Daily Wail Online for articles containing specific words and phrases

What is this?

A member of the Transparency Project asked me whether it is possible to scrape news articles published on various online news outlets for articles containing certain trigger words that could be used to flag articles of interest to the members of the Transparency Project.

Here's a basic working prototype of such a tool, which outputs a CSV file with the trigger word or trigger phrase in one column and the link to the article returning a match in the other.

You can’t perform that action at this time.