Skip to content

Latest commit

 

History

History
19 lines (13 loc) · 936 Bytes

README_template.md

File metadata and controls

19 lines (13 loc) · 936 Bytes
Raw

{{ projectName }}

![Scrape data](https://github.com/reuters-graphics/action_{{ projectName }}/workflows/Scrape%20data/badge.svg)

A GitHub Action-based scraper. Built from our house bluprint.

Data endpoint

data.json [https://graphics.thomsonreuters.com/data/2021/{{ projectName }}/data.json](https://graphics.thomsonreuters.com/data/2021/{{ projectName }}/data.json)

Developing

  1. Customize the script in index.js to scrape your data and upload it to somewhere on our graphics.thomsonreuters.com S3 bucket.
  2. Set the cron schedule in .github/workflows/action.yaml.
  3. Create a repository on the reuters-graphics GitHub org. Prefix the name with action_
  4. Commit your repo to GitHub. You can check your action in the Actions tab.
  5. You can run the scraper locally using yarn run scrape