Archive new term

[ondenman]

What does this do?

Uses scraped-page-archive to archive all pages scraped.

Why is this needed?

The country recently had an election coming up (2016-09-04). I hoped that we might be able to archive the previous term before it disappears but it looks like the site now lists the current term. As I had already begun to add the scraper, archiving the current term was only a trivial step -- at least it's now archived for the future.

Archiving it now gives us the chance to go back and re-scrape later even if it disappears.

Checklists:

Scraper Change checklist

• 1. scraper is on Morph.io under the "everypolitician-scrapers" group?
• 2. scraper's GitHub "Website" link points at morph.io page?
• 3. scraper is set to auto-run?
• 4. scraper is archiving?
• 5. legislature has a scraper webhook set?

Adding Archiving:

• 1. we are using at least version 0.5 of scraped_page_archive gem?
• 2. scraper uses scraped_page_archive gem directly or via a suitable strategy?
• 3. MORPH_SCRAPER_CACHE_GITHUB_REPO_URL is configured?
• 4. pages are being archived in new branch of correct scraper repo?

[tmtmtmtm]

This is really doing two different things:

1. Adding archiving
2. Scraping a new term

The description claims it's only doing the first, but the code also does the second. You should only add archiving to a scraper that's already working, or as the first step of an entirely new scraper. So I would suggest you either begin an entirely new scraper, starting with archiving the pages required, and then extending it to scrape the data you want, or adjust the existing scraper to work for the new term (ideally rewriting for Scraped), and then archive that when done.

But trying to do both in one step like this isn't really working.