Implement Providers in "upcoming" script

[mathisbeauty]

Providers for the dates of upcoming launches

Introduction

This PR implements a new structure for the "upcoming" script that simplifies and facilitates the implementation of additional sources of information for the launch date, when the launch is within a specified threshold. I've called each source of information for the script, a "provider". The goal is to make it easier to implement other providers in the future, like trusted users and Twitter scraping, into the calculation of the dates of upcoming launches.

Details & Motivation

In order to:

• Clean the "upcoming" script code and make it more maintainable and testable
• Get ready for the addition of Twitter scraping, trusted users and possibly other APIs or parts of the wiki in the future

The two current providers of launch dates, the main code for retrieving and processing, both the wiki manifest and the data from the Launch Library API, have been separated into different files, leaving in the "upcoming" script mostly the logic merging the dates from both sources and update the database.

With this implementation it should be far less complicated and cumbersome to implement other providers. Also it makes the main script more readable and shorter.

There was a bug in my previous implementation, which calculated wrongly the last update time of the wiki, it used the last time the API fetched the manifest, which was an inaccurate measurement.
The new code relies instead on the history of revisions of the wiki, available here, and keeps track of the UUID of the revision and actually compares the stored launch date and the new launch date in the manifest, to see if it changed.

Finally, the new fields required to make the new code work will only be added to the launch document in the database once it is within the threshold, so it will not alter any launch until it is close enough.

Changes in the database

New Fields

Here are the new fields for the launches (Only added when the launch enters the threshold).

Wiki provider

• last_wiki_launch_date: The most recent launch date from the manifest
• last_wiki_update: The date of the most recent revision we use of the manifest
• last_wiki_revision: The UUID of the revision from which we grab the launch date

LL provider

• last_ll_launch_date: The most recent launch date from Launch Library. Is null if it cannot be reached or the mission name hasn't passed the fuzzy match.
• last_ll_update: The date of the last update of the launch date as given by LL. Is null if it cannot be reached or the mission name hasn't passed the fuzzy match.

General

• launch_date_source: Which provider's date is used to calculate the main dates of the launch. For now, it can be "wiki" or "launch_library".
• last_date_update: The date of the most recent update to the launch date, from the provider in 'launch_date_source'

Fields that have to be removed manually

And these are the fields, added by the old code, that should be removed from the live database because they are replaced and improved by the new ones:

• is_date_from_wiki
• last_wiki_update

[jakewmeyer]

Sorry for the delay, planning on merging this weekend alongside the database changes.