Mark data gaps in the database

[davidrg]

With #7 implemented the desktop client will only contact the server for archive data if:

• The user requested data that outside the timespan covered by the cache database
• There are detected gaps in the cache databases coverage of the requested timespan

Gaps are detected by the desktop client itself when data is downloaded and any data file with detected gaps is marked as incomplete. Any requests for data that cover the entire or middle portion of that data file will result in the data file being re-downloaded.

It would be nice if the Web UI could provide a list of known gaps in the dataset that will never be filled. On download if the desktop clients detected gaps line up with the list of known gaps it would mark the data file as complete despite there being missing data. This would allow the data to be cached indefinitely with no further requests to the server.

So:

• Database table to store gaps per station. Should have start time and end time, description column. Probably a ranged unique index on the timespan.
• Postgresql only desktop report which shows all gaps - both known (entry in the gaps table) and unknown.
• Console tool to detect gaps in the database and populate the gaps table optionally asking user to provide a description for each one
• Archived column on the station table. Default to false.
• Web UI serves up this data along with the full datasets timespan and if the station is archived or not (still producing new data)
• Adjust desktop app to download gaps data as part of the FetchSamples workflow. Detected gaps that line up with data in the gaps file are ignored.
• All data files coming from archived stations should be automatically marked as complete. Gap detection should be skipped and files marked as contiguous.
• SQLite desktop report which shows known gaps only as advertised by the Web UI

Once this, #7 and #8 are done the desktop client should only need to contact the server rarely - mostly for recent data if the app has been closed.

[davidrg]

Mark gaps tool added to admin-tool in commit 88b559e