epsilon merge tool / build-rki-csvs.py: prepare for loss of AGS column #1827
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
when between base and extension a column is lost (when it appears in base, but not in extension) then forward-fill that column into the extension data set, using the last value from the base. this is to accomodate the loss of the column for AGS 16056 in the RKI data set around October 2021 (unclear what happened exactly).
In oct 2021 the AGS 16056 disappeared from the RKI data set. That is, lacking_wrt_ref was not set([11000, 3152]) anymore but set([11000, 3152, 16056]) which is when the program expectedly crashed, leading up to issue #1748. Now, deal with unexpected loss of a column. This is accomodated for in the epsilon merge tool which forward-fills a column when it's in the base but not in the extension.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is to accommodate for the loss of the column for AGS 16056 in the RKI data set around October 2021 (unclear what happened exactly).
See commit messages.
This is to address issue #1748.