Improve logging of the submission validation process

[EvgeniiSkrebtcov]

The enhancement suggested by @Tho-Mat, and based on the discussion in #723

Current Implementation

Issue #723, demonstrated that currently implemented logging is not sufficient for effective troubleshooting of the submission validation process.

Suggested Enhancement

If the kay can not be saved in a DB (e.g. because it is already in database / Date to high / Date to low / unknown), we log this fact, date of the issue and the reason in the application logs.

Expected Benefits

This would have the great advantage of better analyzing certain constellations that can lead to errors.

[Tho-Mat]

Again Risk 6 key is missing in hour-file 2020-09-15-hour-22.zip

[michael-burwig]

Again Risk 6 key is missing in hour-file 2020-09-16-hour-22.zip

@Tho-Mat, thanks for the pointer.
I assume you mean 2020-09-15-hour-22.zip, don't you?

[Tho-Mat]

@michael-burwig
I assume you mean 2020-09-15-hour-22.zip, don't you?
Sorry you are right, i will correct it.

[Tho-Mat]

@michael-burwig
And again Risk key missing in hour-file 2020-09-17-hour-14.zip
This time the use has already date of 18.Sep at 16:00 - 17:00 German time on 17.Sep.
Maybe he/she/it is one day in the future???

[EvgeniiSkrebtcov]

Hello @Tho-Mat ,

Thank you for the additional information. We are investigating this further together with the mobile team.

[Tho-Mat]

@EvgeniiSkrebtcov
and again for 2020-09-19-hour-08.zip

[Tho-Mat]

and again for 2020-09-21-hour-06.zip

[Tho-Mat]

another for
2020-09-21-hour-09.zip

[Tho-Mat]

some new sets arrived at
2020-09-21-hour-17
2020-09-21-hour-20
2020-09-22-hour-00
so in the last 24h at least 5 of 175 users are assumed to have the wrong time.

That looks to much. Maybe the assumption is not correct.
Since only the sets with keys in the far future could be identified, there could be 10 or more with the wrong time.

[ndegendogo]

What exactly is the planned proceeding / timeline for this ticket?

As to my understanding, this is already (fully or only partly?) implemented and merged (PR #783), but I don't (don't yet?) see it mentioned in the release notes for the upcoming release 1.4.
Also I understand that this feature could help in the analysis of missing keys (#723), which is currently only an observed symptom and some guesswork on the reason / root cause, but not yet understood - or am I missing something?

What is the prio to analyse / understand the root cause of the missing keys?

[mibrasap]

Hello @ndegendogo,
we created and merged a new PR #797 where we improved logging for the submission validation. Meanwhile, the release notes for release 1.4 are updated as well.
The planned release date for release 1.4 is 30.09.2020.