Generated files for same chunk differ in binary

[christian-kirschnick]

After running the testdata profile twice, I noticed that the same file for the same chunk differs on byte level, although the file size is identical. This should not happen, as it will cause problems on CDN.

For the index file, it looks like missing sorting is causing the problem, one example:

[23,18,21,22,19,20]

[pithumke]

Bug probably introduced through #96 when changing most List<...>'s to Collection<...>'s. DiagnosisKeyService now returns a sorted List, so to fix this bug, we probably just need to change the code back to using List<...>'s.

[johanneseschrig]

Logic has changed, not relevant anymore

[pithumke]

How is this issue not relevant anymore? IMHO, generated files for the same chunk still shouldn't differ in binary.

[christian-kirschnick]

Since the generation code has changed a lot & Timon and me are working on #203, we will have an eye on this as well. I will reopen this, in case #203 will not fix it.

[christian-kirschnick]

Re-opening, still valid.

[christian-kirschnick]

Changed to sorted list for the TEK Exports:
https://github.com/corona-warn-app/cwa-server/compare/issues/148-binary-diff

Then ran the assembly process twice and picked the 20 hour package for current day.

Unfortunately,
colordiff -y <(xxd 20) <(xxd 20_2)

Still shows some diffs.

[UnchartedBull]

Managed to convince the ZIP Library to create byte-identical archives. The payload signing creates different bytes for the same chunk each time though. Having the signing algorithm create the same bytes for the same chunk would be a security risk, so we can't create byte-identical archives for the same chunk.