Skip to content
This repository has been archived by the owner on May 16, 2023. It is now read-only.

Add random key padding #609

Merged
merged 8 commits into from Jun 22, 2020
Merged

Add random key padding #609

merged 8 commits into from Jun 22, 2020

Conversation

pithumke
Copy link
Member

@pithumke pithumke commented Jun 20, 2020

Summary:

Documentation for the random-key-padding-multiplier property from application.yaml:

# The number of keys to save to the DB for every real submitted key.
# Example: If the 'random-key-padding-multiplier' is set to 10, and 5 keys are being submitted,
# then the 5 real submitted keys will be saved to the DB, plus an additional 45 keys with
# random 'key_data'. All properties, besides the 'key_data', of the additional keys will be
# identical to the real key.

Copy link
Member Author

@pithumke pithumke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODOs for tomorrow.

@sonarcloud
Copy link

sonarcloud bot commented Jun 22, 2020

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 0 Security Hotspots to review)
Code Smell A 0 Code Smells

100.0% 100.0% Coverage
0.0% 0.0% Duplication

@pithumke pithumke merged commit 3d58c8f into master Jun 22, 2020
@pithumke pithumke deleted the feature/random-key-padding branch June 22, 2020 08:56
@pithumke pithumke linked an issue Jun 22, 2020 that may be closed by this pull request
pithumke added a commit that referenced this pull request Jun 22, 2020
pithumke added a commit that referenced this pull request Jun 22, 2020
christian-kirschnick added a commit that referenced this pull request Jun 22, 2020
* Introduce new version 1.0.8

* Disable UserDetailsServiceAutoConfiguration (#594)

* Cherry-Pick: Add random key padding (#609)

* Cherry-Pick: Minor code prettification (#612)

Co-authored-by: Johannes Eschrig <johannes.eschrig@sap.com>
Co-authored-by: Pit Humke <pit.humke@sap.com>
christian-kirschnick added a commit that referenced this pull request Jul 10, 2020
* Introduce a new 1.1.0-SNAPSHOT (#465)

* Fix postgres endpoint in README (#467)

* Documentation: Fix broken link in architecture-overview.md (#469)

* Adherence to micrometer naming conventions (#468)

Co-authored-by: Steve BE <5719743+stevesap@users.noreply.github.com>

* alternate postgres address (from container) (#479)

* Mitigate CVE-2020-13692 (#477)

* Add profile documentation to readme (#475)

* Add hostname verification (#473)

* Optimize Docker build with mvn go-offline (#470)

* Fix CVE-2020-13692 in services too (#484)

* S3 compatible storage multithreading (#466)

* Number of bytes should not depend on Locale (#478)

Charset.defaultCharset() depends on the locale
of the JVM and OS. Tests would fail e.g. with
UTF-16 (2 byte per character) encoding. Use US-ASCII
instead to make it obvious we use a 1 byte per
character encoding for test data.

* Add CSP header (#492)

* add file license header checklist item (#491)

* correct dev profile argument (#490)

* Fix grep command (#481)

Without "--", grep interprets search string "-t" as flag

* fix sonarlint-warning in tests (#471)

- replace size-comparison with "hasSameSizeAs"
- remove redundant "isNotNull()"

* Consistently use UTC for LocalDateTime (#488)

* docs for running maven build with services from docker-compose (#494)

* Updating readme (#516)

* Update dependencies (#487)

* remove use of ForkJoinPool in SubmissionController (#399) (#518)

* Fix http client connection pool size (#519)

* Update android app versions (#520)

* Add "ssl-client-verification-verify-hostname" profile (#521)

* Revert "Update android app versions (#520)" (#524)

* Add TLS Ciphers compliant to BSI-TR-02102-2 (#525)

* Restrict TLS Version

* Specify explicit ciphers

* Add ciphers

Co-authored-by: Pit Humke <pithumke@users.noreply.github.com>

* Add stubs for planned documentation (#528)

* Update android app versions (#529)

* remove unnecessary import (#532)

* Restict allowed actuator routes (#540)

* Renaming singing decorator (#539)

Co-authored-by: Steve BE <5719743+stevesap@users.noreply.github.com>

* Set test root log level to 'off' for distribution (#543)

* Add tests to ensure requests are monitored (#544)

Co-authored-by: Christian Kirschnick <christian.kirschnick@sap.com>

* Fix batch counter concurrency issue (#551)

* Enabling roles also for local development & docker setup (#552)

* rename app.coronawarn.server.services.distribution.assembly.structure.directory.decorators package in test to 'decorator' to match main package (#555)

* Change SSL-related profiles to be secure by default (#541)

* Fix connection pool size configuration (#561)

* Updated timestamp format (#566)

* Fix SSL defaults (#564)

* moved details to description property (#558)

* Update log level for invalid TAN syntax check (#553)

* Streamline log level for submissionservice (#550)

* Fix fake delay updates (#570)

* final adjustments for params (#576)

* adjusting log statement (#583)

* adjusting log statement

* code review

* S3 Continuation Token (#572)

* Object Store / S3 Publisher Test (#585)

* add test for upload error

* add ObjectStoreAccess Tests

* Add validator for submissionTimestamp (closes #557) (#573)

* Add validator for submissionTimestamp
* [refactor] extract and reuse buildDiagnosisKey helper methods

* Add additional unit tests (#588)

* Doc: TLS (#571)

* Disable UserDetailsServiceAutoConfiguration (#594)

* nit: corrected typo (#592)

* Declutter SubmissionController (#554)

* DOC: Distribution Service/Spring Profiles (#559)

* DOC: Submission Service\Spring Profiles & Impact (#534)

* use version path parts from application configuration (#596)

* Fix/appconfig validation (#598)

- Fixes bug where app configurations could be written even if validation failed at runtime.
- Adds unit test for AppConfigurationDirectory
- Corrects JavaDoc in AppConfigurationDirectory
- Extracts test helper method
- Refactors AppConfigurationProvider into Configuration that provides ApplicationConfiguration bean

* Use non-root user inside the containers (#597)

* Add missing blanks in warn messages (#599)

* Create runner tests (#604)

* Add unit test for ObjectStoreAccess construction bucketExists check

* wip

* cleanup

* Clean up new lines

* Remove unnecessary public modifiers

Co-authored-by: Michael Burwig <michael.burwig@sap.com>

* replace all CR, LF or CRLF (#591)

* Specify TLS named groups (#603)

* Clearing up possible misunderstanding of full architecture vs cwa server (#605)

* Clearing up possible misunderstanding of full architecture vs cwa server (#605)

* Add random key padding (#609)

* Minor code prettification (#612)

* Minor code quality improvements (#575)

* Decrease max random key multiplier to 25 (#615)

* Add manual Log4J shutdown hook (close #589) (#590)

* Object Store Docs (#560)

* Added distribution

* remove links

* Add assembly

* formatting + spelling

* Cleanup files

* Readd DISTRIBUTION.md

* Update docs/DISTRIBUTION.md

Co-authored-by: Pit Humke <pithumke@users.noreply.github.com>

* Update docs/DISTRIBUTION.md

* pr remarks

* add some information for retention

* linting

* Add info about file naming

* Add minor tweaks

Co-authored-by: MKusber <michael.kusber@sap.com>
Co-authored-by: Christian Kirschnick <christian.kirschnick@sap.com>
Co-authored-by: Pit Humke <pithumke@users.noreply.github.com>
Co-authored-by: Michael Burwig <michael.burwig@sap.com>

* Test for VerificationServiceHealthIndicator, remove null check (#619)

* Fix sonar issues (#623)

* Add unit tests for export.bin.checksum (#621)

* Add unit tests for export.bin.checksum

* add license file header

* fix code smell

* Moved non-static import, use of helper for DiagnosisKey

* Make use of AssertJ dedicated methods

* Remove unused import

Co-authored-by: Kevin Pontes <kevin.pontes@sap.com>

* Fix/mime type (#627)

* depending on the s3Key we identify the mime type

608: Index Files: Incorrect mime type  - #608

* get the content type from the localfile and add to headers map

608: Index Files: Incorrect mime type  - #608

* forward the added CONTENT_TYPE header to the s3Client (requestBuilder)

608: Index Files: Incorrect mime type  - #608

* removed keyword: 'final'

608: Index Files: Incorrect mime type

Task-Url: #608

* removed dependency on awssdk

608: Index Files: Incorrect mime type

Task-Url: #608

* Use native query for retention policy (#628)

* #556: Performance Improvement: Disable spring.jpa.open-in-view in (#632)

submission service

Task-Url: #556

* Config/adjust min android version (#633)

* Update app-version-config.yaml

Adjust min android version

* Update app-version-config.yaml

* OSP is OpenStack Platform, OpenShift is OCP (#625)

* Mitigate CVE-2020-11996 (#635)

* Integration Testing on CircleCI (#580)

* failsafe

* add failsafe int test to circle ci

* Docker object store on circle ci

Co-authored-by: Christian Kirschnick <christian.kirschnick@sap.com>

* Enable Docker Layer Caching on CircleCi (#639)

* Submission Svc: Timeout handling  (#602)

* introduce spring transaction default timeout
* introduce feign client timeouts
* add timeout test

* Simplifying DateAggregatingDecorator (#629)

* Create unit test for DiagnosisKeysHourDirectory.

* Add getDistributionTime function to DiagnosisKeyBundler.

* Add tests to DiagnosisKeysDateDirectory and improve HourDirectoryTests.

* Remove DateAggregatingDecorator.

* Add license header.

* Remove unnecessary public visibility for DiagnosisKeysHourDirectoryTest.

* Use native query for retention policy (#628)

* Code formatting.

* Split parameterized test for DiagnosisKeysDateDirectory into separate tests.

* Split parameterized test for DiagnosisKeysHourDirectory into separate tests.

* Remove null value return when the associated date is current distribution date.

* Remove null value return when the associated date is current distribution date.

* Remove TestTuple class.

* Made NilDirectory public and test coverage.

* Made NilDirectory public and test coverage.

* Made NilDirectory public and test coverage.

* Use optional instead of NilDirectory.

* Add minor formatting tweaks

Co-authored-by: KevponSAP <66735382+KevponSAP@users.noreply.github.com>
Co-authored-by: Pit Humke <pithumke@users.noreply.github.com>
Co-authored-by: Steve BE <5719743+stevesap@users.noreply.github.com>
Co-authored-by: Michael Burwig <michael.burwig@sap.com>

* Revert "Consistently use UTC for LocalDateTime (#488)" (#641)

This reverts commit ec099f3.

* switch to spring-2.3.1 (#574)

* switch to spring-2.3.1
* remove CVE-2020-11996 mitigation 
is now resolved by upgrade to spring-2.3.1

* Use maven wrapper as the single source of truth for maven version in the project (#194)

* use maven wrapper as a single source of truth for maven version
* Use ./mvnw instead of mvn
* -set JAVA_HOME variable to jdk 11
* - set export JAVA_HOME without sudo
* - set in the SonarCloud analysis the JAVA_HOME
* - add line for consistency
* - change to use mvnw in docker-compose-it file
* force mvnw script and properties file to use LF eol for windows machines

* Include date index file generation for current date in demo profile. (#643)

* fix javadoc generation errors (#646)

* Update codeowners file (#645)

* Fix client SSL properties (#648)

* Replace data-jpa with data-jdbc (#631)

* Replace data-jpa with data-jdbc
* Remove default constructor from DiagnosisKey
* Clarify test.database.replace statement

* downscale gracefully (#647)

* Failing requests in down-scaling scenario

closes #547

Following
https://dzone.com/articles/graceful-shutdown-spring-boot-applications

We let the tomcat finish it's work (send open async responses -
DeferredResult), before it's terminated (SIGTERM coming from
kubernetes).

* #589 Log4j improper shutdown

same thing as in distribution service

* checkstyle complains about missing javadoc

why is it doing that for this method but not for the others without
javadoc?!?! bug in checkstyle?

* ignore null executors

#547

* add missing file comment 'license'

#547

* fix issues found by sonar

#547

* let's use the spring framework solution

#547

https://dzone.com/articles/configuring-graceful-shutdown-readiness-and-livene

* prefer `DisposableBean` over `@PreDestroy`

#589

* use k8s setting `terminationGracePeriodSeconds`

#547

* Spring Cloud Vault Integration (#624)

* Remove missing .yaml attribute validation test (#652)

* Add dependecies for javadoc generation (#649)

* Move dependency one level up (#654)

* Update with links to cwa-server documentation and JavaDoc (#655)

* Fix javadoc search 404 result issue (#657)

* Fix/attenuation value range (#642)

* Change value range lt to le

* Change parameter lt_10_dbm to le_10_dbm in yaml-files, add unit test for naming mismatch, fix unit test path for broken syntax

* Removed unused enum entry, refactor test

Co-authored-by: Kevin Pontes <kevin.pontes@sap.com>
Co-authored-by: Pit Humke <pithumke@users.noreply.github.com>

* Addition to CODEOWNERS.md (#653)

* Addition to codeowners

* Specify entire teams in codeowners file

* Revert "Specify entire teams in codeowners file"

This reverts commit e101f43.

* Added Frederico to  codeowners.md

Co-authored-by: Michael Burwig <michael.burwig@sap.com>

Co-authored-by: Ole Lilienthal <ole.lilienthal@sap.com>
Co-authored-by: Jan Sievers <jan.sievers@sap.com>
Co-authored-by: selectAll <48492067+selectAll@users.noreply.github.com>
Co-authored-by: Steve BE <5719743+stevesap@users.noreply.github.com>
Co-authored-by: Pit Humke <pithumke@users.noreply.github.com>
Co-authored-by: Timon G <timon.gaebelein@icloud.com>
Co-authored-by: MKusber <michael.kusber@sap.com>
Co-authored-by: Walter S <720628+WalterS@users.noreply.github.com>
Co-authored-by: Nils Knappmier (work account) <54022132+nils4cosee@users.noreply.github.com>
Co-authored-by: Christian Kirschnick <christian.kirschnick@sap.com>
Co-authored-by: Michael Burwig <michael.burwig@sap.com>
Co-authored-by: Paul-Christian Volkmer <pcvolkmer@users.noreply.github.com>
Co-authored-by: Roman Neß <Kaossilat0r.rn@gmail.com>
Co-authored-by: Matthias Wessendorf <mwessend@redhat.com>
Co-authored-by: Stefan Weil <sw@weilnetz.de>
Co-authored-by: Björn Wilmsmann <bjoern@bjoernkw.com>
Co-authored-by: Roberto Duessmann <roberto.duessmann@gmail.com>
Co-authored-by: Michael <66735191+mibrasap@users.noreply.github.com>
Co-authored-by: Kevin Pontes <kevin.pontes@sap.com>
Co-authored-by: Hilmar Falkenberg <hilmarf@googlemail.com>
Co-authored-by: KevponSAP <66735382+KevponSAP@users.noreply.github.com>
Co-authored-by: Hilmar Falkenberg <hilmar.falkenberg@sap.com>
Co-authored-by: Christopher Schmitz <64647277+ChristopherSchmitz@users.noreply.github.com>
Co-authored-by: dabelenda <diego.abelenda@camptocamp.com>
Co-authored-by: Sorin Stefan Iovita <sorin.stefan.iovita@sap.com>
Co-authored-by: Michael Risthaus <mristhaus@gmail.com>
Co-authored-by: Driptaroop Das <driptaroop.das@gmail.com>
Co-authored-by: ioangut <67064882+ioangut@users.noreply.github.com>
Co-authored-by: EugenM-SAP <67458405+EugenM-SAP@users.noreply.github.com>
@pinkit-cwa
Copy link

pinkit-cwa commented Aug 17, 2020

@pithumke Is there a specific reason why you chose to just add random TEKs for every TEK instead of rounding up the number of TEKs to the next 10 or maybe 32?
I am not into statistics, but these guys seem to be, and comment:

For some reason the German server publishes 10 keys for every one really uploaded. (See this github issue.) I don't really buy that as a privacy win TBH - just rounding up to a multiple of 10 would be fine, but at least I think I now understand the numbers.

The discussion for the issue "just" requested to add random TEKs in case there is a small number of TEKs (less than 10 / round up to at least 10 TEKs).

@pithumke
Copy link
Member Author

Hello @pinkit-cwa,
as outlined in this comment and this comment, we chose this approach, because we want to avoid DPP problems arising from statistical analysis of random TEKs while still keeping the 140 threshold required by the RKI (but reaching it faster). Simply duplicating all real keys N times and only forging the key_data portion, avoids this potential problem, was quick and easy to implement and was signed off by the RKI. If we were to instead "round up" the total number of TEKs, we would need to randomly generate full TEKs from scratch (including the transmission risk level and the rolling start interval number), potentially risking those fake keys to be identifiable by statistical analysis.

# then the 5 real submitted keys will be saved to the DB, plus an additional 45 keys with
# random 'key_data'. All properties, besides the 'key_data', of the additional keys will be
# identical to the real key.
random-key-padding-multiplier: ${RANDOM_KEY_PADDING_MULTIPLIER:1}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ChristopherSchmitz - luckily Pit made this padding already configurable and the default is only 1. So as long as security/DPP has no other concerns, we can simply change the value, if required.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Patch Proposals 1.0.8
5 participants