Skip to content

[BEAM-6292] PasswordDecrypter: Delay decryption / Avoid serialization#7339

Closed
mathieublanchard wants to merge 28 commits intoapache:masterfrom
happn-app:beam-6292
Closed

[BEAM-6292] PasswordDecrypter: Delay decryption / Avoid serialization#7339
mathieublanchard wants to merge 28 commits intoapache:masterfrom
happn-app:beam-6292

Conversation

@mathieublanchard
Copy link
Contributor

@mathieublanchard mathieublanchard commented Dec 21, 2018

Currently, the password is decrypted before the serialization of the pipeline and this causes the raw version to be visible to everyone on the staging location.

To avoid this, we delayed the decryption of the password when connecting to the cluster, which ensures that the raw password is never serialized in the pipeline.

n.b. In our case, we use Google KMS to decrypt Cassandra's password


Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- --- --- --- ---
Java Build Status Build Status Build Status Build Status Build Status Build Status Build Status Build Status
Python Build Status --- Build Status
Build Status
Build Status --- --- ---

kennknowles and others added 28 commits December 19, 2018 19:21
This allows for:
- Copying when storage classes and locations differ.
- Copying of larger files (multi-part calls).
- Copying to a bucket with a default customer managed encryption key
(CMEK).

Also adds integration tests for copy and copy_batch.
…est to using PAssert

This fixes a race condition in the tests when writing results to a static map.
We link both the registration invite and the #beam channel invite on the contact
page. Most people clicked on the "Join" link which sent them to the #general
channel after signing up in Slack. They would ask questions there which
bothered some of the ~2000 channel ASF members.

This makes it more obvious that users won't land in the #beam channel after
signing up. It also strategically places the #beam channel link first.
[BEAM-6295] Fix versions in 2.8.0 Java dependencies table
[BEAM-6295] Fix versions in 2.7.0 Java dependencies table
…ator classes in the same package as the class they access
…y reasons, Beam schema support must be explicitly enabled in these sources.
@mathieublanchard mathieublanchard deleted the beam-6292 branch December 23, 2018 12:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants