Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(encryption): add key manager #1706

Merged
merged 26 commits into from
Feb 1, 2024
Merged

Conversation

Samunroyu
Copy link
Collaborator

@Samunroyu Samunroyu commented Dec 1, 2023

This patch implements KeyProvider to manage encryption key from KMS. The KeyProvider
generates encryption key, IV and key version from KMS at the first launch of Replica
Server. And stores them in a file in a shared directory. After then, the process posts
them to KMS to decrypt key. The key is used to encrypt and decrypt data in Replica
Server.

A new config [pegasus.server]hadoop_kms_url has been introduced to provide the KMS URLs.

[pegasus.server]
+ hadoop_kms_url =

@Samunroyu Samunroyu force-pushed the dev/yjw/kms-key branch 2 times, most recently from 87b53f0 to baa1920 Compare December 6, 2023 02:49
src/utils/env.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.h Outdated Show resolved Hide resolved
src/runtime/security/kms_client.h Outdated Show resolved Hide resolved
src/replica/replication_app_base.cpp Outdated Show resolved Hide resolved
src/replica/replication_app_base.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/test/defaul_key_provider_test.cpp Outdated Show resolved Hide resolved
src/replica/default_key_provider.h Outdated Show resolved Hide resolved
src/runtime/security/replica_kms_info.h Outdated Show resolved Hide resolved
src/runtime/security/replica_kms_info.h Outdated Show resolved Hide resolved
src/runtime/security/replica_kms_info.h Outdated Show resolved Hide resolved
src/runtime/security/replica_kms_info.h Outdated Show resolved Hide resolved
src/runtime/security/kms_client.h Outdated Show resolved Hide resolved
src/replica/replication_app_base.h Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/default_key_provider.h Outdated Show resolved Hide resolved
src/replica/default_key_provider.h Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/runtime/security/replica_kms_info.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.h Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/kms_key_provider.h Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
@Samunroyu Samunroyu force-pushed the dev/yjw/kms-key branch 2 times, most recently from 77005ed to c4c2d97 Compare December 25, 2023 03:52
src/replica/kms_key_provider.h Outdated Show resolved Hide resolved
src/replica/kms_key_provider.h Outdated Show resolved Hide resolved
src/runtime/security/replica_kms_info.h Outdated Show resolved Hide resolved
src/runtime/security/replica_kms_info.h Outdated Show resolved Hide resolved
src/runtime/security/replica_kms_info.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.cpp Outdated Show resolved Hide resolved
src/replica/storage/simple_kv/test/case-000.ini Outdated Show resolved Hide resolved
src/runtime/security/kms_client.h Outdated Show resolved Hide resolved
src/runtime/security/kms_client.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.cpp Outdated Show resolved Hide resolved
src/runtime/security/kms_client.cpp Outdated Show resolved Hide resolved
@Samunroyu Samunroyu force-pushed the dev/yjw/kms-key branch 4 times, most recently from 24176f4 to bf2905d Compare January 10, 2024 07:01
src/replica/storage/simple_kv/test/run.sh Outdated Show resolved Hide resolved
src/replica/replication_app_base.h Outdated Show resolved Hide resolved
src/security/kms_client.cpp Outdated Show resolved Hide resolved
src/security/kms_client.cpp Outdated Show resolved Hide resolved
src/security/kms_client.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/security/kms_client.cpp Outdated Show resolved Hide resolved
src/security/kms_client.cpp Outdated Show resolved Hide resolved
src/security/kms_client.cpp Outdated Show resolved Hide resolved
src/security/kms_client.cpp Outdated Show resolved Hide resolved
src/security/kms_client.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
.github/workflows/lint_and_test_cpp.yaml Show resolved Hide resolved
src/test/function_test/security/test_kms_client.cpp Outdated Show resolved Hide resolved
src/test/function_test/security/test_kms_client.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/replica/replica_stub.cpp Outdated Show resolved Hide resolved
src/test/function_test/security/test_kms_client.cpp Outdated Show resolved Hide resolved
src/test/function_test/security/test_kms_client.cpp Outdated Show resolved Hide resolved
src/test/function_test/security/test_kms_client.cpp Outdated Show resolved Hide resolved
src/test/function_test/security/test_kms_client.cpp Outdated Show resolved Hide resolved
src/test/function_test/security/test_kms_client.cpp Outdated Show resolved Hide resolved
src/security/kms_client.cpp Outdated Show resolved Hide resolved
src/security/kms_client.cpp Outdated Show resolved Hide resolved
@empiredan empiredan merged commit 8fd6a6e into apache:master Feb 1, 2024
84 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build cpp github scripts type/config-change Added or modified configuration that should be noted on release note of new version.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants