Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Yandex Cloud KMS #847

Merged
merged 5 commits into from Jan 9, 2021
Merged

Support for Yandex Cloud KMS #847

merged 5 commits into from Jan 9, 2021

Conversation

luc-lynx
Copy link
Contributor

@luc-lynx luc-lynx commented Jan 8, 2021

This PR introduces support for Yandex Cloud KMS.

Some notes:

  • Yandex Cloud SDK doesn't have credentials discovery like AWS SDK, so I implemented an approach when a user either passes a file name with credentials or relies on wal-g attempt to get an access token from virtual machine metadata service. AWS SDK uses similar approach.
  • AES-256 is used to encrypt Data Encryption Key.
  • In contrast to AWS KMS, Data Encryption Key is generated by Yandex Cloud KMS.
  • Tests use a mocked version of key and don't make api calls to Yandex Cloud KMS. AWS KMS support uses the same approach. However in my opinion it's better to come up with tests for AWS KMS and YC KMS.

Copy link
Collaborator

@x4m x4m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks!

@x4m
Copy link
Collaborator

x4m commented Jan 9, 2021

Is the operation of obtaining key time-consuming?
I mean we are doing wal-push in PG for each 16Mb file. Will not reading key be a bottleneck?

@luc-lynx
Copy link
Contributor Author

luc-lynx commented Jan 9, 2021

Is the operation of obtaining key time-consuming?

Just an API call to YC KMS. In case of AWS KMS it also requires one API call. On the KMS side it's just one encryption operation, so it shouldn't be slow.

I mean we are doing wal-push in PG for each 16Mb file.

Does it mean that wal-g binary is executed on every 16Mb chunk?

@x4m
Copy link
Collaborator

x4m commented Jan 9, 2021

Does it mean that wal-g binary is executed on every 16Mb chunk?
Yes

@x4m x4m merged commit c310471 into wal-g:master Jan 9, 2021
@x4m
Copy link
Collaborator

x4m commented Jan 9, 2021

Thanks, pushed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants