Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate dotpg #255

Open
wants to merge 3 commits into
base: master
from

Conversation

@patrobinson
Copy link
Contributor

commented Oct 17, 2018

This is both a proposal and pull request. I propose that for version 2 of StackMaster we should remove dotgpg support. Dotgpg is difficult to use for a couple of reasons, firstly anytime someone joins or leaves our team we have to re-encrypt our secrets. Secondly there's a long running issue where dotgpg will fail if there's an expired key.

Also removing Dotgpg will simplify our release process, which at the moment requires that we build two seperate versions one of which has to be built on a Windows machine.

This PR prints a deprecation warning when the secrets parameter resolver is used and updates the documentation.

patrobinson added some commits Oct 17, 2018

@mipearson
Copy link

left a comment

... propose that for version 2 of StackMaster we should remove dotgpg support

This is the point where I stopped reading and hit "Approve".

@envatopoho
Copy link

left a comment

Cool!

@jacobbednarz

This comment has been minimized.

Copy link
Member

commented Oct 17, 2018

Dotgpg is difficult to use for a couple of reasons, firstly anytime someone joins or leaves our team we have to re-encrypt our secrets.

This is quite a handy feature of GPG in general and I don't think a lack of on/off boarding process should warrant a removal of it. This ensures that even if someone manages to access sensitive data after they have left the team, they cannot decrypt.

... Secondly there's a long running issue where dotgpg will fail if there's an expired key.

Has anyone submitted a PR for this? I see we raised it but no actions?

Also removing Dotgpg will simplify our release process, which at the moment requires that we build two seperate versions one of which has to be built on a Windows machine.

Travis CI now supports Windows CI so this process could be automated based on tagging or master merge if you wanted. Regardless of this PR, I'd propose we move to automating the release process seeing how Rubygems and GitHub releases are both accessible out of the box.

I'm not particularly sold on Parameter Store or 1Password as the only ongoing solution for secrets primarily for the fact they both have usage caveats that dotgpg and local encrypted storage addressed.

@viraptor

This comment has been minimized.

Copy link
Contributor

commented Oct 17, 2018

For a better discussion, could you expand this?

both have usage caveats that dotgpg and local encrypted storage addressed.

Regarding access after someone leaves the team, don't PS and 1Password address this with permissions already? In the first case, AWS account/permissions are dropped. In the second, team membership is revoked.
I think the access revocation is actually better in this situation - if anyone gets access to the encrypted version, they likely get access to the whole git repo, not just the last commit (either via someone's workstation or remote repo access). That means they can forever access the old commits using their old key - we'd need to roll credentials every time people leave to prevent this.

@jacobbednarz

This comment has been minimized.

Copy link
Member

commented Oct 17, 2018

@viraptor The usage caveats aren't anything to do with on/off boarding permissioning, they are related to the integration and service usage. For instance, 1Password CLI is only available for subscription based services and parameter store requires managing KMS + carefully avoiding the request/size limitations. Last POC we did using parameter store for replacing an existing system seen us hitting these limitations when spinning up reasonably sized fleets. Both of these are hurdles to anyone wanting to implement stack master with sensitive data either for themselves or a small team.

In the second, team membership is revoked.

Only if it can phone home 😉 If the machine isn't connected to the internet, this isn't something that happens.

That means they can forever access the old commits using their old key - we'd need to roll credentials every time people leave to prevent this.

Jaz looks after rolling most of our shared credentials and teams should be looking after their own credential management which includes a regular rotation of credentials so this should be mitigated.

@patrobinson

This comment has been minimized.

Copy link
Contributor Author

commented Oct 17, 2018

parameter store requires managing KMS + carefully avoiding the request/size limitations. Last POC we did using parameter store for replacing an existing system seen us hitting these limitations when spinning up reasonably sized fleets.

StackMaster isn't likely to hit the same limitations because you only access KMS and ParameterStore when you create/update the stack, not every time you spin up a new ec2 instance.

I don't see how rotating keys matters, either you do it or you don't, the underlying encryption mechanism doesn't impact that and as Stan points out, 1Password and KMS provide forward secrecy. So I don't see how removing GPG support creates more risk. It's the same risk with the benefit of not having to manage the encryption keys ourselves, which is a net positive to me.

@patrobinson patrobinson added this to In progress in 2.0 release Oct 18, 2018

@patrobinson patrobinson removed this from In progress in 2.0 release Oct 18, 2018

@simpsora

This comment has been minimized.

Copy link
Contributor

commented Nov 5, 2018

👍 to deprecating the dotgpg resolver, but I'd like it if there were some other offline mechanism. Both the 1P and PS resolvers require access to external systems, which isn't ideal.

Is an ejson--based resolver a possibility? Would it have the same issues that the dotgpg one does?

@patrobinson

This comment has been minimized.

Copy link
Contributor Author

commented Nov 5, 2018

@simpsora No, I think with https://github.com/envato/ejson_wrapper we could make that work especially since there are already EJSON ruby bindings.

@stevehodgkiss

This comment has been minimized.

Copy link
Member

commented Nov 5, 2018

Yeah an EJSON secret resolver would be easy enough to write using EJSON wrapper. Access to decrypt secrets could be managed via permissions on the KMS key EJSON wrapper uses (to encrypt EJSON's actual private key).

@patrobinson

This comment has been minimized.

Copy link
Contributor Author

commented Nov 5, 2018

Awesome. Anybody volunteering to write the parameter resolver 😃 ?

@orien

This comment has been minimized.

Copy link
Member

commented May 23, 2019

Is it possible to extract the Dotgpg secret resolver into a separate (and optional) gem for those who may want to continue using it?

@patrobinson

This comment has been minimized.

Copy link
Contributor Author

commented May 24, 2019

@orien that would be possible and is already supported by StackMaster

@orien

This comment has been minimized.

Copy link
Member

commented Jul 18, 2019

I've created envato/stack_master-gpg_parameter_resolver and extracted the secret parameter resolver. Some lightweight tests prove this works fine.

Is the name ok?

@viraptor

This comment has been minimized.

Copy link
Contributor

commented Jul 19, 2019

Just an FYI: there's a new tool being built for file encryption based on SSH keys with nice integration to github keys and much of the complexity of gpg removed. Worth a read: https://docs.google.com/document/d/11yHom20CrsuX8KQJXBBw04s80Unjv8zCg_A7sPAX_9Y/view#
Could be a good replacement once released.

@patrobinson

This comment has been minimized.

Copy link
Contributor Author

commented Jul 19, 2019

@orien looks great!

@stevehodgkiss

This comment has been minimized.

Copy link
Member

commented Jul 19, 2019

Nice one @orien! The name seems ok. I think ideally we would have named this resolver gpg to begin with and used gpg: my_secret rather than secret: my_secret.

Can we remove the secret resolver from StackMaster with a fallback message to install https://github.com/envato/stack_master-gpg_parameter_resolver ?

@patrobinson

This comment has been minimized.

Copy link
Contributor Author

commented Jul 19, 2019

I think that'd be a breaking change and have to wait till 2.0. But we can certainly add here a link to the gem

@patrobinson

This comment has been minimized.

Copy link
Contributor Author

commented Aug 11, 2019

Given there's an alternative offline mechanism (ejson) and GPG is now available via a seperate Gem I am going to merge this unless there are objections.

@orien

orien approved these changes Aug 11, 2019

@@ -211,7 +209,9 @@ into parameters of dependent stacks.

### Secret

Note: This resolver is not supported on Windows, you can instead use the [Parameter Store](#parameter-store).
**DEPRECATED**: This parameters resolver is deprecated and will be removed in version 2.0. You may install the [seperate gem](https://github.com/envato/stack_master-gpg_parameter_resolver) or use [Parameter Store](#parameter-store) or [1Password](#1password-lookup).

This comment has been minimized.

Copy link
@orien

orien Aug 12, 2019

Member

Let's provide the option for EJSON+KMS also.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.