New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add encrypted secrets #28038

Merged
merged 41 commits into from Feb 23, 2017

Conversation

@kaspth
Member

kaspth commented Feb 16, 2017

Fixes #25095

Adds config/secrets.yml.enc using Sekrets to allow storing production app secrets right in source control. When running bin/rails secrets:edit the file is decrypted and opened in a tmp file that then gets destroyed after the file is encrypted again and written back to its original location.

Differs sligthly in the proposed first-call experience in that we generate a key instead of asking for it. Also supports storing the key in config/secrets.yml.key instead of always going the RAILS_MASTER_KEY.

Pending:

  • Tests.
  • Restricting rails s boot when no decryption key is present.
  • Merging encrypted into the standard secrets after decryption.

@kaspth kaspth added the railties label Feb 16, 2017

@kaspth kaspth added this to the 5.1.0 milestone Feb 16, 2017

@kaspth kaspth self-assigned this Feb 16, 2017

@kaspth kaspth requested a review from dhh Feb 16, 2017

Show outdated Hide outdated railties/lib/rails/commands/secrets/edit_command.rb
require "active_support/core_ext/string/strip"
require "rails/generators/rails/app/app_generator"
require "rails/engine"

This comment has been minimized.

@kaspth

kaspth Feb 16, 2017

Member

Sekrets depends on Rails::Engine out of the box.

@kaspth

kaspth Feb 16, 2017

Member

Sekrets depends on Rails::Engine out of the box.

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Feb 16, 2017

Member

Later I'd like to inline just the parts of Sekrets we need, since it seems to a lot more than we require as well as pull in a number of dependencies (nothing wrong with that, but it's best Rails keeps slim).

@ahoward how would you feel about that? We'd inline the parts we're using into it's own private namespace and give you credit with: # Adapted from Ara T. Howard's Sekrets gem..

Also: as a heads up, it seems Sekrets isn't distributed with a license :)

Member

kaspth commented Feb 16, 2017

Later I'd like to inline just the parts of Sekrets we need, since it seems to a lot more than we require as well as pull in a number of dependencies (nothing wrong with that, but it's best Rails keeps slim).

@ahoward how would you feel about that? We'd inline the parts we're using into it's own private namespace and give you credit with: # Adapted from Ara T. Howard's Sekrets gem..

Also: as a heads up, it seems Sekrets isn't distributed with a license :)

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Feb 16, 2017

Member

Dang, I've got trouble getting https://github.com/kaspth/rails/blob/226db6a15bc5ce5aca1854d7d2a00ab221795f86/railties/lib/rails/application.rb#L388 to recognize config/secrets.yml.enc. config.paths don't seem to be too willing to cooperate, I imagine I need to add a glob to it somehow.

Member

kaspth commented Feb 16, 2017

Dang, I've got trouble getting https://github.com/kaspth/rails/blob/226db6a15bc5ce5aca1854d7d2a00ab221795f86/railties/lib/rails/application.rb#L388 to recognize config/secrets.yml.enc. config.paths don't seem to be too willing to cooperate, I imagine I need to add a glob to it somehow.

@maclover7

Overall makes sense to me, nice work @kaspth. Added a few grammar comments.

Assuming we should add some documentation about this new feature?

Re config.paths, I think we just need to change this line.

Show outdated Hide outdated railties/lib/rails/commands/secrets/USAGE
`RAILS_MASTER_KEY` environment variable, with the former taking precedence.
By default Rails adds `config/secrets.yml.key` and `config/secrets.yml.enc` to
`.gitignore`. If you don't use Git add these to the respective version control

This comment has been minimized.

@maclover7

maclover7 Feb 16, 2017

Member

use Git, add

@maclover7

maclover7 Feb 16, 2017

Member

use Git, add

Show outdated Hide outdated railties/lib/rails/commands/secrets/USAGE
Rails looks for the decryption key in config/secrets.yml.key as well as the
`RAILS_MASTER_KEY` environment variable, with the former taking precedence.
By default Rails adds `config/secrets.yml.key` and `config/secrets.yml.enc` to

This comment has been minimized.

@maclover7

maclover7 Feb 16, 2017

Member

default, Rails

@maclover7

maclover7 Feb 16, 2017

Member

default, Rails

Show outdated Hide outdated railties/lib/rails/commands/secrets/USAGE
to edit production secrets.
When the temporary file is next saved the contents are encrypted and written to
config/secrets.yml.enc while the file itself is destroyed to protect secrets

This comment has been minimized.

@maclover7

maclover7 Feb 16, 2017

Member

secrets from leaking.

@maclover7

maclover7 Feb 16, 2017

Member

secrets from leaking.

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Feb 17, 2017

Member

I don't understand why config/secrets.yml.enc would be gitignored? Isn't the whole point to check the encrypted secrets into the repo, but keeping the key out of the repo? So think we should remove that file from gitignore, leaving only the key file there.

I'm also hesitant to generating this key upfront. It's pretty easy for someone then starting a new project to use this feature but forget to do something about the key file, like commit it to 1Password. If they forget that, then change computers, the encrypted data is inaccessible. IMO, better that there's a kick-off for the first time you use encrypted secrets that explains that we're generating the key, putting it there, and you should store it securely somewhere outside of the repo.

Otherwise I'm liking the simplicity of this a lot!

Member

dhh commented Feb 17, 2017

I don't understand why config/secrets.yml.enc would be gitignored? Isn't the whole point to check the encrypted secrets into the repo, but keeping the key out of the repo? So think we should remove that file from gitignore, leaving only the key file there.

I'm also hesitant to generating this key upfront. It's pretty easy for someone then starting a new project to use this feature but forget to do something about the key file, like commit it to 1Password. If they forget that, then change computers, the encrypted data is inaccessible. IMO, better that there's a kick-off for the first time you use encrypted secrets that explains that we're generating the key, putting it there, and you should store it securely somewhere outside of the repo.

Otherwise I'm liking the simplicity of this a lot!

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Feb 17, 2017

Member

@jeremy Could you review as well?

Member

dhh commented Feb 17, 2017

@jeremy Could you review as well?

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Feb 17, 2017

Member

So I'd add bin/rails secrets:new or something like that for the initialization. Could also just combine the two steps and have bin/rails secrets, which would initialize if secrets.yml.enc is missing, and otherwise go to the edit session.

Member

dhh commented Feb 17, 2017

So I'd add bin/rails secrets:new or something like that for the initialization. Could also just combine the two steps and have bin/rails secrets, which would initialize if secrets.yml.enc is missing, and otherwise go to the edit session.

@dhh dhh requested a review from jeremy Feb 17, 2017

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Feb 18, 2017

Member

I don't understand why config/secrets.yml.enc would be gitignored?

It shouldn't! That was me being a dummy, thanks 👍

Member

kaspth commented Feb 18, 2017

I don't understand why config/secrets.yml.enc would be gitignored?

It shouldn't! That was me being a dummy, thanks 👍

Show outdated Hide outdated Gemfile.lock
fattr (>= 2.2.1)
highline (>= 1.6.15)
main (>= 5.1.1)
map (>= 6.3.0)

This comment has been minimized.

@matthewd

matthewd Feb 18, 2017

Member

That's a lot of deps 😐

@matthewd

matthewd Feb 18, 2017

Member

That's a lot of deps 😐

This comment has been minimized.

@simi

simi Feb 18, 2017

Contributor

I was thinking the same.

I'm not sure if this is really worth it to integrate since you can do almost all of this as described in https://github.com/ahoward/sekrets/blob/master/README when you need this feature. I think the only difference is key generation (where Sekrets will ask you for key). That can be solved in Sekrets and I think it will be enough to update guides to recommend this approach for storing secret data within project.

But maybe I'm missing something.

@simi

simi Feb 18, 2017

Contributor

I was thinking the same.

I'm not sure if this is really worth it to integrate since you can do almost all of this as described in https://github.com/ahoward/sekrets/blob/master/README when you need this feature. I think the only difference is key generation (where Sekrets will ask you for key). That can be solved in Sekrets and I think it will be enough to update guides to recommend this approach for storing secret data within project.

But maybe I'm missing something.

This comment has been minimized.

@kaspth

kaspth Feb 18, 2017

Member

Agreed: #28038 (comment) 😊

@kaspth
@@ -80,7 +80,7 @@ def paths
@paths ||= begin
paths = super
paths.add "config/database", with: "config/database.yml"
paths.add "config/secrets", with: "config/secrets.yml"
paths.add "config/secrets", with: "config", glob: "secrets.yml{,.enc}"

This comment has been minimized.

@kaspth

kaspth Feb 18, 2017

Member

@maclover7 thanks for the tip!

@kaspth

kaspth Feb 18, 2017

Member

@maclover7 thanks for the tip!

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Feb 18, 2017

Member

We could potentially try to remove the need for the config/secrets.yml.enc file entirely and just enable encryption on string keys in config/secrets.yml.

So when reading config/secrets.yml we traverse it and any string values starting with a signature ala this we decrypt. Then on write we just encrypt every string value (and prefix the signature).

Member

kaspth commented Feb 18, 2017

We could potentially try to remove the need for the config/secrets.yml.enc file entirely and just enable encryption on string keys in config/secrets.yml.

So when reading config/secrets.yml we traverse it and any string values starting with a signature ala this we decrypt. Then on write we just encrypt every string value (and prefix the signature).

kaspth added some commits Feb 15, 2017

Use $EDITOR and await edits.
Adds support for atom and other editors with open commands that
return immediately.
Inline ignore method.
Couldn't be called without moving to the generator. Prefer duplication.
Run generator when executing secrets:edit.
Adds a more pleasant first boot experience where the key and file
plus ignoring them is set up if they're not present.
Revise secrets flow.
* Add support for immediately exiting editors such as atom
  (though done crudely with Time.now), while keeping vim etc. happy.
* Prefill content if the file has none (in time, I'd like to inline sekrets
  with proper attribution and making it easier to do this prefill before the editing).
* Add puts statement to guide clue the user in to what's happening.
Add Rails::Secrets.
Helps parsing yml files and wraps Sekrets to make it easier to remove later.
Don't ignore the encrypted secrets.
The whole point of this exercise is to add it to version control! 😇
Poof! …and then sekrets was gone.
Removes sekrets as a dependency but leaves an attribution in its place.
Use a wonky way to find the right root.
`Rails` isn't guaranteed to respond to root in tests.
Ensure key stays within the cipher's key_len.
Ruby 2.4 enforces a specific key length. Handle this restriction by generating
correct keys from the get go.

Also skip the needless key digest (it's already hex'ed), and adopt
ActiveSupport::MessageEncryptor's aes-256-cbc encryption mode.
Test new and `app:update`ing apps encrypted secrets flow.
The config/secrets.yml, config/secrets.yml.key, config/secrets.yml.enc
files should be there.

Expect .key to contain a suitably hex'y key.

Expect .enc to be encrypted, the inspected contents are only there after
reading and *then* decrypting. Not just at read time.

Finally, expect .gitignore to not contain the generated key file.
Show outdated Hide outdated railties/lib/rails/commands/secrets/secrets_command.rb
Rails::Generators::AppGenerator.new([ Rails.root ]).setup_encrypted_secrets
end
def edit

This comment has been minimized.

@kaspth

kaspth Feb 19, 2017

Member

Neatness: this change also spills into improvements to the command infrastructure.

Previously we couldn't handle bin/rails secrets, bin/rails secrets:setup and bin/rails secrets:edit in one command file. But with some "smarter" lookup we can 💪

(I say "smarter" because the command lookups could use some simplifying but that's too much for this PR. So pour more coal into this 🚂, less changing lanes.)

@kaspth

kaspth Feb 19, 2017

Member

Neatness: this change also spills into improvements to the command infrastructure.

Previously we couldn't handle bin/rails secrets, bin/rails secrets:setup and bin/rails secrets:edit in one command file. But with some "smarter" lookup we can 💪

(I say "smarter" because the command lookups could use some simplifying but that's too much for this PR. So pour more coal into this 🚂, less changing lanes.)

kaspth added some commits Feb 19, 2017

Split secrets:edit into secrets:setup and secrets:edit.
Also add a general secrets command that does setup first, then
edits.

Required some more finagling to the command infrastructure.
Move encrypted secrets to own generator.
Allows us to much more easily explain what's happening.

Also makes this an manual opt-in for updating apps.
@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Feb 19, 2017

Member

Split out the secrets:edit setup parts into a secrets:setup command. Here's what it looks like:

screen shot 2017-02-19 at 20 52 44

And if you're not using Git:

screen shot 2017-02-19 at 20 54 25

I think we could try to match other version control ignore files via /^.\w+ignore, but I don't know how big of deal that support would be.

Member

kaspth commented Feb 19, 2017

Split out the secrets:edit setup parts into a secrets:setup command. Here's what it looks like:

screen shot 2017-02-19 at 20 52 44

And if you're not using Git:

screen shot 2017-02-19 at 20 54 25

I think we could try to match other version control ignore files via /^.\w+ignore, but I don't know how big of deal that support would be.

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Feb 19, 2017

Member

The extra explanation now doesn't read too great on rails new though, so I wonder if we should provide a shush mode:

      create  config/initializers/cors.rb
      create  config/initializers/filter_parameter_logging.rb
      create  config/initializers/inflections.rb
      create  config/initializers/mime_types.rb
      create  config/initializers/new_framework_defaults.rb
      create  config/initializers/wrap_parameters.rb
      create  config/locales
      create  config/locales/en.yml
Adding config/secrets.yml.key to store the encryption key: 2339a6ec44ddb46c2333b0ba3fd1c68c

Save this in a password manager your team can access.

If you lose the key, no one, including you, can access any encrypted secrets.

      create  config/secrets.yml.key

Ignoring config/secrets.yml.key so it won't end up in Git history:

      append  .gitignore

Adding config/secrets.yml.enc to store secrets that needs to be encrypted.

      create  config/secrets.yml.enc

For now the file contains this but it's been encrypted with the generated key:

production:
  secret_key_base: da84a3f26dc5ad272160fc632f83c2c976eb5de26bbe1ff7a757a5a9f9d1f1711c3eb8f2988ec841c8dc4bf45f1ef8e11c99bb3dbc3f3612a9a1abc812f306d3

You can edit encrypted secrets with `bin/rails secrets:edit`.
      create  config/boot.rb
      create  config/database.yml
      create  db
      create  db/seeds.rb
      create  lib
Member

kaspth commented Feb 19, 2017

The extra explanation now doesn't read too great on rails new though, so I wonder if we should provide a shush mode:

      create  config/initializers/cors.rb
      create  config/initializers/filter_parameter_logging.rb
      create  config/initializers/inflections.rb
      create  config/initializers/mime_types.rb
      create  config/initializers/new_framework_defaults.rb
      create  config/initializers/wrap_parameters.rb
      create  config/locales
      create  config/locales/en.yml
Adding config/secrets.yml.key to store the encryption key: 2339a6ec44ddb46c2333b0ba3fd1c68c

Save this in a password manager your team can access.

If you lose the key, no one, including you, can access any encrypted secrets.

      create  config/secrets.yml.key

Ignoring config/secrets.yml.key so it won't end up in Git history:

      append  .gitignore

Adding config/secrets.yml.enc to store secrets that needs to be encrypted.

      create  config/secrets.yml.enc

For now the file contains this but it's been encrypted with the generated key:

production:
  secret_key_base: da84a3f26dc5ad272160fc632f83c2c976eb5de26bbe1ff7a757a5a9f9d1f1711c3eb8f2988ec841c8dc4bf45f1ef8e11c99bb3dbc3f3612a9a1abc812f306d3

You can edit encrypted secrets with `bin/rails secrets:edit`.
      create  config/boot.rb
      create  config/database.yml
      create  db
      create  db/seeds.rb
      create  lib

kaspth added some commits Feb 22, 2017

Fix test broken after config rename.
When renaming the secrets config to read_encrypted_secrets.

The frameworks tests break because the remove the config/environments
files, which included production.rb that had the flag enabled.

As such some of the tests couldn't find a secret_key_base.

They also couldn't a key base because the we no longer generate a secret
key base in the production environment in secrets.yml.

Fix this by assigning a key base to our config.
Assign read_encrypted_secrets up front.
Spares the mangling in the test.
Simplify the file writes.
Skip teh blocks.
Don't disrupt the existing Heroku workflow.
People can use encrypted secrets but that's a separate action,
later.
@rafaelfranca

👏

@dhh dhh merged commit 1166094 into rails:master Feb 23, 2017

1 of 2 checks passed

continuous-integration/travis-ci/pr The Travis CI build failed
Details
codeclimate no new or fixed issues
Details

@kaspth kaspth deleted the kaspth:encrypted-secrets branch Feb 23, 2017

@simi

This comment has been minimized.

Show comment
Hide comment
@simi

simi Feb 23, 2017

Contributor

Will be nice to add this to CHANGELOG also.

Contributor

simi commented Feb 23, 2017

Will be nice to add this to CHANGELOG also.

@drnic

This comment has been minimized.

Show comment
Hide comment
@drnic

drnic Feb 24, 2017

Contributor

Is production.secret_key_base value relevant if new encrypted secrets is enabled? Is it the same or diff to RAILS_MASTER_KEY?

If we run rake secrets:setup can we replace/delete config/secrets.yml to avoid ppl using it?

Contributor

drnic commented Feb 24, 2017

Is production.secret_key_base value relevant if new encrypted secrets is enabled? Is it the same or diff to RAILS_MASTER_KEY?

If we run rake secrets:setup can we replace/delete config/secrets.yml to avoid ppl using it?

@drnic

This comment has been minimized.

Show comment
Hide comment
Contributor

drnic commented Feb 24, 2017

def generate_key
cipher = new_cipher
SecureRandom.hex(cipher.key_len)[0, cipher.key_len]

This comment has been minimized.

@emboss

emboss Feb 24, 2017

This is limiting the key's entropy. The output of

SecureRandom.hex(n).size # => 2n

is twice as long because n is telling SecureRandom "give me an output that is equivalent to n random bytes". Hex encoding encodes every byte as two characters, so the resulting hex string would be twice as long.

For the default AES-256 n = 32, and by cutting the 64 hex bytes down to 32 again, the entropy in that value is now only equivalent to 16 random bytes anymore. This makes brute forcing the key much easier: Instead of 2^256 now there are only 2^128 possible keys.

I would suggest simply using the full output of SecureRandom.hex(cipher.key_len) and decoding the resulting 64 hex bytes when the key is read from the file, yielding the 32-byte binary key.

@emboss

emboss Feb 24, 2017

This is limiting the key's entropy. The output of

SecureRandom.hex(n).size # => 2n

is twice as long because n is telling SecureRandom "give me an output that is equivalent to n random bytes". Hex encoding encodes every byte as two characters, so the resulting hex string would be twice as long.

For the default AES-256 n = 32, and by cutting the 64 hex bytes down to 32 again, the entropy in that value is now only equivalent to 16 random bytes anymore. This makes brute forcing the key much easier: Instead of 2^256 now there are only 2^128 possible keys.

I would suggest simply using the full output of SecureRandom.hex(cipher.key_len) and decoding the resulting 64 hex bytes when the key is read from the file, yielding the 32-byte binary key.

This comment has been minimized.

@nickmalcolm

nickmalcolm May 3, 2017

Contributor

Looks like this has been addressed: #28139

@nickmalcolm

nickmalcolm May 3, 2017

Contributor

Looks like this has been addressed: #28139

end
def new_cipher
OpenSSL::Cipher.new("aes-256-cbc")

This comment has been minimized.

@emboss

emboss Feb 24, 2017

It would be preferable to use an AEAD cipher mode like AES-GCM that also guarantees the integrity of the resulting ciphertext. As a fallback when the underlying OpenSSL version does not support GCM yet, "Encrypt-then-Mac" could be used. ActiveSupport::MessageEncryptor already supports this with its #encrypt_and_sign method - would it be possible to reuse it in this context?

Let me know what you think, I'd be happy to help.

@emboss

emboss Feb 24, 2017

It would be preferable to use an AEAD cipher mode like AES-GCM that also guarantees the integrity of the resulting ciphertext. As a fallback when the underlying OpenSSL version does not support GCM yet, "Encrypt-then-Mac" could be used. ActiveSupport::MessageEncryptor already supports this with its #encrypt_and_sign method - would it be possible to reuse it in this context?

Let me know what you think, I'd be happy to help.

This comment has been minimized.

@morgoth

morgoth Feb 24, 2017

Member

@emboss you might be interested in #28139

@morgoth

morgoth Feb 24, 2017

Member

@emboss you might be interested in #28139

This comment has been minimized.

@emboss

emboss Feb 24, 2017

@morgoth OK, it's already being discussed, that's great, thank you for the hint!

@emboss

emboss Feb 24, 2017

@morgoth OK, it's already being discussed, that's great, thank you for the hint!

@coderanger

This comment has been minimized.

Show comment
Hide comment
@coderanger

coderanger Feb 26, 2017

Sorry to sealion in here but someone referred me to this PR from Twitter and it seems like you've set things up to use only a single key. This will likely make key rotations in production environments either very tricky or impossible. I would ask if you have the time before the feature goes live to make it possible to keep multiple copies of encrypted settings such that new code and new keys don't have to be updated in perfect lockstep.

Also I would recommend warning people about the dangers of storing the key in an environment variable. It does make life easier for PaaS users but caveat emptor. Tools like Airbrake and Sentry will need to be updated ASAP to not automatically log the environment variable (and configuration for each should be used in the interim). Also storing a key in an environment variable means it is passed to all subprocesses of Rails, which can be an issue for things like ImageMagick for thumbnail generation.

coderanger commented Feb 26, 2017

Sorry to sealion in here but someone referred me to this PR from Twitter and it seems like you've set things up to use only a single key. This will likely make key rotations in production environments either very tricky or impossible. I would ask if you have the time before the feature goes live to make it possible to keep multiple copies of encrypted settings such that new code and new keys don't have to be updated in perfect lockstep.

Also I would recommend warning people about the dangers of storing the key in an environment variable. It does make life easier for PaaS users but caveat emptor. Tools like Airbrake and Sentry will need to be updated ASAP to not automatically log the environment variable (and configuration for each should be used in the interim). Also storing a key in an environment variable means it is passed to all subprocesses of Rails, which can be an issue for things like ImageMagick for thumbnail generation.

@schneems

This comment has been minimized.

Show comment
Hide comment
@schneems

schneems Feb 26, 2017

Member

Key rotation would be amazing. We don't have that for secret_key_base now do we?

Member

schneems commented Feb 26, 2017

Key rotation would be amazing. We don't have that for secret_key_base now do we?

@schneems

This comment has been minimized.

Show comment
Hide comment
@schneems

schneems Feb 26, 2017

Member

Also rollbar and sentry AFAIK use env var whitelisting so they won't need to be updated. If airbrake does something different then it has bigger problems than this PR.

Member

schneems commented Feb 26, 2017

Also rollbar and sentry AFAIK use env var whitelisting so they won't need to be updated. If airbrake does something different then it has bigger problems than this PR.

@coderanger

This comment has been minimized.

Show comment
Hide comment
@coderanger

coderanger Feb 26, 2017

Probably the best that can be done for rotation for the moment is to rework the filename structure so rather than a single secrets.yml.enc make room for multiple files. Once authenticated encryption of one type or another gets added, the system can then be improved to try all of them in order until one works. But having the folder structure in place now will make the transition easier even if the rotation system is added after initial release.

coderanger commented Feb 26, 2017

Probably the best that can be done for rotation for the moment is to rework the filename structure so rather than a single secrets.yml.enc make room for multiple files. Once authenticated encryption of one type or another gets added, the system can then be improved to try all of them in order until one works. But having the folder structure in place now will make the transition easier even if the rotation system is added after initial release.

@ahoward

This comment has been minimized.

Show comment
Hide comment
@ahoward

ahoward Mar 1, 2017

Contributor

@coderanger - one of the main uses we've found for app specific secrets is keeping things like credit cards, private certs, etc, stored in a repo. aka, not just *.yml based config. this led to the development of an 'Sekrets.load(path)' api. from there it is trivial to support Sekrets.load("#{ Rails.env }.yml") etc. so totally agree.

Contributor

ahoward commented Mar 1, 2017

@coderanger - one of the main uses we've found for app specific secrets is keeping things like credit cards, private certs, etc, stored in a repo. aka, not just *.yml based config. this led to the development of an 'Sekrets.load(path)' api. from there it is trivial to support Sekrets.load("#{ Rails.env }.yml") etc. so totally agree.

end
def path
@root.join("config", "secrets.yml.enc").to_s

This comment has been minimized.

@morgoth

morgoth Mar 21, 2017

Member

@kaspth Do you think this could be somehow configurable? I can imagine having few encrypted files based on environment

@morgoth

morgoth Mar 21, 2017

Member

@kaspth Do you think this could be somehow configurable? I can imagine having few encrypted files based on environment

This comment has been minimized.

@kaspth

kaspth Mar 21, 2017

Member

I'm open to hearing that argument. But let's revisit that with a written up use case once 5.1 has been out for some time proper 😊

@kaspth

kaspth Mar 21, 2017

Member

I'm open to hearing that argument. But let's revisit that with a written up use case once 5.1 has been out for some time proper 😊

This comment has been minimized.

@januszm

januszm May 23, 2017

👍 for making it configurable. Actually I would not use this feature at all until it's possible to set different encrypted secretes for production, development or staging (or any other) environment.

@januszm

januszm May 23, 2017

👍 for making it configurable. Actually I would not use this feature at all until it's possible to set different encrypted secretes for production, development or staging (or any other) environment.

This comment has been minimized.

@ce07c3

ce07c3 Jun 7, 2017

I cannot use this feature until there is a configureable option for the paths.

@ce07c3

ce07c3 Jun 7, 2017

I cannot use this feature until there is a configureable option for the paths.

This comment has been minimized.

@kaspth

kaspth Jun 7, 2017

Member

I'm open to hearing that argument. But let's revisit that with a written up use case

@ce07c3 ^ all yours. Volunteer the write up if you're in a hurry 😊

@kaspth

kaspth Jun 7, 2017

Member

I'm open to hearing that argument. But let's revisit that with a written up use case

@ce07c3 ^ all yours. Volunteer the write up if you're in a hurry 😊

This comment has been minimized.

@ce07c3

ce07c3 Jun 7, 2017

@kaspth I have messy but working code now. I named my files secrets.yml.enc.staging|production|..., which clashed with if path.end_with?(".enc") (took a while to figure that out), but now I am wondering: are there identical by value but different sets of paths used? I did an override on path and key_path inside Rails::Secrets class << self, but those aren't called until later. Any idea how I am supposed to wire custom paths up? Also, how is @read_encrypted_secrets set?

@ce07c3

ce07c3 Jun 7, 2017

@kaspth I have messy but working code now. I named my files secrets.yml.enc.staging|production|..., which clashed with if path.end_with?(".enc") (took a while to figure that out), but now I am wondering: are there identical by value but different sets of paths used? I did an override on path and key_path inside Rails::Secrets class << self, but those aren't called until later. Any idea how I am supposed to wire custom paths up? Also, how is @read_encrypted_secrets set?

This comment has been minimized.

@ce07c3

ce07c3 Jun 7, 2017

@kaspth Also, how is @read_encrypted_secrets set? figured that out :) I was a bit tired. The rest I am still wondering about, perhaps I just need to dig a bit deeper.

@ce07c3

ce07c3 Jun 7, 2017

@kaspth Also, how is @read_encrypted_secrets set? figured that out :) I was a bit tired. The rest I am still wondering about, perhaps I just need to dig a bit deeper.

This comment has been minimized.

@morgoth

morgoth Jun 21, 2017

Member

@ce07c3 Are you still working on making it configurable? Can I help you somehow?

@morgoth

morgoth Jun 21, 2017

Member

@ce07c3 Are you still working on making it configurable? Can I help you somehow?

This comment has been minimized.

@RyanSnodgrass

RyanSnodgrass Sep 22, 2017

I don't know if anyone will hear this, but I would also like to get a different master key per environment.

@RyanSnodgrass

RyanSnodgrass Sep 22, 2017

I don't know if anyone will hear this, but I would also like to get a different master key per environment.

# Attempt to read encrypted secrets from `config/secrets.yml.enc`.
# Requires an encryption key in `ENV["RAILS_MASTER_KEY"]` or
# `config/secrets.yml.key`.
config.read_encrypted_secrets = true

This comment has been minimized.

@claudiob

claudiob Mar 22, 2017

Member

Hello @kaspth 😄

The USAGE file above says:

A shared: top level key is also supported such that any keys there is merged into the other environments.

This leads me to believe that any "shared" secret key will automatically be available in development.

However this is not true: encrypted secrets are currently only read in production.

How do you feel about changing this and reading encrypted secrets in every environment?

If you think about it, config/secrets.yml.enc is already organized by environment (just like config/secrets.yml), so even if you read it in development, you will not be importing the keys under production:, only those under shared: and development:.

@claudiob

claudiob Mar 22, 2017

Member

Hello @kaspth 😄

The USAGE file above says:

A shared: top level key is also supported such that any keys there is merged into the other environments.

This leads me to believe that any "shared" secret key will automatically be available in development.

However this is not true: encrypted secrets are currently only read in production.

How do you feel about changing this and reading encrypted secrets in every environment?

If you think about it, config/secrets.yml.enc is already organized by environment (just like config/secrets.yml), so even if you read it in development, you will not be importing the keys under production:, only those under shared: and development:.

This comment has been minimized.

@kaspth

kaspth Mar 25, 2017

Member

We skirted away from silently reading them in development just because users have a key and it's not our recommendation to use encrypted secrets as dev secrets storage.

@kaspth

kaspth Mar 25, 2017

Member

We skirted away from silently reading them in development just because users have a key and it's not our recommendation to use encrypted secrets as dev secrets storage.

This comment has been minimized.

@joemsak

joemsak Apr 14, 2017

@kaspth Could you help me understand better what you recommend for development environment? I feel like my "development" keys are still just as worthy of keeping secret and using in the same way as production, no?

I tried finding something in edge guides or blog posts about upcoming rails 5.1 updates, and the PRs/issues around encrypted keys, but I don't find any advice on how to handle secret keys in my test / development environments, as it pertains to this new feature. In the past, I have been using Dotenv.

Thanks

@joemsak

joemsak Apr 14, 2017

@kaspth Could you help me understand better what you recommend for development environment? I feel like my "development" keys are still just as worthy of keeping secret and using in the same way as production, no?

I tried finding something in edge guides or blog posts about upcoming rails 5.1 updates, and the PRs/issues around encrypted keys, but I don't find any advice on how to handle secret keys in my test / development environments, as it pertains to this new feature. In the past, I have been using Dotenv.

Thanks

This comment has been minimized.

@kaspth

kaspth Apr 14, 2017

Member

At the moment we don't recommend using encrypted secrets in dev/test.

@kaspth

kaspth Apr 14, 2017

Member

At the moment we don't recommend using encrypted secrets in dev/test.

This comment has been minimized.

@joemsak

joemsak Apr 14, 2017

Right, I got that. So, what is recommended? I need clarity on that.

Keep using Dotenv, or?

@joemsak

joemsak Apr 14, 2017

Right, I got that. So, what is recommended? I need clarity on that.

Keep using Dotenv, or?

This comment has been minimized.

@kaspth

kaspth Apr 14, 2017

Member

Whatever works for you 😊

@kaspth

kaspth Apr 14, 2017

Member

Whatever works for you 😊

This comment has been minimized.

@januszm

januszm May 23, 2017

I'd say that the recommended way is to use the same method for storing/accessing secrets in either production or dev environment.

@januszm

januszm May 23, 2017

I'd say that the recommended way is to use the same method for storing/accessing secrets in either production or dev environment.

@claudiob

This comment has been minimized.

Show comment
Hide comment
@claudiob

claudiob Mar 22, 2017

Member

@kaspth @dhh

Once I have enabled encrypted secrets, I always need to open the editor to add a new key:

$ rails secrets:edit
[… add a new key in the editor, save and exit …]
# New secrets encrypted and saved.

How do you feel about a new feature to programmatically add encrypted secrets without needing an editor?

$ rails secrets:add API_KEY=123
# New secret API_KEY encrypted and saved.

The concept is very similar to the way in which Travis encrypts environment variables.

Member

claudiob commented Mar 22, 2017

@kaspth @dhh

Once I have enabled encrypted secrets, I always need to open the editor to add a new key:

$ rails secrets:edit
[… add a new key in the editor, save and exit …]
# New secrets encrypted and saved.

How do you feel about a new feature to programmatically add encrypted secrets without needing an editor?

$ rails secrets:add API_KEY=123
# New secret API_KEY encrypted and saved.

The concept is very similar to the way in which Travis encrypts environment variables.

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Mar 22, 2017

Member
Member

dhh commented Mar 22, 2017

@toastercup toastercup referenced this pull request Mar 22, 2017

Closed

Rails 5.1 Upgrade #405

@ce07c3

This comment has been minimized.

Show comment
Hide comment
@ce07c3

ce07c3 Jun 21, 2017

ce07c3 commented Jun 21, 2017

@unikitty37

This comment has been minimized.

Show comment
Hide comment
@unikitty37

unikitty37 Jul 18, 2017

Just a small point, but could somebody add a few lines to the USAGE file (and, preferably, the Rails 5.1 release notes as that's what comes up when you search the web) detailing how you actually get the secrets OUT again?

The release notes say

Secrets will be decrypted in production, using a key stored either in the RAILS_MASTER_KEY environment variable, or in a key file.

but they forget to mention where they're decrypted TO, and I suspect most people wanting to use this feature won't be too keen on having to spelunk through the 17 changed files on this PR…

unikitty37 commented Jul 18, 2017

Just a small point, but could somebody add a few lines to the USAGE file (and, preferably, the Rails 5.1 release notes as that's what comes up when you search the web) detailing how you actually get the secrets OUT again?

The release notes say

Secrets will be decrypted in production, using a key stored either in the RAILS_MASTER_KEY environment variable, or in a key file.

but they forget to mention where they're decrypted TO, and I suspect most people wanting to use this feature won't be too keen on having to spelunk through the 17 changed files on this PR…

@ce07c3

This comment has been minimized.

Show comment
Hide comment
@ce07c3

ce07c3 Jul 18, 2017

ce07c3 commented Jul 18, 2017

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Jul 18, 2017

Member

Please do investigate, then!

Member

kaspth commented Jul 18, 2017

Please do investigate, then!

@ce07c3

This comment has been minimized.

Show comment
Hide comment
@ce07c3

ce07c3 Jul 18, 2017

ce07c3 commented Jul 18, 2017

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Jul 18, 2017

Member

It's an encrypted yaml file. Hence it's also yaml when it's decrypted, not just for "editing purposes".

Member

kaspth commented Jul 18, 2017

It's an encrypted yaml file. Hence it's also yaml when it's decrypted, not just for "editing purposes".

@vizcay

This comment has been minimized.

Show comment
Hide comment
@vizcay

vizcay Aug 3, 2017

Contributor

I've just updated to Rails 5.1, moved all my keys to config/secrets.key.enc, updated .profile exporting RAILS_MASTER_KEY and then my deployment process started to fail because capistrano didn't have access to this variable.. I fixed it by adding set :default_env, { RAILS_MASTER_KEY: IO.read('config/secrets.yml.key') } to my config/deploy.rb file.. maybe someone finds this useful.

Contributor

vizcay commented Aug 3, 2017

I've just updated to Rails 5.1, moved all my keys to config/secrets.key.enc, updated .profile exporting RAILS_MASTER_KEY and then my deployment process started to fail because capistrano didn't have access to this variable.. I fixed it by adding set :default_env, { RAILS_MASTER_KEY: IO.read('config/secrets.yml.key') } to my config/deploy.rb file.. maybe someone finds this useful.

@ce07c3

This comment has been minimized.

Show comment
Hide comment
@ce07c3

ce07c3 Aug 3, 2017

ce07c3 commented Aug 3, 2017

tlwr added a commit to tlwr/personal-photos that referenced this pull request Sep 3, 2017

Use rails built-in encrypted secrets
Instead of ignoring `config/secrets.yml` and having to manage that,
rails provides built-in encrypted secrets functionality.

More information here: rails/rails#28038
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment