-
Notifications
You must be signed in to change notification settings - Fork 21.8k
Add support for multi environment credentials. #33521
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
r? @schneems (@rails-bot has picked a reviewer for you, use r? to override) |
r? @kaspth |
How do you handle editing different credential files? (What key do you use to decrypt, where do you put the different keys for the different files) |
@sidot3291 editing is already possible: To decrypt on staging there is always |
Let's figure out a sensible file structure, so users don't have to muck with For starters, let's try it out with Then the global credentials file + master key is still geared towards production (and staging could use that as well) and the majority use case. But people could even do There's more things to decide, but I think that makes the flow easy to understand and gives us an avenue to explore further. |
@kaspth thanks for feedback. Also this way I guess it won't be possible to share it between development/test (which usually are almost the same) Also I think it will make a trouble for heroku-like flow, when staging/qa apps are using production environment, so it wouldn't be possible to specify separate encrypted credentials file for them - https://devcenter.heroku.com/articles/multiple-environments |
Credentials should always be encrypted. They'll have to merge the unencrypted ones in in some other way.
For now my stance is: too bad. I really don't care much for the
What's the Heroku flow for the 5.2 credentials? Why do they need separate files? Isn't the point that staging uses production credentials? Sounds like a case for |
I think having What about: When calling
or
If one would really need to use development/test credentials (which cannot be unencrypted) then can set path by hand (eventually When calling
For a heroku-like flow it would be a little weird as there would be There is a still gotcha what to do for development/test, but maybe exploring usage of # config/s3.yml
production:
access_key_id: <%= Rails.application.credentials.s3_access_key_id %>
staging:
access_key_id: <%= Rails.application.credentials.s3_access_key_id %>
development:
access_key_id: "dev-key" and then calling |
Here's what I'd like to see. By default, everything stays as it is, but we provide the following path to allow an environment-specific credentials file. It's automatically configured and loaded if the file config/credentials/$environment.yml.enc is available. The key used to decrypt will be either config/credentials/$environment.key or ENV["RAILS_#{Rails.env.upcase}_KEY"]. There's no merging. If you're in the development environment, and config/credentials/development.yml.enc is found, then that'll be loaded. We won't even try to do anything with config/credentials.yml.enc. We won't try to merge some intersection of both files either. These are separate files and separate stores of credentials. No interest in doing anything with unencrypted credentials for this feature. The word "credentials" should mean "encrypted". This approach will require no configuration. All we'd need is a way to specify an environment when using the credential commands. Beyond this, I see any other use of encrypted configuration as an exercise for the developer. We have the primitives for saving and loading encrypted files. That should be enough. |
Would the above work if environment is production, but the enc file is a staging one? Staging would usually run in production mode. |
I would suggest letting config/credentials be what’s used for production (and therefore by any other environment that doesn’t specify its own file).
… On Aug 12, 2018, at 18:10, Prashanth Acharya M ***@***.***> wrote:
Would the above work if environment is production, but the enc file is a staging one? Staging would usually run in production mode.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.
|
OK, I can start with this, it's already some improvement. |
Definitely +1 for the multi environment credentials. Here's my two cents on the subject. I like the idea of exposing I see why the no configuration approach with the Lastly, dropping any kind of support for unencrypted credentials sounds great. Cheers. |
👍 on the idea but I'll suggest that we expose an environment variable The main reason is to be able in development to run a production-like environment if one has the production master key (which we can already set using the |
I completely agree with @Edouard-chin. We should have a simple approach to set where the credentials file is. I've always used had applications that run in |
Riaz, that use case would be solved with the environment approach suggested
above. You can have qa, staging, beta, whatever environments.
…On Mon, Aug 13, 2018 at 12:13 PM, Riaz Virani ***@***.***> wrote:
I completely agree with @Edouard-chin <https://github.com/Edouard-chin>.
We should have a simple approach to set where the credentials file is. I've
always used had applications that run in production mode, but are
actually QA, staging, beta, or other environments. Those environments might
use something like a Stripe test key that needs to be different from the
production key.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#33521 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAAKtXbkSaIMl4M7H6VGcSIkPxuwQBEpks5uQc_LgaJpZM4Vs--G>
.
|
ddaa3dd
to
4db8348
Compare
I pushed some changes. Fallback to This could be done by adding |
Definitely would love to see @schneems' input here for how all of this can coexist with the Heroku ENV variable flow. One possibility is to allow alligator tags in the encrypted yml file, like you can normally do in other yml files. Example: In
then the app can be deployed in different heroku container apps, as they are all running in (I don't know if there would be any security implications in allowing alligator tags in the encrypted yml files -- I tried alligator tags on a vanilla |
Generally, I don't see people on Heroku using encrypted credential store at all. Instead, they use the |
@dhh @kaspth This is what I was talking about freeletics@df12269 - it gives possibility to customize paths and not rely only on Rails env. Let me know if this would be fine or if we should go with solution present in this PR. |
@morgoth Quick cursory look says this is the right direction. Look for a environment-specific credentials first, if not found, fall back to global credentials. |
4db8348
to
b42c453
Compare
b42c453
to
b42efbe
Compare
Definitely need to fix this. Like the idea of not loading the app if we can get away with it. Another option is to have a config for not exploding if the key is missing that would be triggered by the credentials editing command.
… On Dec 24, 2018, at 05:39, Christos Zisopoulos ***@***.***> wrote:
While evaluating this feature, I came across a problem trying to create a new environment for my app.
Once you have created your first environment (e.g. development) you might start adding configuration in initialisers that tries to access encrypted credentials like so:
# config/initializers/sidekiq.rb
Sidekiq::Web.set :session_secret, Rails.application.credentials.secret_key_base!
That works fine when running rails credentials:edit --environment development
If you now decide to create a staging environment, doing rails credentials:edit --environment staging will fail with something similar to ...ordered_options.rb:49:in 'method_missing': :secret_key_base is blank (KeyError)
A great many gems that access 3rd party APIs use an initialiser for configuring credentials so the above situation is quite likely to occur.
I worked around the issue by creating a new empty Rails app, running rails credentials:edit --environment staging in it, and copying across the generated credential files.
Still, I now have to be careful to add any new keys I need to all existing environments before writing the code that uses them.
An alternative solution would be for rails credentials:edit to not load the application just to edit a file.
Is there any reason the entire application is loaded before editing the credentials?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.
|
@dhh I had a stab at it here: #34789 Not entirely sure about my testing strategy though. I couldn't quite get my head around the entire Also, seemingly unrelated failure on Travis https://travis-ci.org/rails/rails/jobs/472199122 ( |
def credentials | ||
@credentials ||= encrypted("config/credentials.yml.enc") | ||
@credentials ||= encrypted(config.credentials.content_path, key_path: config.credentials.key_path) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there an issue here where this line is ignoring env_key
? If I set the key I used to encrypt my credentials file in RAILS_STAGING_KEY
for example, I think it is being ignored here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Although RAILS_#{options[:environment].upcase}_KEY
was originally supported, it was removed in #33928. Use RAILS_MASTER_KEY
instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perfect, I see how to use it now thanks
It sounds very bad. It adds a lot of complexity to scenarios where that complexity is completely redundand. It creates a lot of extra work.
Encrypted credentials can be useful in some scenarios (these scenarios are unknown to me, however. There always are better places to store credentials than git repository). Unencrypted credentials are useful in many common and basic scenarios. I'm very disappointed by dropping unencrypted credentials. In many of my use cases this adds a lot of deployment complexity and provides no extra security at all. |
I see your point that editing encrypted credentials is not as simple as unencrypted ones used to be, but that's the price you pay for the extra security. If there's no need to secure your development configuration, then why use credentials file in the first place? Isn't it simpler to just user configuration files directly, which already have support for multiple environments (I'm talking about |
That is definitely possible, but adds fragmentation. Database is, well, for database, not for API keys, tokens, passwords etc. If your application is written to take values out of config, then you cannot use encrypted credentials. If your application is written to read encrypted credentials, then you cannot easily replace it with plaintext values. You have to commit to one way and then use it for everything. And if 3rd-party library is reading credentials, then you are completely out of luck. E.g. in Rails 3 app I just had a Kubernetes secret mounted over This is a real-world illustration to my first post. This was (and remains) perfectly secure scheme without any encryption at all. It can't be more secure without extrnal mechanisms (you can't store key more securely than data), and there is a lot of management burden mainly due to inability to conveniently edit credentials. Mandating encrypted credentials due to "extra security" is simiar to banning knives since someone can accidentally cut himself. |
@makkarpov @mladenilic I would encourage you to use our discuss forums for these types of conversations. 🙏 |
Better late than never but maybe Cre should be part of rails because nobody like to spell it all out. |
Utilise multi environment credentials introduced via rails/rails#33521 This changes fixes an issue when setting up the developer environment. Executing `bin/setup` generated the following error: ``` == Preparing database == rails aborted! ActiveSupport::MessageEncryptor::InvalidMessage: ActiveSupport::MessageEncryptor::InvalidMessage railsdevs.io/config/environment.rb:5:in `<top (required)>' Caused by: OpenSSL::Cipher::CipherError: railsdevs.io/config/environment.rb:5:in `<top (required)>' ``` This due to 'public' developers not having access to the `config/master.key`. 'owner' developers should update `config/master.key` to `config/credentials/production.key` following this change, ensuring that the renamed key is not committed to version control (which should be ensured by the change to `.gitignore`)
Utilise multi environment credentials introduced via rails/rails#33521 This changes fixes an issue when setting up the developer environment. Executing `bin/setup` generated the following error: ``` == Preparing database == rails aborted! ActiveSupport::MessageEncryptor::InvalidMessage: ActiveSupport::MessageEncryptor::InvalidMessage railsdevs.io/config/environment.rb:5:in `<top (required)>' Caused by: OpenSSL::Cipher::CipherError: railsdevs.io/config/environment.rb:5:in `<top (required)>' ``` This due to 'public' developers not having access to the `config/master.key`. 'owner' developers should update `config/master.key` to `config/credentials/production.key` following this change, ensuring that the renamed key is not committed to version control (which should be ensured by the change to `.gitignore`)
Utilise multi environment credentials introduced via rails/rails#33521 This changes fixes an issue when setting up the developer environment. Executing `bin/setup` generated the following error: ``` == Preparing database == rails aborted! ActiveSupport::MessageEncryptor::InvalidMessage: ActiveSupport::MessageEncryptor::InvalidMessage railsdevs.io/config/environment.rb:5:in `<top (required)>' Caused by: OpenSSL::Cipher::CipherError: railsdevs.io/config/environment.rb:5:in `<top (required)>' ``` This due to 'public' developers not having access to the `config/master.key`. 'owner' developers should update `config/master.key` to `config/credentials/production.key` following this change, ensuring that the renamed key is not committed to version control (which should be ensured by the change to `.gitignore`) Resources: * rails/rails#33521 * https://blog.saeloun.com/2019/10/10/rails-6-adds-support-for-multi-environment-credentials.html
Utilise multi environment credentials introduced via rails/rails#33521 This changes fixes an issue when setting up the developer environment. Executing `bin/setup` generated the following error: ``` == Preparing database == rails aborted! ActiveSupport::MessageEncryptor::InvalidMessage: ActiveSupport::MessageEncryptor::InvalidMessage railsdevs.io/config/environment.rb:5:in `<top (required)>' Caused by: OpenSSL::Cipher::CipherError: railsdevs.io/config/environment.rb:5:in `<top (required)>' ``` This due to 'public' developers not having access to the `config/master.key`. 'owner' developers should update `config/master.key` to `config/credentials/production.key` following this change, ensuring that the renamed key is not committed to version control (which should be ensured by the change to `.gitignore`) Resources: * rails/rails#33521 * https://blog.saeloun.com/2019/10/10/rails-6-adds-support-for-multi-environment-credentials.html
Usage:
If one wants to use staging encrypted credentials:
This will create files
config/credentials/staging.yml.enc
andconfig/credentials/staging.key
When calling
Rails.application.credentials
in staging environment, it takes precedence over defaultconfig/credentials.yml.enc
Default paths can be overwritten by setting
config.credentials.content_path
andconfig.credentials.key_path
It is backward compatible.
Another try for #30067 (comment)