Add support for multi environment credentials.

[morgoth]

Usage:

If one wants to use staging encrypted credentials:

rails credentials:edit --environment staging

This will create files config/credentials/staging.yml.enc and config/credentials/staging.key
When calling Rails.application.credentials in staging environment, it takes precedence over default
config/credentials.yml.enc
Default paths can be overwritten by setting config.credentials.content_path and config.credentials.key_path

It is backward compatible.

Another try for #30067 (comment)

[rails-bot]

r? @schneems

(@rails-bot has picked a reviewer for you, use r? to override)

[morgoth]

r? @kaspth

[sidot3291]

How do you handle editing different credential files? (What key do you use to decrypt, where do you put the different keys for the different files)

[morgoth]

@sidot3291 editing is already possible:
bin/rails encrypted:edit config/your-enctyped-file.yml.enc --key config/your-master.key

To decrypt on staging there is always config/master.key or RAILS_MASTER_KEY but it should be fine - on staging/production you don't need to decrypt 2 files, so you can set appropriate master key for given env

[kaspth]

Let's figure out a sensible file structure, so users don't have to muck with config.credentials_file. It'll simply use the env appropriate file or fall back on credentials.yml.enc.

For starters, let's try it out with config/environments/development/credentials.yml.enc and config/environments/development/master.key. Then credentials:edit needs to support a --environment option.

Then the global credentials file + master key is still geared towards production (and staging could use that as well) and the majority use case. But people could even do credentials:edit --production if they'd want it split.

There's more things to decide, but I think that makes the flow easy to understand and gives us an avenue to explore further.

[morgoth]

@kaspth thanks for feedback.
What if one wants to use development/test credentials without encryption? Is it fine to look for config/environments/development/credentials.yml.enc first and if not found, fallback to config/environments/development/credentials.yml? I know that it somehow defeats the original idea, but many mention it as a pain point.

Also this way I guess it won't be possible to share it between development/test (which usually are almost the same)

Also I think it will make a trouble for heroku-like flow, when staging/qa apps are using production environment, so it wouldn't be possible to specify separate encrypted credentials file for them - https://devcenter.heroku.com/articles/multiple-environments

[kaspth]

What if one wants to use development/test credentials without encryption?

Credentials should always be encrypted. They'll have to merge the unencrypted ones in in some other way.

Also this way I guess it won't be possible to share it between development/test (which usually are almost the same)

For now my stance is: too bad. I really don't care much for the development.credentials.yml.enc constructs or some such. That or my config.credentials.content_path suggestion in the main thread.

Also I think it will make a trouble for heroku-like flow, when staging/qa apps are using production environment, so it wouldn't be possible to specify separate encrypted credentials file for them - https://devcenter.heroku.com/articles/multiple-environments

What's the Heroku flow for the 5.2 credentials? Why do they need separate files? Isn't the point that staging uses production credentials? Sounds like a case for config.credentials.content_path.

[morgoth]

I think having config.credentials.content_path would solve many complains.

What about:

When calling Rails.application.credentials

1. take a file from config.credentials.content_path if set
2. if not set, fallback to config/credentials.yml.enc
3. for new apps (Rails 6.0?) generate a line in config/environments/production.rb: config.credentials.content_path = "config/environments/production/credentials.yml.enc

or

1. take a file from config.credentials.content_path always
2. by default set it to config/credentials.yml.enc (so it's compatible with current apps)
3. do not expose config.credentials.content_path in newly generated apps, but allow to set it (for people that need it)
4. Eventually deprecate (in Rails 6.0) old location of config/credentials.yml.enc and by default generate file in config/environments/production/credentials.yml.enc, to even more emphasize it's a production thingy and not a general tool for all envs.

If one would really need to use development/test credentials (which cannot be unencrypted) then can set path by hand (eventually config.credentials.key_path could also be exposed). But I don't think that using encrypted development/test files make much sense overall.

When calling bin/rails credentials:edit --environment staging:

1. just look in the folder config/environments/staging/ for credentails.yml.enc and master.key (this obviously cannot take config.credentials.content_path)

For a heroku-like flow it would be a little weird as there would be config/environments/staging folder, but no config/environments/staging.rb. However it would work at least.

There is a still gotcha what to do for development/test, but maybe exploring usage of Rails.application.config_for(:feature) could be an option? Like:

# config/s3.yml
production:
  access_key_id: <%= Rails.application.credentials.s3_access_key_id %>
staging:
  access_key_id: <%= Rails.application.credentials.s3_access_key_id %>
development:
  access_key_id: "dev-key"

and then calling Rails.application.config_for(:s3)["access_key_id"] in the app code. This still is not that nice in syntax and config_for doesn't cache read file.
Anyway that could be discussed separately - just a random thought :)

[dhh]

Here's what I'd like to see. By default, everything stays as it is, but we provide the following path to allow an environment-specific credentials file. It's automatically configured and loaded if the file config/credentials/$environment.yml.enc is available. The key used to decrypt will be either config/credentials/$environment.key or ENV["RAILS_#{Rails.env.upcase}_KEY"].

There's no merging. If you're in the development environment, and config/credentials/development.yml.enc is found, then that'll be loaded. We won't even try to do anything with config/credentials.yml.enc. We won't try to merge some intersection of both files either. These are separate files and separate stores of credentials.

No interest in doing anything with unencrypted credentials for this feature. The word "credentials" should mean "encrypted".

This approach will require no configuration. All we'd need is a way to specify an environment when using the credential commands.

Beyond this, I see any other use of encrypted configuration as an exercise for the developer. We have the primitives for saving and loading encrypted files. That should be enough.

[prashantham]

Would the above work if environment is production, but the enc file is a staging one? Staging would usually run in production mode.

[dhh]

I would suggest letting config/credentials be what’s used for production (and therefore by any other environment that doesn’t specify its own file).

…

On Aug 12, 2018, at 18:10, Prashanth Acharya M ***@***.***> wrote: Would the above work if environment is production, but the enc file is a staging one? Staging would usually run in production mode. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[morgoth]

OK, I can start with this, it's already some improvement.
@schneems maybe you could share your experience with this feature on Heroku. Are people using encrypted credentials?

[mladenilic]

Definitely +1 for the multi environment credentials.

Here's my two cents on the subject. I like the idea of exposing config.credentials_file. This would allow credentials path to be set per environment and it can fallback to config/credentials.yml.enc, so the change is backward compatible.

I see why the no configuration approach with the config/credentials/$environment.yml.enc can be appealing, but I don't like the confusion it will introduce with multiple options for credentials: config/credentials/$environment.yml.enc and config/credentials.yml.enc.

Lastly, dropping any kind of support for unencrypted credentials sounds great.

Cheers.

[Edouard-chin]

It's automatically configured and loaded if the file config/credentials/$environment.yml.enc is available.

👍 on the idea but I'll suggest that we expose an environment variable CREDENTIALS_PATH (or something) that will take precedence over it.

The main reason is to be able in development to run a production-like environment if one has the production master key (which we can already set using the RAILS_MASTER_KEY env)

[rvirani1]

I completely agree with @Edouard-chin. We should have a simple approach to set where the credentials file is. I've always used had applications that run in production mode, but are actually QA, staging, beta, or other environments. Those environments might use something like a Stripe test key that needs to be different from the production key.

[dhh]

Riaz, that use case would be solved with the environment approach suggested above. You can have qa, staging, beta, whatever environments.

…

On Mon, Aug 13, 2018 at 12:13 PM, Riaz Virani ***@***.***> wrote: I completely agree with @Edouard-chin <https://github.com/Edouard-chin>. We should have a simple approach to set where the credentials file is. I've always used had applications that run in production mode, but are actually QA, staging, beta, or other environments. Those environments might use something like a Stripe test key that needs to be different from the production key. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#33521 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAKtXbkSaIMl4M7H6VGcSIkPxuwQBEpks5uQc_LgaJpZM4Vs--G> .

[morgoth]

I pushed some changes.
I'm not sure about having different name for env key (RAILS_PRODUCTION_KEY). I think using always RAILS_MASTER_KEY for all cases would be more convenient, as there is no need to have RAILS_PRODUCTION_KEY and RAILS_STAGING_KEY in the same time.

---

Fallback to config/credentials.yml.enc could go already to Rails 5.x.
What about having a new default for Rails 6.0 to always genereate config/credentials/production.yml.enc and config/credentials/production.key in new apps?
One reason to do it, would be to avoid having 2 ways of doing the same and to emphasize that encrypted credentials is for production-like environments.

This could be done by adding Rails.config.credentials.content_path and Rails.config.credentials.key_path that would be set to currently running environment by default, but allowing user to overwrite it (heroku-like flow). And also this way we would not break upgrading apps from 5.x to 6 as it could be done by load_defaults https://github.com/rails/rails/blob/master/railties/lib/rails/application/configuration.rb#L117-L132

[fredngo]

@schneems maybe you could share your experience with this feature on Heroku. Are people using encrypted credentials?

Definitely would love to see @schneems' input here for how all of this can coexist with the Heroku ENV variable flow.

One possibility is to allow alligator tags in the encrypted yml file, like you can normally do in other yml files.

Example: In config/credentials/production.yml.enc, you would be able to write:

some_heroku_plugin_key: <%= ENV['SOME_HEROKU_PLUGIN_KEY' %>
some_other_api_key: axifdsr43jhgf9f

then the app can be deployed in different heroku container apps, as they are all running in production environment, but each still has their own ENV variables.

(I don't know if there would be any security implications in allowing alligator tags in the encrypted yml files -- I tried alligator tags on a vanilla credentials.yml.enc and it does not work, so it must have been disabled for a reason.)

[schneems]

Generally, I don't see people on Heroku using encrypted credential store at all. Instead, they use the heroku config env var store we provide them directly. As long as new behavior doesn't require people to use the encrypted config stores then it won't affect that story much.

[morgoth]

@dhh @kaspth This is what I was talking about freeletics@df12269 - it gives possibility to customize paths and not rely only on Rails env.

Let me know if this would be fine or if we should go with solution present in this PR.

[dhh]

@morgoth Quick cursory look says this is the right direction. Look for a environment-specific credentials first, if not found, fall back to global credentials.

[serggl]

while we all wait for Rails 6 release, there is an easy way to add a multi environment credentials functionality to your Rails 5.2 apps without 3rd party gems. Just add a little piece of code to your application.rb:

module YourAppModule
  class Application < Rails::Application
    def credentials
      if Rails.env.production?
        super
      else
        encrypted(
          "config/credentials.#{Rails.env.downcase}.yml.enc",
          key_path: "config/#{Rails.env.downcase}.key"
        )
      end
    end
  end
end

Running RAILS_ENV=staging bin/rails credentials:edit will add some staging credentials for you (create a staging.key key file first ofcourse)

[christos]

While evaluating this feature, I came across a problem trying to create a new environment for my app.

Once you have created your first environment (e.g. development) you might start adding configuration in initialisers that tries to access encrypted credentials like so:

# config/initializers/sidekiq.rb
Sidekiq::Web.set :session_secret, Rails.application.credentials.secret_key_base!

That works fine when running rails credentials:edit --environment development

If you now decide to create a staging environment, doing rails credentials:edit --environment staging will fail with something similar to ...ordered_options.rb:49:in 'method_missing': :secret_key_base is blank (KeyError)

A great many gems that access 3rd party APIs use an initialiser for configuring credentials so the above situation is quite likely to occur.

I worked around the issue by creating a new empty Rails app, running rails credentials:edit --environment staging in it, and copying across the generated credential files.

Still, I now have to be careful to add any new keys I need to all existing environments before writing the code that uses them.

An alternative solution would be for rails credentials:edit to not load the application just to edit a file.

Is there any reason the entire application is loaded before editing the credentials?

[dhh]

Definitely need to fix this. Like the idea of not loading the app if we can get away with it. Another option is to have a config for not exploding if the key is missing that would be triggered by the credentials editing command.

…

On Dec 24, 2018, at 05:39, Christos Zisopoulos ***@***.***> wrote: While evaluating this feature, I came across a problem trying to create a new environment for my app. Once you have created your first environment (e.g. development) you might start adding configuration in initialisers that tries to access encrypted credentials like so: # config/initializers/sidekiq.rb Sidekiq::Web.set :session_secret, Rails.application.credentials.secret_key_base! That works fine when running rails credentials:edit --environment development If you now decide to create a staging environment, doing rails credentials:edit --environment staging will fail with something similar to ...ordered_options.rb:49:in 'method_missing': :secret_key_base is blank (KeyError) A great many gems that access 3rd party APIs use an initialiser for configuring credentials so the above situation is quite likely to occur. I worked around the issue by creating a new empty Rails app, running rails credentials:edit --environment staging in it, and copying across the generated credential files. Still, I now have to be careful to add any new keys I need to all existing environments before writing the code that uses them. An alternative solution would be for rails credentials:edit to not load the application just to edit a file. Is there any reason the entire application is loaded before editing the credentials? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

[christos]

@dhh I had a stab at it here: #34789

Not entirely sure about my testing strategy though. I couldn't quite get my head around the entire railties testing harness.

Also, seemingly unrelated failure on Travis https://travis-ci.org/rails/rails/jobs/472199122 (mysql2/mariadb)

[l33z3r]

Is there an issue here where this line is ignoring env_key ? If I set the key I used to encrypt my credentials file in RAILS_STAGING_KEY for example, I think it is being ignored here

[jonathanhefner]

Although RAILS_#{options[:environment].upcase}_KEY was originally supported, it was removed in #33928. Use RAILS_MASTER_KEY instead.

[l33z3r]

Perfect, I see how to use it now thanks

[makkarpov]

@mladenilic

Lastly, dropping any kind of support for unencrypted credentials sounds great.

It sounds very bad. It adds a lot of complexity to scenarios where that complexity is completely redundand. It creates a lot of extra work.

1. Development: any developer needs to be able to run the application. So, it needs access to development credentials. These credentials are not so secret, they contain dummy passwords to local database, dummy keys (all-zeros or changeit or 010203...) and similar things. So you have two options:
   (1) send encryption key to every developer by other means (why? you already have access control at the repo. code is available for developers only)
   (2) commit key file to repository alongside with file (why encryption then? just to make editing harder?)

   1.1 If you have "real" testing credentials alongside dummy development credentials the best thing that you can do is to make "development.yml.template" file and ask each developer to fill in this template. Encryption does not help here. You still either share the key (and expose all credentials) or don't share (and application will not even start).

2. Production: I have a trusted server that runs my code. I have credentials stored at that server. I don't need any encryption there, because server is trusted. I don't need even to store these credentials in repository. I just set something like CREDENTIALS_FILE=/some/path/credentials.yml and it's ok. These plaintext credentials are just stored outside of git repo. There is no need to commit binary blob that no one can look, no one can edit and no one can use. Commiting encrypted credentials is just upload of useless binary data to Git.

Encrypted credentials can be useful in some scenarios (these scenarios are unknown to me, however. There always are better places to store credentials than git repository). Unencrypted credentials are useful in many common and basic scenarios.

I'm very disappointed by dropping unencrypted credentials. In many of my use cases this adds a lot of deployment complexity and provides no extra security at all.

[mladenilic]

@makkarpov

I see your point that editing encrypted credentials is not as simple as unencrypted ones used to be, but that's the price you pay for the extra security. If there's no need to secure your development configuration, then why use credentials file in the first place? Isn't it simpler to just user configuration files directly, which already have support for multiple environments (I'm talking about config/environments/... and config/database.yml)?

[makkarpov]

That is definitely possible, but adds fragmentation. Database is, well, for database, not for API keys, tokens, passwords etc. environment/*.rb can be used, but still, this is a general-purpose config. API keys and tokens are still credentials, and whether you want to encrypt them is a per-environment choice.

If your application is written to take values out of config, then you cannot use encrypted credentials. If your application is written to read encrypted credentials, then you cannot easily replace it with plaintext values. You have to commit to one way and then use it for everything. And if 3rd-party library is reading credentials, then you are completely out of luck.

E.g. in Rails 3 app I just had a Kubernetes secret mounted over secrets.yml file. Kubernetes secrets are already protected by Kubernetes itself, and there is no need to further encrypt them. At least there is no more protected storage for key. Credentials are already stored in most secure location possible in Kubernetes system. Moreover, there is no interactive session to do rails credentials:edit. Host system is not aware of Rails at all (it doesn't even have Ruby). Docker container has ruby, but (1) container has no write access to secret (2) Rails is not aware of Kubernetes existence and cannot edit secrets.

This is a real-world illustration to my first post. This was (and remains) perfectly secure scheme without any encryption at all. It can't be more secure without extrnal mechanisms (you can't store key more securely than data), and there is a lot of management burden mainly due to inability to conveniently edit credentials. Mandating encrypted credentials due to "extra security" is simiar to banning knives since someone can accidentally cut himself.

[zzak]

@makkarpov @mladenilic I would encourage you to use our discuss forums for these types of conversations. 🙏

[khalilgharbaoui]

Better late than never but maybe Cre should be part of rails because nobody like to spell it all out.
https://github.com/khalilgharbaoui/cre
@zzak i wil continue to conversate on the discuss forums thanks for the tip.