Update secrets to use modern crypto #28139

stouset · 2017-02-23T22:56:54Z

Fixes #28135 by replacing the default mode with AES-128-GCM, allowing the mode to be configured manually, correctly generating keys, and using a random initialization vector on every encryption.

rafaelfranca · 2017-02-23T23:00:18Z

railties/lib/rails/generators/rails/app/templates/config/environments/production.rb.tt

@@ -17,6 +17,7 @@ Rails.application.configure do
  # Attempt to read encrypted secrets from `config/secrets.yml.enc`.
  # Requires an encryption key in `ENV["RAILS_MASTER_KEY"]` or
  # `config/secrets.yml.key`.
+  # config.secrets_cipher = "aes-128-gcm"


I'm all to add a config for this but I don't think we should promote it by default.

I agree. I only erred on including it because existing users of the beta will need to know how to either switch back (bad), or migrate to GCM (better).

There's also the unfortunate case of machines with ancient OpenSSL < 1.0.1, which doesn't support GCM.

rafaelfranca · 2017-02-23T23:02:45Z

railties/lib/rails/application/configuration.rb

@@ -53,6 +53,7 @@ def initialize(*)
        @x                               = Custom.new
        @enable_dependency_loading       = false
        @read_encrypted_secrets          = false
+        @secrets_cipher                  = "aes-128-gcm"


I think we can remove this default here since we set it at bootstrap.

Is it better to leave it here and not set it in the bootstrap (to keep defaults all in the same place)?

(fixed, assuming your answer to the above question is a yes)

rafaelfranca · 2017-02-23T23:03:20Z

railties/test/secrets_test.rb

+      end_of_secrets
+
+      Rails.application.config.secrets_cipher = "aes-128-cbc"
+      Rails.application.instance_variable_set(:@secrets, nil) # Dance around caching 💃🕺


lol for the emoji

rafaelfranca · 2017-02-23T23:04:25Z

railties/lib/rails/application/configuration.rb

@@ -14,7 +14,7 @@ class Configuration < ::Rails::Engine::Configuration
                    :ssl_options, :public_file_server,
                    :session_options, :time_zone, :reload_classes_only_on_change,
                    :beginning_of_week, :filter_redirect, :x, :enable_dependency_loading,
-                    :read_encrypted_secrets
+                    :read_encrypted_secrets, :secrets_cipher


We need to add an explanation for this config at configuration.md in guides.

I'll add that in a moment.

Done. 798280f

rafaelfranca · 2017-02-23T23:04:41Z

Thank you so much for the pull request.

matthewd · 2017-02-23T23:08:31Z

railties/lib/rails/secrets.rb

@@ -30,20 +32,24 @@ def parse(paths, env:)
      end

      def generate_key
-        cipher = new_cipher
-        SecureRandom.hex(cipher.key_len)[0, cipher.key_len]
+        new_cipher.random_key


This is random bytes, right?

If so, I think we should probably pack/unpack to hex, to make it safer for people to move around.

Encoding to a human-readable format should be the responsibility of an outer layer; internally, crypto libraries should only ever deal with bytes. It's too easy to get things wrong otherwise (see: this exact method getting key generation wrong due to mixing and matching hex-encoding and binary).

ISTM this is the outer layer. This class is responsible for receiving the key from the user (3 lines down), so it's going to have to know how to convert it back to raw bytes.

This method is currently a simple pass-through to the cipher doing the real work, so it feels reasonable for this to hexify -- and for the key method below to de-hexify. That keeps said work well away from the "real stuff" down in encrypt/decrypt.

Done.

29057f1

ptoomey3 · 2017-02-23T23:49:42Z

railties/lib/rails/secrets.rb

-        cipher(:decrypt, data)
+        cipher = new_cipher.decrypt
+        cipher.key = key
+        cipher.iv = data[0, cipher.iv_len]


Should you be checking cipher.authenticated? here as well just in case anyone swaps out config.secrets_cipher?

EDIT: I meant to post this one line below.

Good catch. 2bc7446#diff-bd3587684013cd6ff722fc8118d4bc22R47

ptoomey3 · 2017-02-24T00:04:23Z

railties/lib/rails/secrets.rb

+
+        cipher.key = key
+        cipher.iv = data[0, cipher.iv_len]
+        cipher.auth_tag = data[cipher.iv_len, auth_tag_len]


I don't think you can even call the setter:

cipher = OpenSSL::Cipher.new("aes-128-cbc") cipher.auth_tag = "" OpenSSL::Cipher::CipherError: authentication tag not supported by this cipher

sigh, you're again right. Sorry, haven't yet figured a way to test the file in isolation, and I'm muxing this with the work I actually get paid to do. :)

1fef784dd63598df121de2c9f70cca9556b88439

haven't yet figured a way to test the file in isolation

Either of cd railties; bundle exec ruby -Itest test/secrets_test.rb or cd railties; bin/test test/secrets_test.rb should do it, if that helps.

bdewater · 2017-02-24T04:20:16Z

railties/lib/rails/secrets.rb

+        cipher.iv = data[0, cipher.iv_len]
+
+        cipher.auth_tag = data[cipher.iv_len, auth_tag_len] if
+          cipher.authenticated?


This seems to miss a check for auth_tag byte length, see ruby/openssl#63

I've implemented this for MessageEncryptor here:

rails/activesupport/lib/active_support/message_encryptor.rb

Lines 110 to 113 in 46bb76a

# Currently the OpenSSL bindings do not raise an error if auth_tag is

# truncated, which would allow an attacker to easily forge it. See

# https://github.com/ruby/openssl/issues/63

raise InvalidMessage if aead_mode? && auth_tag.bytes.length != 16

The only possible way for this to produce fewer bytes of the auth_tag is if the remainder of the message is empty (and an empty message validates against whatever fragment of an auth_tag was provided).

Either way I think we can agree that the OpenSSL API for this is absolutely garbage. I'll put in a check for it as an extra precaution.

Apparently Ruby 2.4 has a new auth_tag_len= method on Cipher, which potentially forces this check to happen internally to OpenSSL. What's the minimum Ruby version supported by Rails 5.1?

It's Ruby 2.2.2, as of 2.4 the OpenSSL gem has been pulled out of the standard library and gemified as version 2.0. Unfortunately that requires at least Ruby 2.3.

While we're on the subject of OpenSSL API, according to the gem C source for ossl_cipher_set_auth_tag_len: "Note that not all AEAD ciphers support this method." 😒

vipulnsward · 2017-02-24T04:40:09Z

railties/lib/rails/secrets.rb

@@ -12,10 +13,15 @@ def initialize
      end
    end

+    # OpenSSL::Cipher uses a hardcoded tag length of 16 bytes
+    AUTH_TAG_LEN = 16


This constant is not available from openssl lib? If it is we should use that.

It isn't. It's just hardcoded as 16 in the Ruby source.

bdewater · 2017-02-24T04:56:08Z

Come to think of it, ActiveSupport is a dependency of Railties (unless I'm mistaken). Why wasn't MessageEncryptor used to implement this feature in the first place, or aren't we switching to it now? It already does everything (including AEAD since #25874) we're trying to fix here.

stouset · 2017-02-24T08:57:54Z

@bdewater Now you tell me. :(

I can rewrite this to use MessageEncryptor if you'd prefer.

kaspth · 2017-02-24T09:21:41Z

@bdewater that was the plan, but with all the back and forth on the user experience changes I forgot to make the change.

kaspth · 2017-02-24T09:25:00Z

guides/source/configuring.md

@@ -140,6 +140,8 @@ defaults to `:debug` for all environments. The available log levels are: `:debug

 * `secrets.secret_key_base` is used for specifying a key which allows sessions for the application to be verified against a known secure key to prevent tampering. Applications get `secrets.secret_key_base` initialized to a random key present in `config/secrets.yml`.

+* `config.secrets_cipher` allows you to change the cipher used to protect the `config/secrets.yml` file. This defaults to `"aes-128-gcm"`. Versions of OpenSSL earlier than 1.0.01, unfortunately, don't support GCM mode, so degrading this to `"aes-128-cbc"` may be necessary. If you find yourself in this situation, strongly consider updating OpenSSL to a modern version if at all possible rather than weakening this setting.


Not correct. config/secrets.yml.enc isn't an encrypted version of config/secrets.yml. They're separate.

I'm okay with requiring a specific >= OpenSSL version since encrypted secrets is an optional feature.

👍

secrets.yml.enc being a totally different thing than secrets.yml is very counterintuitive. Shouldn't things like the secret_key_base in there be something that goes into the encrypted version too?

Apologies if I'm a bit rusty — haven't used Rails extensively in about a year at this point.

Keeping both secrets.yml and secrets.yml.enc seems like a booby trap. It'd be easy to put a secret in the clear version and check it in.

Shouldn't things like the secret_key_base in there be something that goes into the encrypted version too?

By default secrets.yml reads a SECRET_KEY_BASE from the ENV, but yes when opting in to encrypted secrets you'd move it stuff it in config/secrets.yml.enc.

@wlipa perhaps, but I'm not sure how likely people would be to do that. E.g. the command to edit encrypted secrets is bin/rails secrets:edit so if you haven't run that, then it's not the right file 😊

That's reasonable. The naming is still confusing, because now secrets.yml is for things that are not actually secret.

@matthewd If you have a file named secrets.yml, people are going to put production secrets into it. Documentation won't matter. Comments in the YAML file warning against it won't matter. <blink><marquee><font color="red"> won't matter.

Having that file is begging people to do the wrong thing. If it's supposed to be for development-only, have it be set in config/environments/development.rb.

@stouset those who would just jam secrets into a file that sounds secret-y are not who we design Rails for. We trust that people will try to do the right thing and that they can handle conventions as well as adjusting to new conventions. Such as when we introduced secrets.yml in 4.1 and when we're now adding encrypted secrets alongside that.

Speaking as an infosec person, I don't think that's a reasonable position to take.

API design matters. It doesn't matter how smart and capable your users are if your APIs guide users to make the wrong choice. Naming a file secrets.yml and putting things that look like secrets into them is sending people the wrong message, and users — smart, capable users — will misunderstand its purpose and make mistakes. In this case, security-impacting mistakes.

On the flip side, what's the actual cost of changing the design of this feature to guide people away from making mistakes? My guess is: not much.

Your argument reminds me of this image:

Bottles on the left are cleaning chemicals. Bottles on the right are juice. You're taking a gamble that the developers Rails caters to are never going to mistake the bottles on the left for the bottles on the right. I've seen from repeated experience that they will.

I'd totally expect config/secrets.yml to be the decrypted config/secrets.enc, too… even knowing what's actually happening and why. 😊

The mixed heritage behind this feature is muddling things up. We have a mix of approaches like

config/secrets.yml is a gitignored file that's copied in as needed in dev and on production deploy

config/secrets.yml is a checked-in file that doesn't contain actual secrets, just ERB preprocessing that references environment variables with actual secrets

config/secrets.yml is a checked-in file that contains actual secrets, but only for dev/test.

And they're competing for mindshare and, literally, file naming.

In any case, this is way out of scope for this PR. Good topic for further investigation.

kaspth · 2017-02-24T09:26:32Z

railties/lib/rails/application/bootstrap.rb

@@ -80,6 +80,7 @@ module Bootstrap
      end

      initializer :set_secrets_root, group: :all do
+        Rails::Secrets.cipher = config.secrets_cipher


Let's dispense with the config and decide on the best encryption default.

kaspth · 2017-02-24T09:48:45Z

Thanks so much for the pull request! Reporting security issues very early in our release phase means a lot ❤️

Given your suggestions I think most of this could be boiled down to:

gem "openssl", ">= 1.0.01" # Enforce OpenSSL that supports GCM authenticated ciphers.
require "openssl"

require "active_support/message_encryptor"

# Later…

def generate_key
  encryptor.random_key.unpack("H*").first # Requires adding `random_key` to MessageEncryptor.
end

def encrypt(text)
  encryptor.encrypt_and_sign(text)
end

def decrypt(data)
  encryptor.decrypt_and_verify(data)
end

private
  def encryptor
    @encryptor ||= ActiveSuport::MessageEncryptor.new(key, cipher: "aes-128-gcm") # If we pick an `authenticated?` cipher the encryptor handles `auth_tag` checks, iv seed etc.
  end

bdewater · 2017-02-24T12:30:06Z

OpenSSL (not the gem, the system library) 1.0.1 was released in March 2012 and support for it ended at the end of last year. Besides AES-GCM it also brought support for TLSv1.1 and 1.2. I think it's a fair assumption that somebody's server already has this or a newer version installed for modern internet crypto.

stouset · 2017-02-24T18:11:25Z

@bdewater You'd be surprised.

$ /usr/sbin/system_profiler SPSoftwareDataType | grep "System Version"
      System Version: macOS 10.12.3 (16D32)
$ /usr/bin/openssl version                                  
OpenSSL 0.9.8zh 14 Jan 2016
$ /usr/bin/openssl ciphers | grep -I GCM

Obviously users can install newer OpenSSL versions with homebrew. Just pointing out that older OpenSSL can be endemic, and at the very least this does require users to install a new third-party dependency on their local machines in order to manipulate the config/secrets.yml.enc. Happy to remove the configuration though.

bdewater · 2017-02-24T21:37:39Z

Surprised indeed! I was expecting something like RHEL 6 lagging behind, but that's on 1.0.1 too.

jeremy · 2017-02-24T22:00:35Z

(Note that OpenSSL has been deprecated on macOS since 10.7, though. Ruby builders/installers all provide their own or link to Homebrew.)

kaspth · 2017-03-01T20:00:20Z

@stouset interested in trying out #28139 (comment)?

If you'd rather pass this on, I'd happy to take it over from here. You'd get full credit in every commit I make 😊

stouset · 2017-03-01T20:01:38Z

Yep, apologies. Happy to take this over the finish line.

stouset · 2017-03-01T20:49:21Z

Ok, this is done and tests pass.

That said, default for MessageEncryptor is still AES-256-CBC. I'm not sure the level of pain involved in migrating, but AES-128-GCM should be the default going forward. 256-bit AES is overkill and incurs a ~40% performance penalty, plus GCM itself is significantly faster (and less prone to a mistake) than a custom two-pass cipher+hmac.

Interestingly, SHA-1 as a digest for non-authenticated ciphers in MessageEncryptor isn't actually of concern, since it's used in an HMAC which only requires the underlying compression function to be weakly collision resistant. But it's probably totally possible to rip out the non-AEAD code entirely if we require Ruby 2.2+, which depends on OpenSSL >= 1.0.1 (and thus has access to GCM).

bdewater · 2017-03-01T20:54:25Z

railties/lib/rails/secrets.rb

-        cipher = new_cipher
-        SecureRandom.hex(cipher.key_len)[0, cipher.key_len]
+        cipher = OpenSSL::Cipher.new(CIPHER)
+        key = SecureRandom.random_bytes(cipher.key_len)


No need to unpack with SecureRandom.hex(cipher.key_len)

Fixes rails#28135.

stouset · 2017-03-01T22:55:59Z

@rafaelfranca Any further comments? This is good to go otherwise.

rafaelfranca

LGTM. @kaspth @matthewd anything else?

rafaelfranca · 2017-03-02T00:00:49Z

railties/lib/rails/secrets.rb

+        cipher = OpenSSL::Cipher.new(CIPHER)
+        key = SecureRandom.random_bytes(cipher.key_len)
+
+        key.unpack('H*').first


Just a small code style thing, this string should be double-quoted, but don't bother changing only this, I can do it.

Was already done. :)

Edit: whoops, apparently the change hadn't been pushed. It went a way as a result of using SecureRandom#hex.

kaspth · 2017-03-02T18:14:38Z

@stouset thanks so much for the help on this! It wasn't my intention to have encrypted secrets use sub par encryption, but I goofed on that. So it's very nice of you to jump in and help fix that mistake! ❤️

kaspth · 2017-03-02T18:17:41Z

As to the usability concerns brought up in #28139 (comment), you're right! I take what I said there back and we should do something about it. Currently we're debating fixing this through better names for the files.

That was the meat of your argument, right? That knowing secrets.yml isn't the same as secrets.yml.enc is tough for users?

wlipa · 2017-03-02T18:26:21Z

Consider having something like a secrets folder with a top level secrets file and then a folder for each environment. This will address the issue of different security needs for different envs.

At each level, you can either have secrets.yml (plaintext), or secrets.yml.enc (encrypted), but it would be an error to have both. That way, once an approach for the given env is chosen, it will be natural to keep the same method, which will reduce accidental check-in errors.

secrets/
  secrets.yml [or]
  secrets.yml.enc
  development/
    secrets.yml [or]
    secrets.yml.enc

The Rails.secrets would be the union of the top level secrets and the env specific secrets.

stouset · 2017-03-02T18:42:18Z

@kaspth No worries. Even I got it wrong a few times, and (in theory) I should know what I'm doing.

Yes, that's the meat of my argument. I don't necessarily have any overwhelmingly-good ideas — designing something is harder than tearing it down ;) and I've been out of the Rails ecosystem for a bit so I don't have a good intuition for what would be idiomatic here any more.

That can definitely be another ticket, though. Having a file named secrets that has secret-looking things in it is going to result in people putting real secrets there (speaking from experience here, I've seen this happen at previous companies).

Includes a script to ease an app's upgrade.

rafaelfranca added this to the 5.1.0 milestone Feb 23, 2017

rafaelfranca requested review from jeremy, matthewd, tenderlove and eileencodes February 23, 2017 22:59

rafaelfranca added railties security labels Feb 23, 2017

rafaelfranca requested changes Feb 23, 2017

View reviewed changes

matthewd reviewed Feb 23, 2017

View reviewed changes

stouset force-pushed the update-secrets-to-use-modern-crypto branch 4 times, most recently from 89740d0 to 4c82242 Compare February 23, 2017 23:43

ptoomey3 reviewed Feb 23, 2017

View reviewed changes

stouset force-pushed the update-secrets-to-use-modern-crypto branch 2 times, most recently from 54c55c5 to 0a043fe Compare February 23, 2017 23:58

ptoomey3 reviewed Feb 24, 2017

View reviewed changes

stouset force-pushed the update-secrets-to-use-modern-crypto branch 2 times, most recently from 5140ad1 to 4fbe02e Compare February 24, 2017 00:30

bdewater reviewed Feb 24, 2017

View reviewed changes

vipulnsward reviewed Feb 24, 2017

View reviewed changes

kaspth reviewed Feb 24, 2017

View reviewed changes

morgoth mentioned this pull request Feb 24, 2017

Add encrypted secrets #28038

Merged

3 tasks

pixeltrix mentioned this pull request Mar 1, 2017

Fix `new_cipher': uninitialized constant #<Class:Rails::Secrets>::OpenSSL (NameError) #28235

Closed

stouset force-pushed the update-secrets-to-use-modern-crypto branch 2 times, most recently from f056970 to 21ffacb Compare March 1, 2017 20:42

bdewater reviewed Mar 1, 2017

View reviewed changes

Default Secrets to AES-128-GCM, using ActiveSupport::MessageEncryptor

6aa6f9a

Fixes rails#28135.

rafaelfranca approved these changes Mar 2, 2017

View reviewed changes

stouset force-pushed the update-secrets-to-use-modern-crypto branch from 21ffacb to 6aa6f9a Compare March 2, 2017 00:39

jeremy approved these changes Mar 2, 2017

View reviewed changes

simi approved these changes Mar 2, 2017

View reviewed changes

kaspth merged commit 0203c37 into rails:master Mar 2, 2017

stouset deleted the update-secrets-to-use-modern-crypto branch March 2, 2017 18:39

kaspth added a commit that referenced this pull request Mar 2, 2017

[ci skip] Add changelog entry for #28139.

6a3235b

Includes a script to ease an app's upgrade.

kaspth mentioned this pull request Mar 21, 2017

5.1.0.rc1. Problem on heroku during rake db:migrate. #28508

Closed

stouset mentioned this pull request Jun 20, 2017

crypto.createCipher should not work with AES-CTR nodejs/node#13801

Closed

	# Currently the OpenSSL bindings do not raise an error if auth_tag is
	# truncated, which would allow an attacker to easily forge it. See
	# https://github.com/ruby/openssl/issues/63
	raise InvalidMessage if aead_mode? && auth_tag.bytes.length != 16

		@@ -140,6 +140,8 @@ defaults to `:debug` for all environments. The available log levels are: `:debug

		* `secrets.secret_key_base` is used for specifying a key which allows sessions for the application to be verified against a known secure key to prevent tampering. Applications get `secrets.secret_key_base` initialized to a random key present in `config/secrets.yml`.

		* `config.secrets_cipher` allows you to change the cipher used to protect the `config/secrets.yml` file. This defaults to `"aes-128-gcm"`. Versions of OpenSSL earlier than 1.0.01, unfortunately, don't support GCM mode, so degrading this to `"aes-128-cbc"` may be necessary. If you find yourself in this situation, strongly consider updating OpenSSL to a modern version if at all possible rather than weakening this setting.

Update secrets to use modern crypto #28139

Update secrets to use modern crypto #28139

Conversation

stouset commented Feb 23, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rafaelfranca commented Feb 23, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ptoomey3 Feb 23, 2017 • edited

Choose a reason for hiding this comment

stouset Feb 23, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bdewater Feb 24, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bdewater commented Feb 24, 2017 • edited

stouset commented Feb 24, 2017 • edited

kaspth commented Feb 24, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stouset Mar 1, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stouset Mar 1, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaspth commented Feb 24, 2017

bdewater commented Feb 24, 2017 • edited

stouset commented Feb 24, 2017 • edited

bdewater commented Feb 24, 2017

jeremy commented Feb 24, 2017

kaspth commented Mar 1, 2017

stouset commented Mar 1, 2017

stouset commented Mar 1, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stouset commented Mar 1, 2017

rafaelfranca left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stouset Mar 2, 2017 • edited

Choose a reason for hiding this comment

kaspth commented Mar 2, 2017

kaspth commented Mar 2, 2017

wlipa commented Mar 2, 2017 • edited

stouset commented Mar 2, 2017 • edited

ptoomey3 Feb 23, 2017 •

edited

stouset Feb 23, 2017 •

edited

bdewater Feb 24, 2017 •

edited

bdewater commented Feb 24, 2017 •

edited

stouset commented Feb 24, 2017 •

edited

stouset Mar 1, 2017 •

edited

stouset Mar 1, 2017 •

edited

bdewater commented Feb 24, 2017 •

edited

stouset commented Feb 24, 2017 •

edited

stouset commented Mar 1, 2017 •

edited

stouset Mar 2, 2017 •

edited

wlipa commented Mar 2, 2017 •

edited

stouset commented Mar 2, 2017 •

edited