New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update secrets to use modern crypto #28139

Merged
merged 1 commit into from Mar 2, 2017

Conversation

@stouset
Contributor

stouset commented Feb 23, 2017

Fixes #28135 by replacing the default mode with AES-128-GCM, allowing the mode to be configured manually, correctly generating keys, and using a random initialization vector on every encryption.

Show outdated Hide outdated ...ails/generators/rails/app/templates/config/environments/production.rb.tt
Show outdated Hide outdated railties/lib/rails/application/configuration.rb
Show outdated Hide outdated railties/test/secrets_test.rb
Show outdated Hide outdated railties/lib/rails/application/configuration.rb
@rafaelfranca

This comment has been minimized.

Show comment
Hide comment
@rafaelfranca

rafaelfranca Feb 23, 2017

Member

Thank you so much for the pull request.

Member

rafaelfranca commented Feb 23, 2017

Thank you so much for the pull request.

Show outdated Hide outdated railties/lib/rails/secrets.rb
Show outdated Hide outdated railties/lib/rails/secrets.rb
Show outdated Hide outdated railties/lib/rails/secrets.rb
Show outdated Hide outdated railties/lib/rails/secrets.rb
Show outdated Hide outdated railties/lib/rails/secrets.rb
@bdewater

This comment has been minimized.

Show comment
Hide comment
@bdewater

bdewater Feb 24, 2017

Contributor

Come to think of it, ActiveSupport is a dependency of Railties (unless I'm mistaken). Why wasn't MessageEncryptor used to implement this feature in the first place, or aren't we switching to it now? It already does everything (including AEAD since #25874) we're trying to fix here.

Contributor

bdewater commented Feb 24, 2017

Come to think of it, ActiveSupport is a dependency of Railties (unless I'm mistaken). Why wasn't MessageEncryptor used to implement this feature in the first place, or aren't we switching to it now? It already does everything (including AEAD since #25874) we're trying to fix here.

@stouset

This comment has been minimized.

Show comment
Hide comment
@stouset

stouset Feb 24, 2017

Contributor

@bdewater Now you tell me. :(

I can rewrite this to use MessageEncryptor if you'd prefer.

Contributor

stouset commented Feb 24, 2017

@bdewater Now you tell me. :(

I can rewrite this to use MessageEncryptor if you'd prefer.

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Feb 24, 2017

Member

@bdewater that was the plan, but with all the back and forth on the user experience changes I forgot to make the change.

Member

kaspth commented Feb 24, 2017

@bdewater that was the plan, but with all the back and forth on the user experience changes I forgot to make the change.

Show outdated Hide outdated guides/source/configuring.md
Show outdated Hide outdated railties/lib/rails/application/bootstrap.rb
@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Feb 24, 2017

Member

Thanks so much for the pull request! Reporting security issues very early in our release phase means a lot ❤️

Given your suggestions I think most of this could be boiled down to:

gem "openssl", ">= 1.0.01" # Enforce OpenSSL that supports GCM authenticated ciphers.
require "openssl"

require "active_support/message_encryptor"

# Later…

def generate_key
  encryptor.random_key.unpack("H*").first # Requires adding `random_key` to MessageEncryptor.
end

def encrypt(text)
  encryptor.encrypt_and_sign(text)
end

def decrypt(data)
  encryptor.decrypt_and_verify(data)
end

private
  def encryptor
    @encryptor ||= ActiveSuport::MessageEncryptor.new(key, cipher: "aes-128-gcm") # If we pick an `authenticated?` cipher the encryptor handles `auth_tag` checks, iv seed etc.
  end
Member

kaspth commented Feb 24, 2017

Thanks so much for the pull request! Reporting security issues very early in our release phase means a lot ❤️

Given your suggestions I think most of this could be boiled down to:

gem "openssl", ">= 1.0.01" # Enforce OpenSSL that supports GCM authenticated ciphers.
require "openssl"

require "active_support/message_encryptor"

# Later…

def generate_key
  encryptor.random_key.unpack("H*").first # Requires adding `random_key` to MessageEncryptor.
end

def encrypt(text)
  encryptor.encrypt_and_sign(text)
end

def decrypt(data)
  encryptor.decrypt_and_verify(data)
end

private
  def encryptor
    @encryptor ||= ActiveSuport::MessageEncryptor.new(key, cipher: "aes-128-gcm") # If we pick an `authenticated?` cipher the encryptor handles `auth_tag` checks, iv seed etc.
  end
@bdewater

This comment has been minimized.

Show comment
Hide comment
@bdewater

bdewater Feb 24, 2017

Contributor

OpenSSL (not the gem, the system library) 1.0.1 was released in March 2012 and support for it ended at the end of last year. Besides AES-GCM it also brought support for TLSv1.1 and 1.2. I think it's a fair assumption that somebody's server already has this or a newer version installed for modern internet crypto.

Contributor

bdewater commented Feb 24, 2017

OpenSSL (not the gem, the system library) 1.0.1 was released in March 2012 and support for it ended at the end of last year. Besides AES-GCM it also brought support for TLSv1.1 and 1.2. I think it's a fair assumption that somebody's server already has this or a newer version installed for modern internet crypto.

@morgoth morgoth referenced this pull request Feb 24, 2017

Merged

Add encrypted secrets #28038

3 of 3 tasks complete
@stouset

This comment has been minimized.

Show comment
Hide comment
@stouset

stouset Feb 24, 2017

Contributor

@bdewater You'd be surprised.

$ /usr/sbin/system_profiler SPSoftwareDataType | grep "System Version"
      System Version: macOS 10.12.3 (16D32)
$ /usr/bin/openssl version                                  
OpenSSL 0.9.8zh 14 Jan 2016
$ /usr/bin/openssl ciphers | grep -I GCM

Obviously users can install newer OpenSSL versions with homebrew. Just pointing out that older OpenSSL can be endemic, and at the very least this does require users to install a new third-party dependency on their local machines in order to manipulate the config/secrets.yml.enc. Happy to remove the configuration though.

Contributor

stouset commented Feb 24, 2017

@bdewater You'd be surprised.

$ /usr/sbin/system_profiler SPSoftwareDataType | grep "System Version"
      System Version: macOS 10.12.3 (16D32)
$ /usr/bin/openssl version                                  
OpenSSL 0.9.8zh 14 Jan 2016
$ /usr/bin/openssl ciphers | grep -I GCM

Obviously users can install newer OpenSSL versions with homebrew. Just pointing out that older OpenSSL can be endemic, and at the very least this does require users to install a new third-party dependency on their local machines in order to manipulate the config/secrets.yml.enc. Happy to remove the configuration though.

@bdewater

This comment has been minimized.

Show comment
Hide comment
@bdewater

bdewater Feb 24, 2017

Contributor

Surprised indeed! I was expecting something like RHEL 6 lagging behind, but that's on 1.0.1 too.

Contributor

bdewater commented Feb 24, 2017

Surprised indeed! I was expecting something like RHEL 6 lagging behind, but that's on 1.0.1 too.

@jeremy

This comment has been minimized.

Show comment
Hide comment
@jeremy

jeremy Feb 24, 2017

Member

(Note that OpenSSL has been deprecated on macOS since 10.7, though. Ruby builders/installers all provide their own or link to Homebrew.)

Member

jeremy commented Feb 24, 2017

(Note that OpenSSL has been deprecated on macOS since 10.7, though. Ruby builders/installers all provide their own or link to Homebrew.)

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Mar 1, 2017

Member

@stouset interested in trying out #28139 (comment)?

If you'd rather pass this on, I'd happy to take it over from here. You'd get full credit in every commit I make 😊

Member

kaspth commented Mar 1, 2017

@stouset interested in trying out #28139 (comment)?

If you'd rather pass this on, I'd happy to take it over from here. You'd get full credit in every commit I make 😊

@stouset

This comment has been minimized.

Show comment
Hide comment
@stouset

stouset Mar 1, 2017

Contributor

Yep, apologies. Happy to take this over the finish line.

Contributor

stouset commented Mar 1, 2017

Yep, apologies. Happy to take this over the finish line.

@stouset

This comment has been minimized.

Show comment
Hide comment
@stouset

stouset Mar 1, 2017

Contributor

Ok, this is done and tests pass.

That said, default for MessageEncryptor is still AES-256-CBC. I'm not sure the level of pain involved in migrating, but AES-128-GCM should be the default going forward. 256-bit AES is overkill and incurs a ~40% performance penalty, plus GCM itself is significantly faster (and less prone to a mistake) than a custom two-pass cipher+hmac.

Interestingly, SHA-1 as a digest for non-authenticated ciphers in MessageEncryptor isn't actually of concern, since it's used in an HMAC which only requires the underlying compression function to be weakly collision resistant. But it's probably totally possible to rip out the non-AEAD code entirely if we require Ruby 2.2+, which depends on OpenSSL >= 1.0.1 (and thus has access to GCM).

Contributor

stouset commented Mar 1, 2017

Ok, this is done and tests pass.

That said, default for MessageEncryptor is still AES-256-CBC. I'm not sure the level of pain involved in migrating, but AES-128-GCM should be the default going forward. 256-bit AES is overkill and incurs a ~40% performance penalty, plus GCM itself is significantly faster (and less prone to a mistake) than a custom two-pass cipher+hmac.

Interestingly, SHA-1 as a digest for non-authenticated ciphers in MessageEncryptor isn't actually of concern, since it's used in an HMAC which only requires the underlying compression function to be weakly collision resistant. But it's probably totally possible to rip out the non-AEAD code entirely if we require Ruby 2.2+, which depends on OpenSSL >= 1.0.1 (and thus has access to GCM).

Show outdated Hide outdated railties/lib/rails/secrets.rb
@stouset

This comment has been minimized.

Show comment
Hide comment
@stouset

stouset Mar 1, 2017

Contributor

@rafaelfranca Any further comments? This is good to go otherwise.

Contributor

stouset commented Mar 1, 2017

@rafaelfranca Any further comments? This is good to go otherwise.

@rafaelfranca

LGTM. @kaspth @matthewd anything else?

Show outdated Hide outdated railties/lib/rails/secrets.rb
@jeremy

jeremy approved these changes Mar 2, 2017

@simi

simi approved these changes Mar 2, 2017

@kaspth kaspth merged commit 0203c37 into rails:master Mar 2, 2017

2 checks passed

codeclimate no new or fixed issues
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Mar 2, 2017

Member

@stouset thanks so much for the help on this! It wasn't my intention to have encrypted secrets use sub par encryption, but I goofed on that. So it's very nice of you to jump in and help fix that mistake! ❤️

Member

kaspth commented Mar 2, 2017

@stouset thanks so much for the help on this! It wasn't my intention to have encrypted secrets use sub par encryption, but I goofed on that. So it's very nice of you to jump in and help fix that mistake! ❤️

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Mar 2, 2017

Member

As to the usability concerns brought up in #28139 (comment), you're right! I take what I said there back and we should do something about it. Currently we're debating fixing this through better names for the files.

That was the meat of your argument, right? That knowing secrets.yml isn't the same as secrets.yml.enc is tough for users?

Member

kaspth commented Mar 2, 2017

As to the usability concerns brought up in #28139 (comment), you're right! I take what I said there back and we should do something about it. Currently we're debating fixing this through better names for the files.

That was the meat of your argument, right? That knowing secrets.yml isn't the same as secrets.yml.enc is tough for users?

@wlipa

This comment has been minimized.

Show comment
Hide comment
@wlipa

wlipa Mar 2, 2017

Consider having something like a secrets folder with a top level secrets file and then a folder for each environment. This will address the issue of different security needs for different envs.

At each level, you can either have secrets.yml (plaintext), or secrets.yml.enc (encrypted), but it would be an error to have both. That way, once an approach for the given env is chosen, it will be natural to keep the same method, which will reduce accidental check-in errors.

secrets/
  secrets.yml [or]
  secrets.yml.enc
  development/
    secrets.yml [or]
    secrets.yml.enc

The Rails.secrets would be the union of the top level secrets and the env specific secrets.

wlipa commented Mar 2, 2017

Consider having something like a secrets folder with a top level secrets file and then a folder for each environment. This will address the issue of different security needs for different envs.

At each level, you can either have secrets.yml (plaintext), or secrets.yml.enc (encrypted), but it would be an error to have both. That way, once an approach for the given env is chosen, it will be natural to keep the same method, which will reduce accidental check-in errors.

secrets/
  secrets.yml [or]
  secrets.yml.enc
  development/
    secrets.yml [or]
    secrets.yml.enc

The Rails.secrets would be the union of the top level secrets and the env specific secrets.

@stouset stouset deleted the stouset:update-secrets-to-use-modern-crypto branch Mar 2, 2017

@stouset

This comment has been minimized.

Show comment
Hide comment
@stouset

stouset Mar 2, 2017

Contributor

@kaspth No worries. Even I got it wrong a few times, and (in theory) I should know what I'm doing.

Yes, that's the meat of my argument. I don't necessarily have any overwhelmingly-good ideas — designing something is harder than tearing it down ;) and I've been out of the Rails ecosystem for a bit so I don't have a good intuition for what would be idiomatic here any more.

That can definitely be another ticket, though. Having a file named secrets that has secret-looking things in it is going to result in people putting real secrets there (speaking from experience here, I've seen this happen at previous companies).

Contributor

stouset commented Mar 2, 2017

@kaspth No worries. Even I got it wrong a few times, and (in theory) I should know what I'm doing.

Yes, that's the meat of my argument. I don't necessarily have any overwhelmingly-good ideas — designing something is harder than tearing it down ;) and I've been out of the Rails ecosystem for a bit so I don't have a good intuition for what would be idiomatic here any more.

That can definitely be another ticket, though. Having a file named secrets that has secret-looking things in it is going to result in people putting real secrets there (speaking from experience here, I've seen this happen at previous companies).

kaspth added a commit that referenced this pull request Mar 2, 2017

[ci skip] Add changelog entry for #28139.
Includes a script to ease an app's upgrade.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment