Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Secure "env" configuration #45

Closed
wants to merge 4 commits into from

8 participants

Steve Richert Josh Kalderimis Sven Fuchs Don't Add Me To Your Organization a.k.a The Travis Bot Raffael Schmid Eric Anderson Clemens Müller Pavel Perestoronin
Steve Richert

Hi guys,

I added the ability to public-key-encrypt the env configurations included in the build matrix. Here's why:

Let's say I have an open source Rails app that depends on connecting to an external service such as GitHub or RubyGems. I configure my app to keep those credentials in ENV (a la Heroku). In order to fully test on Travis, I need to include those credentials in the build env without exposing them publicly in .travis.yml.

I borrowed a page from the Campfire notification configuration, where you can encrypt your credentials.

I also cleaned up a bit of the code surrounding SecureConfig and its relation to SslKey.

I think this will open Travis up to many more public apps that rely on private configuration. Thanks and I hope it helps!

Josh Kalderimis
Owner
joshk commented

This is pretty AWESOME

Thank you sooooo much! I have been meaning to do this for ages!

What do you think @svenfuchs @rkh @mattmatt ?

Steve Richert

Thanks. I have a blog post in the works and it's going to be a call to action for people to open source their Rails apps. Being able to test them all on Travis would be a big win.

Josh Kalderimis
Owner
joshk commented
Steve Richert

I notice that people closed-source their Rails apps because they don't know what to do about private configurations like these. For apps that have to be private, Travis Pro is absolutely the right move. Otherwise, I think that apps that can be open sourced should be open sourced.

Sven Fuchs
Owner

looks good to me!

Steve Richert

Rebased against master and green. :green_heart:

Josh Kalderimis
Owner
joshk commented
Steve Richert

Understood, and no problem. Enjoy Railsberry!

Steve Richert

By the way, I added Travis secure env support to Figaro for when this is shipped. Enjoy RailsConf!

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request fails (merged 1cdc574d into 52fb894).

Steve Richert

Hogwash!

Josh Kalderimis
Owner
joshk commented
Steve Richert

Awesome, thank you!

Steve Richert

Rebased. :green_heart:

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request passes (merged d6fe181e into d1945fc).

Josh Kalderimis
Owner
joshk commented
Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request passes (merged 7e588dee into 046bde6).

Sven Fuchs
Owner

what's up with this? can we merge?

Josh Kalderimis
Owner
joshk commented
Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request passes (merged 3f013c8 into 181a8e2).

Steve Richert

Secure env configurations can now be detected (and obfuscated) by travis-worker.

Josh Kalderimis
Owner
joshk commented

Steve, regarding [:env, 'SECURE USE_GIT_REPOS=true'], how about instead [:env, { :secure => 'USE_GIT_REPOS=true' }] ?

Steve Richert

Woof. Okay, I'll have to get back to it another time.

Josh Kalderimis
Owner
joshk commented
Steve Richert

Understood, although this only detects those that begin with "SECURE " so it wouldn't match:

SECURE=asdf FOO=bar HELLO=world
Raffael Schmid
luxflux commented

Just a short question: How will you generate such a key? I saw your code in Figaro, it looks quite easy. Would it be possible to add a form on the website to generate such values? Or a Gem for the cli? (I don't want to use figaro in a non-rails app)...

Oleg Efimov Sannis referenced this pull request in travis-ci/travis-ci
Closed

environment variables #276

Henrik Hodne henrikhodne referenced this pull request in travis-ci/travis-ci
Closed

allow private ENV variables per project #400

Henrik Hodne henrikhodne referenced this pull request in travis-ci/travis-ci
Open

Build Artifacts #532

Eric Anderson

+1, would love this.

Clemens Müller

:heart_eyes: looking forward to this one ...

Josh Kalderimis joshk closed this
Josh Kalderimis
Owner
joshk commented

I am closing this as commits from this PR were included with the PR by @drogus which has since been merged.

Clemens Müller

Is this already usable or is this not yet available on travis-ci.org?

Pavel Perestoronin

+1 Is this usable now?

Steve Richert laserlemon referenced this pull request in laserlemon/figaro
Open

Travis configuration #8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
1  lib/travis.rb
@@ -51,6 +51,7 @@ module Travis
51 51 autoload :Mailer, 'travis/mailer'
52 52 autoload :Model, 'travis/model'
53 53 autoload :Notifications, 'travis/notifications'
  54 + autoload :SecureConfig, 'travis/secure_config'
54 55 autoload :Task, 'travis/task'
55 56
56 57 class << self
9 lib/travis/model/build/matrix.rb
@@ -99,12 +99,21 @@ def matrix_config
99 99 keys.inject([]) do |result, key|
100 100 values = config[key]
101 101 values = [values] unless values.is_a?(Array)
  102 + decrypt_env(values) if key == :env
102 103 values += [values.last] * (size - values.size) if values.size < size
103 104 result << values.map { |value| [key, value] }
104 105 end
105 106 end
106 107 end
107 108
  109 + def decrypt_env(values)
  110 + values.collect! do |value|
  111 + repository.key.secure.decrypt(value) do |env|
  112 + env.insert(0, 'SECURE ')
  113 + end
  114 + end
  115 + end
  116 +
108 117 def expand_matrix_config(config)
109 118 # recursively builds up permutations of values in the rows of a nested array
110 119 matrix = lambda do |*args|
2  lib/travis/model/build/notifications.rb
@@ -125,7 +125,7 @@ def default_email_recipients
125 125 end
126 126
127 127 def notifications
128   - Travis::Notifications::SecureConfig.decrypt((config || {}).fetch(:notifications, {}), repository.key)
  128 + repository.key.secure.decrypt((config || {}).fetch(:notifications, {}))
129 129 end
130 130 end
131 131 end
9 lib/travis/model/ssl_key.rb
... ... @@ -1,4 +1,5 @@
1 1 require 'openssl'
  2 +require 'base64'
2 3
3 4 # A Repository has an SSL key pair that is used to encrypt/decrypt sensitive
4 5 # data so it can be added to a public `.travis.yml` file (e.g. Campfire
@@ -12,6 +13,10 @@ class SslKey < ActiveRecord::Base
12 13
13 14 before_validation :generate_keys, :on => :create
14 15
  16 + def encode(string)
  17 + Base64.encode64(encrypt(string)).strip
  18 + end
  19 +
15 20 def encrypt(string)
16 21 build_key.public_encrypt(string)
17 22 end
@@ -33,6 +38,10 @@ def generate_keys!
33 38 generate_keys
34 39 end
35 40
  41 + def secure
  42 + Travis::SecureConfig.new(self)
  43 + end
  44 +
36 45 private
37 46
38 47 def build_key
1  lib/travis/notifications.rb
@@ -17,7 +17,6 @@ module Notifications
17 17 autoload :Handler, 'travis/notifications/handler'
18 18 autoload :Instrumentation, 'travis/notifications/instrumentation'
19 19 autoload :Subscription, 'travis/notifications/subscription'
20   - autoload :SecureConfig, 'travis/notifications/secure_config'
21 20
22 21 class << self
23 22 include Logging
66 lib/travis/notifications/secure_config.rb
... ... @@ -1,66 +0,0 @@
1   -require 'base64'
2   -
3   -module Travis
4   - module Notifications
5   -
6   - # Decrypts a single configuration value from a configuration file using the
7   - # repository's SSL key.
8   - #
9   - # This is used so people can add encrypted sensitive data to their
10   - # `.travis.yml` file.
11   - class SecureConfig
12   - def self.decrypt(config, key)
13   - self.new(key).decrypt(config)
14   - end
15   -
16   - attr_reader :key
17   -
18   - def initialize(key)
19   - @key = key
20   - end
21   -
22   - def decrypt(config)
23   - return config if config.is_a?(String)
24   -
25   - config.inject(config.class.new) do |result, element|
26   - key, element = element if result.is_a?(Hash)
27   - process(result, key, decrypt_element(key, element))
28   - end
29   - end
30   -
31   - private
32   -
33   - def decrypt_element(key, element)
34   - if element.is_a?(Array) || element.is_a?(Hash)
35   - decrypt(element)
36   - elsif key == :secure
37   - decrypt_value(element)
38   - else
39   - element
40   - end
41   - end
42   -
43   - def process(result, key, value)
44   - if result.is_a?(Array)
45   - result << value
46   - elsif result.is_a?(Hash) && !secure_key?(key)
47   - result[key] = value
48   - result
49   - else
50   - value
51   - end
52   - end
53   -
54   - def decrypt_value(value)
55   - decoded = Base64.decode64(value)
56   - key.decrypt(decoded)
57   - rescue OpenSSL::PKey::RSAError => e
58   - value
59   - end
60   -
61   - def secure_key?(key)
62   - key && key == :secure
63   - end
64   - end
65   - end
66   -end
60 lib/travis/secure_config.rb
... ... @@ -0,0 +1,60 @@
  1 +require 'base64'
  2 +
  3 +module Travis
  4 +
  5 + # Decrypts a single configuration value from a configuration file using the
  6 + # repository's SSL key.
  7 + #
  8 + # This is used so people can add encrypted sensitive data to their
  9 + # `.travis.yml` file.
  10 + class SecureConfig < Struct.new(:key)
  11 + def decrypt(config)
  12 + return config if config.is_a?(String)
  13 +
  14 + config.inject(config.class.new) do |result, element|
  15 + key, element = element if result.is_a?(Hash)
  16 + value = process(result, key, decrypt_element(key, element))
  17 + yield value if block_given?
  18 + value
  19 + end
  20 + end
  21 +
  22 + def encrypt(config)
  23 + { 'secure' => key.encode(config) }
  24 + end
  25 +
  26 + private
  27 +
  28 + def decrypt_element(key, element)
  29 + if element.is_a?(Array) || element.is_a?(Hash)
  30 + decrypt(element)
  31 + elsif key == :secure
  32 + decrypt_value(element)
  33 + else
  34 + element
  35 + end
  36 + end
  37 +
  38 + def process(result, key, value)
  39 + if result.is_a?(Array)
  40 + result << value
  41 + elsif result.is_a?(Hash) && !secure_key?(key)
  42 + result[key] = value
  43 + result
  44 + else
  45 + value
  46 + end
  47 + end
  48 +
  49 + def decrypt_value(value)
  50 + decoded = Base64.decode64(value)
  51 + key.decrypt(decoded)
  52 + rescue OpenSSL::PKey::RSAError => e
  53 + value
  54 + end
  55 +
  56 + def secure_key?(key)
  57 + key && key == :secure
  58 + end
  59 + end
  60 +end
31 spec/travis/model/build/matrix_spec.rb
@@ -188,6 +188,15 @@
188 188 ]
189 189 end
190 190
  191 + it 'decrypts a secure env configuration (single test config)' do
  192 + repository = Factory(:repository)
  193 + single_test_config['env'] = single_test_config.delete('env').map { |env| repository.key.secure.encrypt(env) }
  194 + build = Factory(:build, :config => single_test_config, :repository => repository)
  195 + build.expand_matrix_config(build.matrix_config.to_a).should == [
  196 + [[:rvm, '1.8.7'], [:gemfile, 'gemfiles/rails-3.0.6'], [:env, 'SECURE USE_GIT_REPOS=true']],
  197 + ]
  198 + end
  199 +
191 200 it 'expands the build matrix configuration (multiple tests config)' do
192 201 build = Factory(:build, :config => multiple_tests_config)
193 202 build.expand_matrix_config(build.matrix_config.to_a).should == [
@@ -298,6 +307,8 @@
298 307 end
299 308
300 309 describe :matrix_config do
  310 + let(:repository) { Factory(:repository) }
  311 +
301 312 it 'with string values' do
302 313 build = Factory(:build, :config => { :rvm => '1.8.7', :gemfile => 'gemfiles/rails-2.3.x', :env => 'FOO=bar' })
303 314 expected = [
@@ -308,6 +319,16 @@
308 319 build.matrix_config.should == expected
309 320 end
310 321
  322 + it 'strings with a secure env' do
  323 + build = Factory(:build, :repository => repository, :config => { :rvm => '1.8.7', :gemfile => 'gemfiles/rails-2.3.x', :env => repository.key.secure.encrypt('FOO=bar') })
  324 + expected = [
  325 + [[:rvm, '1.8.7']],
  326 + [[:gemfile, 'gemfiles/rails-2.3.x']],
  327 + [[:env, 'SECURE FOO=bar']]
  328 + ]
  329 + build.matrix_config.should == expected
  330 + end
  331 +
311 332 it 'with two Rubies and Gemfiles' do
312 333 build = Factory(:build, :config => { :rvm => ['1.8.7', '1.9.2'], :gemfile => ['gemfiles/rails-2.3.x', 'gemfiles/rails-3.0.x'] })
313 334 expected = [
@@ -333,6 +354,16 @@
333 354 [[:gemfile, 'gemfiles/rails-2.3.x'], [:gemfile, 'gemfiles/rails-2.3.x']]
334 355 ]
335 356 end
  357 +
  358 + it 'with secure and insecure envs' do
  359 + build = Factory(:build, :repository => repository, :config => { :rvm => '1.8.7', :gemfile => 'gemfiles/rails-2.3.x', :env => [repository.key.secure.encrypt('FOO=bar'), 'FOO=baz'] })
  360 + expected = [
  361 + [[:rvm, '1.8.7'], [:rvm, '1.8.7']],
  362 + [[:gemfile, 'gemfiles/rails-2.3.x'], [:gemfile, 'gemfiles/rails-2.3.x']],
  363 + [[:env, 'SECURE FOO=bar'], [:env, 'FOO=baz']]
  364 + ]
  365 + build.matrix_config.should == expected
  366 + end
336 367 end
337 368 end
338 369
6 spec/travis/notifications/secure_config_spec.rb → spec/travis/secure_config_spec.rb
... ... @@ -1,11 +1,11 @@
1 1 require 'spec_helper'
2 2 require 'support/active_record'
3 3
4   -describe Travis::Notifications::SecureConfig do
  4 +describe Travis::SecureConfig do
5 5
6 6 let(:key) { SslKey.new.tap { |key| key.generate_keys } }
7   - let(:secure_config) { Travis::Notifications::SecureConfig.new(key)}
8   - let(:crypted) { Base64.encode64(key.encrypt('hello world')) }
  7 + let(:secure_config) { Travis::SecureConfig.new(key)}
  8 + let(:crypted) { key.encode('hello world') }
9 9
10 10 it "returns the original value if the config is not a hash" do
11 11 secure_config.decrypt('hello world').should eql('hello world')

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.