Possibility to lock down versions of gems

[ctapobep]

Recently all the TestKitchen projects were failing because serverspec was updated. In order to escape this it's useful to be able to depend on particular version of busser, its plugins and libs like serverspec.

[joventuraz]

Hello, I just wanted to see if I could find a way to achieve what you are asking for.

So far, if you add in your .kitchen.yml the following:

busser:
  version: busser@0.6.0

and rename your choice of plugin to also include the version like the following:

test
`-- integration
    `-- myservice
        `-- serverspec@0.2.6
            |-- helper_spec.rb
            `-- myservice_spec.rb

Test-kitchen will install the busser version you set, and then busser will install the plugin version as well.

If you dig through the code, you can actually see that the names are being split by '@' to look for a version :)

If you want to take it even further, you can also add a Gemfile if the plugin supports it and install specific versions.

test
`-- integration
    `-- myservice
        `-- serverspec@0.2.6
            |-- helper_spec.rb
            `-- myservice_spec.rb
            `-- Gemfile

I think a cleaner way would be to make busser plugin versions against an attribute in the .kitchen.yml. For example:

busser:
  version: busser@0.6.0
  plugins: ['serverspec@0.2.6', 'minitest@x', 'bats@x', 'rspec@x']

Or maybe even make the busser-plugins set in .kitchen.yml have higher precedence than what busser discovers on its own.

[jtimberman]

This should be documented so it is clear to users what the capabilities are. CHEF maintains documentation about the .kitchen.yml here

[jtimberman]

The issue referenced above to track my comment a few minutes ago 😄

[ritterwolf]

Renaming the plugin directory didn't work for me:

-----> Setting up <appserver-centos-65>...
Fetching: thor-0.19.1.gem (100%)       
Fetching: busser-0.6.0.gem (100%)       
       Successfully installed thor-0.19.1
       Successfully installed busser-0.6.0
       2 gems installed
-----> Setting up Busser
       Creating BUSSER_ROOT in /tmp/busser
       Creating busser binstub
       Plugin serverspec installed (version 0.2.6)
-----> Running postinstall for serverspec plugin
       Finished setting up <appserver-centos-65> (0m51.72s).
-----> Verifying <appserver-centos-65>...
       Suite path directory /tmp/busser/suites does not exist, skipping.
       Uploading /tmp/busser/suites/serverspec@0.2.6/hosts_spec.rb (mode=0644)
       Uploading /tmp/busser/suites/serverspec@0.2.6/java_spec.rb (mode=0644)
-----> Running serverspec test suite
       /opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -I/tmp/busser/gems/gems/rspec-support-3.1.1/lib:/tmp/busser/gems/gems/rspec-core-3.1.5/lib /opt/chef/embedded/bin/rspec --pattern /tmp/busser/suites/serverspec/\*\*/\*_spec.rb --color --format documentation
       /tmp/busser/gems/gems/rspec-core-3.1.5/lib/rspec/core/configuration.rb:1105:in `load': cannot load such file -- /home/vagrant/spec (LoadError)
        from /tmp/busser/gems/gems/rspec-core-3.1.5/lib/rspec/core/configuration.rb:1105:in `block in load_spec_files'
        from /tmp/busser/gems/gems/rspec-core-3.1.5/lib/rspec/core/configuration.rb:1105:in `each'
        from /tmp/busser/gems/gems/rspec-core-3.1.5/lib/rspec/core/configuration.rb:1105:in `load_spec_files'
        from /tmp/busser/gems/gems/rspec-core-3.1.5/lib/rspec/core/runner.rb:96:in `setup'
        from /tmp/busser/gems/gems/rspec-core-3.1.5/lib/rspec/core/runner.rb:84:in `run'
        from /tmp/busser/gems/gems/rspec-core-3.1.5/lib/rspec/core/runner.rb:69:in `run'
        from /tmp/busser/gems/gems/rspec-core-3.1.5/lib/rspec/core/runner.rb:37:in `invoke'
        from /tmp/busser/gems/gems/rspec-core-3.1.5/exe/rspec:4:in `<top (required)>'
        from /opt/chef/embedded/bin/rspec:23:in `load'
        from /opt/chef/embedded/bin/rspec:23:in `<main>'
       /opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -I/tmp/busser/gems/gems/rspec-support-3.1.1/lib:/tmp/busser/gems/gems/rspec-core-3.1.5/lib /opt/chef/embedded/bin/rspec --pattern /tmp/busser/suites/serverspec/\*\*/\*_spec.rb --color --format documentation failed
       Ruby Script[/tmp/busser/gems/gems/busser-serverspec-0.2.6/lib/busser/runner_plugin/../serverspec/runner.rb /tmp/busser/suites/serverspec] exit code was 1
>>>>>> Verify failed on instance <appserver-centos-65>.

It seems that while the correct plugin version is loaded, it can't find the tests because the uploaded tests are in /tmp/busser/suites/serverspec@0.2.6 and it's looking for tests in /tmp/busser/suites/serverspec

[joventuraz]

@ritterwolf The version of serverspec 0.2.6 uses rspec 3 which requires the --default-path to be set in order to find the tests. Since this isn't being passed in (and causing the bug which was fixed recently), rspec is looking for a /home/vagrant/spec.

However, your point still stands as the tests are not in /tmp/busser/suites/serverspec

---

@jtimberman I would like to note that the plugins array for the .kitchen.yml has not been implemented and does not work. I suggested it as an idea and working on it. Having this would actually fix the issue from @ritterwolf

[joventuraz]

@ritterwolf A workaround for now is to use busser-rspec instead of busser-serverspec, and create a Gemfile that includes the correct version of the serverspec gem you want.

test
`-- integration
    `-- myservice
        `-- rspec
            |-- helper_spec.rb
            `-- myservice_spec.rb
            `-- Gemfile

[ritterwolf]

@joventuraz you are a lifesaver. Using rspec and a Gemfile seems to have done the trick for me.

[joventuraz]

@ritterwolf happy to help!

[joventuraz]

@jtimberman @ritterwolf I have submitted a pull request that would allow the plugins array in .kitchen.yml work.

When installing a busser-plugin, it will check if a version is specified by the array.

#518

[jtimberman]

Ah cool. Thanks!

@fnichol The PR #518 has a spec, too! Thoughts?

[cheeseplus]

fwiw - if using the rspec workaround, you might notice the output of serverspec gets buried by test-kitchen. To fix, add:

serverspec > 2

set :formatter, :documentation

serverspec < 2

RSpec.configure do |config|      
  config.formatter = :documentation
end

[martinb3]

👍 I'd love to lock down all gems and install new gems, using just one Gemfile as the input, or one list of gems with optional version specifiers.

[joventuraz]

I'm working on a way to be able to specify not only busser plugin versions, but gem versions as well. Maybe I'll even be able to fit in setting test suite locations too! If everything is set in the .kitchen.yml file, then kitchen diagnose will provide very useful information.

So far I'm thinking of something like this:

test:
  suites:
    -
      path: /path/to/serverspec/tests
      gem: serverspec, version: 2.0
    -
      path: /path/to/rspec/tests
      gem: rspec
      cli: '--some extra command'

Obviously its still an idea, and there can be many other things like include or exclude files, assigning a test suite to a suite, etc..

[cheeseplus]

This was fixed upstream test-kitchen/busser-serverspec@a41a6da