Unexpected failures with kitchen-inspec and inspec 1.0.0

[rhass]

Description

I have observed two misbehaviors when using the lastest version of Inspec (1.0.0 and master) via kitchen verify.

1. Context blocks throw Failed to complete #verify action: [undefined methodjoin' for nil:NilClass
   Did you mean? JSON]` See Stacktrace below.
2. Without context blocks, the tests succeed, however the test names are omitted.

Target:  ssh://kitchen@localhost:32834


  gemrc configuration
     ✔  
     ✔  

InSpec and Platform Version

• InSpec 1.0.0
• github: 'chef/inspec', ref: '593bdbc282f66cec5fc5e5c8f397ce171d40e055'
• ChefDK 0.18.30

Replication Case

1.Replace the version pinning here: https://github.com/chef-cookbooks/rubygems/blob/resource_driven/Gemfile#L9 with >= 0.15.2 and add a version pin here https://github.com/chef-cookbooks/rubygems/blob/resource_driven/Gemfile#L7 with >= 1.0.0.
2. Run bundle install
3. Run bundle exec integration:kitchen:bundle-config-centos-72 or bundle exec integration:kitchen:all

Possible Solutions

The current method of mitigation (not solution) is to use the following pin: gem "kitchen-inspec", "< 0.15.2", "~> 0.15"

Stacktrace

Please include the stacktrace output or link to a gist of it, if there is one.

D      ---Nested Exception---
D      Class: Kitchen::ActionFailed
D      Message: Failed to complete #verify action: [undefined method `join' for nil:NilClass
Did you mean?  JSON]
D      ----------------------
D      ------Backtrace-------
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/bundler/gems/inspec-593bdbc282f6/lib/inspec/rspec_json_formatter.rb:438:in `block (2 levels) in print_tests'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/bundler/gems/inspec-593bdbc282f6/lib/inspec/rspec_json_formatter.rb:431:in `each'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/bundler/gems/inspec-593bdbc282f6/lib/inspec/rspec_json_formatter.rb:431:in `block in print_tests'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/bundler/gems/inspec-593bdbc282f6/lib/inspec/rspec_json_formatter.rb:426:in `each'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/bundler/gems/inspec-593bdbc282f6/lib/inspec/rspec_json_formatter.rb:426:in `print_tests'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/bundler/gems/inspec-593bdbc282f6/lib/inspec/rspec_json_formatter.rb:297:in `close'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/rspec-core-3.5.3/lib/rspec/core/reporter.rb:201:in `block in notify'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/rspec-core-3.5.3/lib/rspec/core/reporter.rb:200:in `each'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/rspec-core-3.5.3/lib/rspec/core/reporter.rb:200:in `notify'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/rspec-core-3.5.3/lib/rspec/core/reporter.rb:226:in `close'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/rspec-core-3.5.3/lib/rspec/core/reporter.rb:189:in `close_after'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/rspec-core-3.5.3/lib/rspec/core/reporter.rb:168:in `finish'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/rspec-core-3.5.3/lib/rspec/core/reporter.rb:79:in `report'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/rspec-core-3.5.3/lib/rspec/core/runner.rb:111:in `run_specs'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/bundler/gems/inspec-593bdbc282f6/lib/inspec/runner_rspec.rb:77:in `run'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/bundler/gems/inspec-593bdbc282f6/lib/inspec/runner.rb:112:in `run_tests'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/bundler/gems/inspec-593bdbc282f6/lib/inspec/runner.rb:96:in `run'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/kitchen-inspec-0.15.2/lib/kitchen/verifier/inspec.rb:75:in `call'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/test-kitchen-1.13.2/lib/kitchen/instance.rb:430:in `block in verify_action'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/test-kitchen-1.13.2/lib/kitchen/instance.rb:520:in `synchronize_or_call'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/test-kitchen-1.13.2/lib/kitchen/instance.rb:485:in `block in action'
D      /opt/chefdk/embedded/lib/ruby/2.3.0/benchmark.rb:293:in `measure'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/test-kitchen-1.13.2/lib/kitchen/instance.rb:484:in `action'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/test-kitchen-1.13.2/lib/kitchen/instance.rb:422:in `verify_action'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/test-kitchen-1.13.2/lib/kitchen/instance.rb:355:in `block in transition_to'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/test-kitchen-1.13.2/lib/kitchen/instance.rb:354:in `each'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/test-kitchen-1.13.2/lib/kitchen/instance.rb:354:in `transition_to'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/test-kitchen-1.13.2/lib/kitchen/instance.rb:160:in `verify'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/test-kitchen-1.13.2/lib/kitchen/command.rb:201:in `public_send'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/test-kitchen-1.13.2/lib/kitchen/command.rb:201:in `run_action_in_thread'
D      /home/rhass/src/cookbooks/rubygems/.bundle/vendor/ruby/2.3.0/gems/test-kitchen-1.13.2/lib/kitchen/command.rb:173:in `block (2 levels) in run_action'
D      ----End Backtrace-----

[vjeffrey]

@rhass Looks like the cookbook you reference here has gone through some significant changes over the last week; could you please update the replication case details?

[rhass]

@vjeffrey The issues which I experienced were with the 1.x changes. I found the inspec bug while working on adding the inspec test framework to the cookbook to support the changes I was making.

[vjeffrey]

thanks; i'll look into it.

a couple things: the links posted above that refer to the gemfile throw 404s. the correct link is :https://github.com/chef-cookbooks/rubygems/blob/master/Gemfile#L9

also, bundle exec integration:kitchen:bundle-config-centos-72 does not seem to be a valid command. did you mean bundle exec rake integration:kitchen:bundle-config-centos-72?
that command also fails for me, though it seems to be a ruby related failure...but at least seems like a valid approach.... i'll keep trying..
i'm getting output like this: ```b rake integration:kitchen:compliance-centos-66

Kitchen error: You must first install the Docker CLI tool http://www.docker.io/gettingstarted/, omitting integration:
rake aborted!
Don't know how to build task 'integration:kitchen:compliance-centos-66' (see --tasks)
/opt/chefdk/embedded/bin/bundle:22:in load' /opt/chefdk/embedded/bin/bundle:22:in

'
(See full trace by running task with —trace)```

^^^ using .kitchen.local.yml to override the driver and kitchen-vagrant gem in the gemfile got around it.