Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After upgrading to test-kitchen 1.7.1, openstack fails #997

Closed
JoeTalbot opened this issue Apr 5, 2016 · 10 comments
Closed

After upgrading to test-kitchen 1.7.1, openstack fails #997

JoeTalbot opened this issue Apr 5, 2016 · 10 comments

Comments

@JoeTalbot
Copy link

Description

We updated to chefdk 0.12.0 and test-kitchen 1.7.1. After doing that i am getting a failure when using the openstack provisioner. I am running a windows 10 machine.

Kitchen Version

kitchen --version
Test Kitchen version 1.7.1

ChefDK Version

> chef --version
Chef Development Kit Version: 0.12.0
chef-client version: 12.8.1
berks version: 4.3.0
kitchen version: 1.7.1

Ruby Version

ruby --version
ruby 2.1.6p336 (2015-04-13 revision 50298) [i386-mingw32]

Platform Version

I am running windows 10, with chefdk 0.12.0, and upgraded to test-kitchen 1.7.1 because we had issues with test-kitchen 1.6.0.

Replication Case

I have an existing cookbook that worked with test-kitchen 1.5.0 and the openstack provisioner. After updating i get the following error after doing a kitchen destroy and kitchen converge.

Kitchen Output

-----> Starting Kitchen (v1.7.1)
D      Berksfile found at C:/Users/z069057/development/fftrack/Berksfile, loading Berkshelf
D      Berkshelf 4.3.1 library loaded
D      winrm requested, loading winrm gem (["~> 1.6"])
D      winrm is loaded.
D      winrm-fs requested, loading winrm-fs gem (["~> 0.4.0"])
D      winrm-fs is loaded.
-----> Creating <default-WindowsServer2012R2>...
D      Selected image: 65c54a1d-9632-4f6b-8737-acc8fbee08e7 Windows-Server-2012-std-fast
D      Selected flavor: 9e4561d1-7994-4be8-998f-e5ba9d75be5a lmem-4vcpu
D      Selected net: 26b39da3-7272-418e-bd7e-4827eedf8209 kitchen_network
       OpenStack instance with ID of <17c1bd15-58b6-4eb9-905b-41b39ce5f497> is ready.
D      Waiting for VM to be in ACTIVE state for a max time of:600 seconds
       Attaching floating IP from <ext_vlan1767_net> pool
       Attaching floating IP <10.63.216.100>
D      Using floating ip: 10.63.216.100
       Waiting for server to be ready...
>>>>>> Server 10.63.216.100 (17c1bd15-58b6-4eb9-905b-41b39ce5f497) not reachable. Destroying server...
       OpenStack instance <17c1bd15-58b6-4eb9-905b-41b39ce5f497> destroyed.
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: Failed to complete #create action: [undefined method `split' for nil:NilClass]
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

Kitchen Diagnose

https://gist.github.com/JoeTalbot/7121b7d74f4b1437938a671020803822

NOTE:

This repository is for core issues with the Test-Kitchen framework, so if the issue is with a plugin you may be referred to file the issue with the appropriate plugin.

@mwrock
Copy link
Member

mwrock commented Apr 5, 2016

We'll need a debug log to see where that split is coming from. Can you run with -l debug and include the stacktrace? Thanks!

@JoeTalbot
Copy link
Author

> kitchen converge -l debug
-----> Starting Kitchen (v1.7.1)
D      Berksfile found at C:/Users/z069057/development/fftrack/Berksfile, loading Berkshelf
D      Berkshelf 4.3.1 library loaded
D      winrm requested, loading winrm gem (["~> 1.6"])
D      winrm is loaded.
D      winrm-fs requested, loading winrm-fs gem (["~> 0.4.0"])
D      winrm-fs is loaded.
-----> Creating <default-WindowsServer2012R2>...
D      Selected image: 65c54a1d-9632-4f6b-8737-acc8fbee08e7 Windows-Server-2012-std-fast
D      Selected flavor: 9e4561d1-7994-4be8-998f-e5ba9d75be5a lmem-4vcpu
D      Selected net: 26b39da3-7272-418e-bd7e-4827eedf8209 kitchen_network
       OpenStack instance with ID of <fd4d2569-1d8a-4e98-a8dc-d9d86af858c1> is ready.
D      Waiting for VM to be in ACTIVE state for a max time of:600 seconds
       Attaching floating IP from <ext_vlan1767_net> pool
       Attaching floating IP <10.63.216.100>
D      Using floating ip: 10.63.216.100
       Waiting for server to be ready...
>>>>>> Server 10.63.216.100 (fd4d2569-1d8a-4e98-a8dc-d9d86af858c1) not reachable. Destroying server...
       OpenStack instance <fd4d2569-1d8a-4e98-a8dc-d9d86af858c1> destroyed.
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: Failed to complete #create action: [undefined method `split' for nil:NilClass]
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

D      ------Exception-------
D      Class: Kitchen::ActionFailed
D      Message: Failed to complete #create action: [undefined method `split' for nil:NilClass]
D      ---Nested Exception---
D      Class: NoMethodError
D      Message: undefined method `split' for nil:NilClass
D      ------Backtrace-------
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/winrm-1.7.3/lib/winrm/http/transport.rb:226:in `init_auth'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/winrm-1.7.3/lib/winrm/http/transport.rb:166:in `send_request'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/winrm-1.7.3/lib/winrm/winrm_service.rb:489:in `send_message'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/winrm-1.7.3/lib/winrm/winrm_service.rb:390:in `run_wql'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/winrm-1.7.3/lib/winrm/command_executor.rb:186:in `os_version'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/winrm-1.7.3/lib/winrm/command_executor.rb:145:in `code_page'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/winrm-1.7.3/lib/winrm/command_executor.rb:72:in `block in open'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/winrm-1.7.3/lib/winrm/command_executor.rb:218:in `retryable'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/winrm-1.7.3/lib/winrm/command_executor.rb:71:in `open'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/winrm-1.7.3/lib/winrm/winrm_service.rb:356:in `create_executor'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/transport/winrm.rb:321:in `session'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/transport/winrm.rb:135:in `wait_until_ready'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/kitchen-openstack-3.0.0/lib/kitchen/driver/openstack.rb:382:in `wait_for_server'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/kitchen-openstack-3.0.0/lib/kitchen/driver/openstack.rb:111:in `create'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:449:in `public_send'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:449:in `block in perform_action'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:513:in `call'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:513:in `synchronize_or_call'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:478:in `block in action'
D      C:/opscode/chefdk/embedded/lib/ruby/2.1.0/benchmark.rb:279:in `measure'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:477:in `action'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:449:in `perform_action'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:359:in `create_action'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:348:in `block in transition_to'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:347:in `each'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:347:in `transition_to'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:138:in `converge'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/command.rb:176:in `public_send'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/command.rb:176:in `block (2 levels) in run_action'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/logging-2.1.0/lib/logging/diagnostic_context.rb:450:in `call'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/logging-2.1.0/lib/logging/diagnostic_context.rb:450:in `block in create_with_logging_context'

@mwrock
Copy link
Member

mwrock commented Apr 5, 2016

Thanks for this info! This exception is coming from WinRM's attempt to authenticate via NTLM with the created instance. It engages in a typical auth handshake sending an auth initialization message and then expects a response with a WWW-Authenticate header included. The above exception comes from the fact that such a header is not included.

Clearly the WinRM gem could use a MUCH better error message here. That aside, I'm wondering if there is a proxy standing between your Test-Kitchen host and the created test instance that is "hijacking" this handshake? If so, you could try setting the HTTP_PROXY environment variable to point to your proxy.

Another thing you could do to workaround this error is force basic authentication by setting the transport's winrm_transport config setting to plaintext:

The yaml would look like this:

transport:
  name: winrm
  winrm_transport: plaintext

Prior to 1.6.0, we used basic auth by default but 1.6 forward leverages a ruby implementation of ntlm which can provide a much more secure auth story over basic auth. I'd really be curious to know if there is a proxy at play here.

@JoeTalbot
Copy link
Author

Thank you for the quick response. I am assuming our proxy is involved I already have the HTTP_PROXY pointed to our proxy, we actually have 2 and i have tried both, and i still get the error from above.

However when i add the:

transport:
  name: winrm
  winrm_transport: plaintext

to my kitchen.yml file, that fixed the issue for me. Thanks!

@mwrock
Copy link
Member

mwrock commented Apr 5, 2016

glad you are unblocked! closing here. sounds like any potential issues should live in the winrm gem.

@mwrock mwrock closed this as completed Apr 5, 2016
@JoeTalbot
Copy link
Author

Sorry, i spoke too soon, now it throws an error at the end of the run. Everything completes but it ends with an error:

         * execute[Add Target_Policy_CA01.pem to Target CA bundle] action run
           - execute cat Target_Policy_CA01.pem >> cacert.pem
         * env[SSL_CERT_FILE] action create (up to date)
       Recipe: certs_tgt::windows
         * cookbook_file[C:/opscode/chef/embedded/ssl/certs/Target_Root.pem] action create_if_missing (up to date)
         * powershell_script[add_target_root_to_cert_store] action run
           - execute "C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe" -NoLogo -NonInteractive -NoProfile -ExecutionPolicy Bypass -InputFormat None -File "C:/Users/OPENST~1/AppData/Local/Temp/chef-script20160405-1668-lp0yt1.ps1"
       Recipe: fftrack::default
         * directory[c:\temp\z069057] action create
           - create new directory c:\temp\z069057
         * directory[D:\fftrack\files] action create

         Running handlers:
       [2016-04-05T21:39:23-07:00] ERROR: Running exception handlers
         Running handlers complete
       [2016-04-05T21:39:23-07:00] ERROR: Exception handlers complete
         Chef Client failed. 16 resources updated in 03 minutes 51 seconds
       [2016-04-05T21:39:23-07:00] FATAL: Stacktrace dumped to C:/Users/OPENST~1/AppData/Local/Temp/kitchen/cache/chef-stacktrace.out
       [2016-04-05T21:39:23-07:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2016-04-05T21:39:23-07:00] FATAL: NoMethodError: private method `select' called for nil:NilClass

D      Cleaning up local sandbox in C:/Users/z069057/AppData/Local/Temp/default-WindowsServer2012R2-sandbox-20160405-11688-2ip67e
>>>>>> Converge failed on instance <default-WindowsServer2012R2>.
>>>>>> Please see .kitchen/logs/default-WindowsServer2012R2.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: WinRM exited (1) for command: [$env:http_proxy = "http://proxy-mdha.target.com:8080"
$env:HTTP_PROXY = "http://proxy-mdha.target.com:8080"
$env:https_proxy = "http://proxy-mdha.target.com:8080"
$env:HTTPS_PROXY = "http://proxy-mdha.target.com:8080"
$env:no_proxy = "127.0.0.1,localhost,10.*,target.com,corp.target.com,hq.target.com,dist.target.com,Email.target.com,stores.target.com,labs.target.com"
$env:NO_PROXY = "127.0.0.1,localhost,10.*,target.com,corp.target.com,hq.target.com,dist.target.com,Email.target.com,stores.target.com,labs.target.com"
            $script = @'
Add-Type -AssemblyName System.Core
start-sleep -seconds 5;
`$npipeClient = new-object System.IO.Pipes.NamedPipeClientStream(`$env:ComputerName,
  `'task`', [System.IO.Pipes.PipeDirection]::Out);
`$npipeclient.connect();
`$pipeWriter = new-object System.IO.StreamWriter(`$npipeClient);
`$pipeWriter.AutoFlush = `$true;
& C:\opscode\chef\bin\chef-client.bat --local-mode --config $env:TEMP\kitchen\client.rb --log_level auto --force-formatter --no-color --json-attributes $env:TEMP\kitchen\dna.json --chef-zero-port 8889 |
  foreach-object {} {`$pipewriter.writeline(`$_)} {
    `$pipewriter.writeline("SCHEDULED_TASK_DONE: `$LastExitCode");
    `$pipewriter.dispose();
    `$npipeclient.dispose()
  }

'@
            $ExecutionContext.InvokeCommand.ExpandString($Script) |
              out-file "$env:temp/kitchen/chef-client-script.ps1"
            try {
  Add-Type -AssemblyName System.Core
  $npipeServer = new-object System.IO.Pipes.NamedPipeServerStream('task',
    [System.IO.Pipes.PipeDirection]::In)
  $pipeReader = new-object System.IO.StreamReader($npipeServer)
  schtasks /run /tn "chef-tk" /i
  $npipeserver.waitforconnection()
  $host.ui.writeline('Connected to the scheduled task.')
  while ($npipeserver.IsConnected) {
    $output = $pipereader.ReadLine()
    if ($output -like 'SCHEDULED_TASK_DONE:*') {
      $exit_code = ($output -replace 'SCHEDULED_TASK_DONE:').trim()
    }
    else { $host.ui.WriteLine($output) } } }
finally {
  $pipereader.dispose()
  $npipeserver.dispose()
  $host.setshouldexit($exit_code)
}

]
>>>>>> ----------------------
D      Converge failed on instance <default-WindowsServer2012R2>.
D      ------Exception-------
D      Class: Kitchen::InstanceFailure
D      Message: Converge failed on instance <default-WindowsServer2012R2>.  Please see .kitchen/logs/default-WindowsServer2012R2.log for more details
D      ---Nested Exception---
D      Class: Kitchen::ActionFailed
D      Message: WinRM exited (1) for command: [$env:http_proxy = "http://proxy-mdha.target.com:8080"
$env:HTTP_PROXY = "http://proxy-mdha.target.com:8080"
$env:https_proxy = "http://proxy-mdha.target.com:8080"
$env:HTTPS_PROXY = "http://proxy-mdha.target.com:8080"
$env:no_proxy = "127.0.0.1,localhost,10.*,target.com,corp.target.com,hq.target.com,dist.target.com,Email.target.com,stores.target.com,labs.target.com"
$env:NO_PROXY = "127.0.0.1,localhost,10.*,target.com,corp.target.com,hq.target.com,dist.target.com,Email.target.com,stores.target.com,labs.target.com"
            $script = @'
Add-Type -AssemblyName System.Core
start-sleep -seconds 5;
`$npipeClient = new-object System.IO.Pipes.NamedPipeClientStream(`$env:ComputerName,
  `'task`', [System.IO.Pipes.PipeDirection]::Out);
`$npipeclient.connect();
`$pipeWriter = new-object System.IO.StreamWriter(`$npipeClient);
`$pipeWriter.AutoFlush = `$true;
& C:\opscode\chef\bin\chef-client.bat --local-mode --config $env:TEMP\kitchen\client.rb --log_level auto --force-formatter --no-color --json-attributes $env:TEMP\kitchen\dna.json --chef-zero-port 8889 |
  foreach-object {} {`$pipewriter.writeline(`$_)} {
    `$pipewriter.writeline("SCHEDULED_TASK_DONE: `$LastExitCode");
    `$pipewriter.dispose();
    `$npipeclient.dispose()
  }

'@
            $ExecutionContext.InvokeCommand.ExpandString($Script) |
              out-file "$env:temp/kitchen/chef-client-script.ps1"
            try {
  Add-Type -AssemblyName System.Core
  $npipeServer = new-object System.IO.Pipes.NamedPipeServerStream('task',
    [System.IO.Pipes.PipeDirection]::In)
  $pipeReader = new-object System.IO.StreamReader($npipeServer)
  schtasks /run /tn "chef-tk" /i
  $npipeserver.waitforconnection()
  $host.ui.writeline('Connected to the scheduled task.')
  while ($npipeserver.IsConnected) {
    $output = $pipereader.ReadLine()
    if ($output -like 'SCHEDULED_TASK_DONE:*') {
      $exit_code = ($output -replace 'SCHEDULED_TASK_DONE:').trim()
    }
    else { $host.ui.WriteLine($output) } } }
finally {
  $pipereader.dispose()
  $npipeserver.dispose()
  $host.setshouldexit($exit_code)
}

]
D      ------Backtrace-------
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/provisioner/base.rb:76:in `rescue in call'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/provisioner/base.rb:79:in `call'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:373:in `block in converge_action'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:513:in `call'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:513:in `synchronize_or_call'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:478:in `block in action'
D      C:/opscode/chefdk/embedded/lib/ruby/2.1.0/benchmark.rb:279:in `measure'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:477:in `action'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:369:in `converge_action'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:348:in `block in transition_to'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:347:in `each'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:347:in `transition_to'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/instance.rb:138:in `converge'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/command.rb:176:in `public_send'
D      C:/Users/z069057/AppData/Local/chefdk/gem/ruby/2.1.0/gems/test-kitchen-1.7.1/lib/kitchen/command.rb:176:in `block (2 levels) in run_action'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/logging-2.1.0/lib/logging/diagnostic_context.rb:450:in `call'
D      C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/logging-2.1.0/lib/logging/diagnostic_context.rb:450:in `block in create_with_logging_context'
D      ----------------------
D      [WinRM] closing remote shell 047F623A-CD93-4F43-A114-5BDBDD842838 on http://10.63.216.100:5985/wsman
D      [WinRM] remote shell 047F623A-CD93-4F43-A114-5BDBDD842838 closed

@mwrock
Copy link
Member

mwrock commented Apr 5, 2016

ah ok. looks like you are using the chef-zero-scheduled-task provisioner. So the stack trace soes not include the "true" error. But you should be able to get that on the test instance. Note these lines in the output:

   [2016-04-05T21:39:23-07:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2016-04-05T21:39:23-07:00] FATAL: NoMethodError: private method `select' called for nil:NilClass

Could you add that stacktrace here?

@mwrock mwrock reopened this Apr 5, 2016
@JoeTalbot
Copy link
Author

Here is the stacktrace:

Generated at 2016-04-06 18:09:23 -0700
NoMethodError: private method `select' called for nil:NilClass
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/formatters/error_inspectors/resource_failure_inspector.rb:102:in `filtered_bt'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/formatters/error_inspectors/resource_failure_inspector.rb:38:in `add_explanation'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/formatters/error_mapper.rb:72:in `resource_failed'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/formatters/base.rb:139:in `resource_failed'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/formatters/doc.rb:293:in `resource_failed'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/event_dispatch/dispatcher.rb:43:in `call'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/event_dispatch/dispatcher.rb:43:in `block in call_subscribers'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/event_dispatch/dispatcher.rb:34:in `each'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/event_dispatch/dispatcher.rb:34:in `call_subscribers'
(eval):2:in `resource_failed'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/resource.rb:608:in `rescue in run_action'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/resource.rb:594:in `run_action'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/runner.rb:73:in `run_action'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/runner.rb:105:in `block (2 levels) in converge'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/runner.rb:105:in `each'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/runner.rb:105:in `block in converge'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/resource_collection/resource_list.rb:84:in `block in execute_each_resource'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/resource_collection/resource_list.rb:82:in `execute_each_resource'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/runner.rb:104:in `converge'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/client.rb:668:in `block in converge'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/client.rb:663:in `catch'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/client.rb:663:in `converge'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/client.rb:702:in `converge_and_save'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/client.rb:280:in `run'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/application.rb:252:in `run_with_graceful_exit_option'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/application.rb:228:in `block in run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/local_mode.rb:44:in `with_server_connectivity'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/application.rb:211:in `run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/application/client.rb:445:in `block in interval_run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/application/client.rb:435:in `loop'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/application/client.rb:435:in `interval_run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/application/client.rb:424:in `run_application'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/lib/chef/application.rb:58:in `run'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.8.1-universal-mingw32/bin/chef-client:26:in `<top (required)>'
C:/opscode/chef/bin/chef-client:60:in `load'
C:/opscode/chef/bin/chef-client:60:in `<main>'

Also, i noticed when i am waiting for WINRM to connect to openstack it displays this on the screen

       [WinRM] connection failed. retrying in 3 seconds: #<WinRM::WinRMHTTPTransportError: Unable to parse WinRM response: Missing end tag for 'BR' (got "FONT")
Line: 7
Position: 129
Last 80 unconsumed characters:
 (503).>
       [WinRM] connection failed. retrying in 3 seconds: #<WinRM::WinRMHTTPTransportError: Unable to parse WinRM response: Missing end tag for 'BR' (got "FONT")
Line: 7
Position: 129
Last 80 unconsumed characters:
 (503).>
       [WinRM] connection failed. retrying in 3 seconds: #<WinRM::WinRMHTTPTransportError: Unable to parse WinRM response: Missing end tag for 'BR' (got "FONT")
Line: 7
Position: 129
Last 80 unconsumed characters:
 (503).>
       [WinRM] connection failed. retrying in 3 seconds: #<WinRM::WinRMHTTPTransportError: Unable to parse WinRM response: Missing end tag for 'BR' (got "FONT")
Line: 7
Position: 129
Last 80 unconsumed characters:
 (503).>
       [WinRM] Established

       Adding OpenStack hint for ohai


           Directory: C:\chef\ohai\hints


       Mode                LastWriteTime     Length Name
       ----                -------------     ------ ----
       -a---          4/6/2016   6:02 PM          2 openstack.json

Ultimately it connects and works just fine, i am just curious about the wirnrm error.

@mwrock
Copy link
Member

mwrock commented Apr 6, 2016

Ugh. That looks like an error inside of the chef formatter that is expecting a backtrace from a resource error and does not see it. Hard to say from this data what's going on there but it doesn't seem test-kitchen related.

The WinRM errors you see below are likely related somehow to your proxy. That is WinRM saying, "I sent a SOAP request to a Winrm endpoint, but just got back some junk HTML." This is common when it is instead getting responses from a proxy.

It is strange that it "self-corrects" after a few tries. I have 0 experience with openstack environments so there could be other things at play that I'm just not familiar with.

@JoeTalbot
Copy link
Author

After a little more testing, i found that i was trying to create a directory before a drive was mounted and formatted, but instead of throwing the correct error for the directory resource it bombed out with that issue. Changing from the chef_zero_scheduled_task provisioner to chef_zero fixed the issue for me. I will open an issue for the scheduled task provisoner, thank you for your help @mwrock!

@test-kitchen test-kitchen locked and limited conversation to collaborators Nov 16, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants