Don't reset locale in Kitchen::Driver::Base run_command()

[mikelococo]

Problem Description

When using the kitchen-vagrant driver with boxes that use unicode characters in their metadata.json, test-kitchen dies with the following stack trace (notice at the top of the stack trace, file-content with the US-ASCII encoding is blowing up with an invalid character):

$ kitchen destroy
-----> Starting Kitchen (v1.2.1)
-----> Destroying <default-ubuntu-1404>...
       File content encoding: US-ASCII
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: Failed to complete #destroy action: [Expected process to exit with [0], but received '1'
---- Begin output of vagrant destroy -f ----
STDOUT: File content encoding: US-ASCII
STDERR: /home/mike/.vagrant.d/gems/gems/json-1.8.1/lib/json/common.rb:155:in `encode': "\xC3" on US-ASCII (Encoding::InvalidByteSequenceError)
        from /home/mike/.vagrant.d/gems/gems/json-1.8.1/lib/json/common.rb:155:in `initialize'
        from /home/mike/.vagrant.d/gems/gems/json-1.8.1/lib/json/common.rb:155:in `new'
        from /home/mike/.vagrant.d/gems/gems/json-1.8.1/lib/json/common.rb:155:in `parse'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/box.rb:69:in `initialize'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/box_collection.rb:299:in `new'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/box_collection.rb:299:in `block (3 levels) in find'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/box_collection.rb:284:in `each'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/box_collection.rb:284:in `block (2 levels) in find'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/box_collection.rb:277:in `each'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/box_collection.rb:277:in `block in find'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/box_collection.rb:416:in `block in with_collection_lock'
        from /opt/vagrant/embedded/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/box_collection.rb:415:in `with_collection_lock'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/box_collection.rb:263:in `find'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/vagrantfile.rb:157:in `block in machine_config'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/vagrantfile.rb:190:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/vagrantfile.rb:190:in `machine_config'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/vagrantfile.rb:45:in `machine'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/environment.rb:497:in `machine'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/plugin/v2/command.rb:168:in `block in with_target_vms'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/plugin/v2/command.rb:201:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/plugin/v2/command.rb:201:in `block in with_target_vms'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/plugin/v2/command.rb:200:in `map'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/plugin/v2/command.rb:200:in `with_target_vms'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/plugins/commands/destroy/command.rb:30:in `execute'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/cli.rb:42:in `execute'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/environment.rb:252:in `cli'
        from /opt/vagrant/bin/../embedded/gems/gems/vagrant-1.6.3/bin/vagrant:166:in `<main>'
---- End output of vagrant destroy -f ----
Ran vagrant destroy -f returned 1]
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

If I use --log-level debug to peak at the Vagrantfile test-kitchen is generating, and run vagrant up outside of test-kitchen using the same Vagrantfile, the creation completes without issue. This indicates that test-kitchen is changing the locale prior to calling vagrant.

Steps to Reproduce

I'm currently using test-kitchen with kitchen-vagrant as the driver, with vagrant-lxc as my vagrant provider and using https://github.com/fgrehm/vagrant-lxc-base-boxes as the basebox, but any vagrant basebox with unicode characters in the metadata.json for the box should trigger the bug. The metadata.json in this case looks like:

{
  "provider": "lxc",
  "version":  "1.0.0",
  "built-on": "Sáb Mai  3 14:33:09 UTC 2014"
}

Notice the unicode character in the "built-on" field, this is invalid in the US-ASCII encoding.

If you need a github repo that can be used to repro this using the setup that triggered it for me, I can set one up. Otherwise the following is a handy ruby-one-liner to show the relevant encodings (Vagrant uses File.open().read to read metadata.json): ruby -e 'puts Encoding.default_external; puts File.open("some_filename").read.encoding'

Additional Analysis

1. My workstation's locale is LANG=en_US.UTF-8
2. Both test-kitchen and vagrant sources have utf-8 "magic comments" in order to set the source encoding, and by implication the external encoding (Encode.default_external) to utf-8.
3. In test-kitchen, the two relevant methods seem to be Kitchen::Shellout's run_command and Kitchen::Driver::Base, which together are used to execute the vagrant driver comands.
4. The code which is incorrectly setting the locale to US-ASCII appears to be in Kitchen::Shellout's run_command which appears to have a workaround targeted at Ruby 1.8.x compatibility that hard-resets the locale to "C" unless an :environment hash item is passed in preventing that behavior. Is Ruby 1.8 even supported by test-kitchen anymore? Should that behavior be gutted?
5. If eliminating the ruby 1.8 workaround in Kitchen::Shellout isn't possible, one can override the behavior in this case simply by adding :environment => nil to the run_command call in Kitchen::Driver::Base.

[smurawski]

@mikelococo Thanks for the great bug report and sorry for the delayed response. Is this still an issue for you?

[mikelococo]

Thanks for checking in, it's no longer an issue. Both mixlib-shellout and test-kitchen have done some generally applicable work on improving UTF-8 support that resolved this bug as a side-effect.

Closing.