New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UndefinedConversionError: "\xEF" from ASCII-8BIT to UTF-8 #276

Closed
seththoenen opened this Issue Sep 6, 2017 · 7 comments

Comments

Projects
None yet
3 participants
@seththoenen

seththoenen commented Sep 6, 2017

Cookbook version

4.0.0, 4.2.0, 5.0.0

Chef-client version

13.3.42

Platform Details

Windows Server 2008 R2 Standard, 64 bit.

Scenario:

We are running the chef-client and audit cookbooks on 100+ of our Windows Server 2008 R2 systems and the chef-client run is failing during the audit report. The audits execute successfully, but won't report into Chef Automate (due to the Chef run blowing up).

Steps to Reproduce:

This only happens sometimes. It's happening all over production, but not in our lab environments (possibly a difference in system templates or possibly Active Directory configurations). We are only experiencing this issue with Windows Server 2008 R2 64-bit, in conjunction with the audit cookbook. Removing the audit cookbook from the run list will yield a successful Chef-client run. The audit cookbook is configured to grab compliance profiles from Chef Automate through the Chef Server.

Expected Result:

You should get an error similar to UndefinedConversionError: "\xEF" from ASCII-8BIT to UTF-8

Note: I found this code in the Windows cookbook that may assist in troubleshooting this issue. It seems to address the \xEF conversion on Windows: https://github.com/chef-cookbooks/windows/blob/master/resources/certificate_binding.rb#L126

Actual Result:

Generated at 2017-09-06 11:14:53 -0500
UncaughtThrowError: uncaught throw #<Encoding::UndefinedConversionError: "\xEF" from ASCII-8BIT to UTF-8>
C:/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:83:in `throw'
C:/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:83:in `rescue in run_report_safely'
C:/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:86:in `run_report_safely'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/handler.rb:125:in `block in run_report_handlers'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/handler.rb:123:in `each'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/handler.rb:123:in `run_report_handlers'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/handler.rb:135:in `block in <class:Handler>'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/client.rb:443:in `block in run_completed_successfully'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/client.rb:442:in `each'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/client.rb:442:in `run_completed_successfully'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/client.rb:301:in `run'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application.rb:273:in `run_with_graceful_exit_option'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application.rb:249:in `block in run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/local_mode.rb:44:in `with_server_connectivity'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application.rb:232:in `run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application/client.rb:469:in `sleep_then_run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application/client.rb:458:in `block in interval_run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application/client.rb:457:in `loop'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application/client.rb:457:in `interval_run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application/client.rb:441:in `run_application'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application.rb:59:in `run'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/bin/chef-client:26:in `<top (required)>'
C:/opscode/chef/bin/chef-client:58:in `load'
C:/opscode/chef/bin/chef-client:58:in `<main>'

>>>> Caused by Encoding::UndefinedConversionError: "\xEF" from ASCII-8BIT to UTF-8
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.6.0/lib/rspec/core/formatters/json_formatter.rb:55:in `encode'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.6.0/lib/rspec/core/formatters/json_formatter.rb:55:in `to_json'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.6.0/lib/rspec/core/formatters/json_formatter.rb:55:in `close'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.6.0/lib/rspec/core/reporter.rb:206:in `block in notify'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.6.0/lib/rspec/core/reporter.rb:205:in `each'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.6.0/lib/rspec/core/reporter.rb:205:in `notify'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.6.0/lib/rspec/core/reporter.rb:238:in `close'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.6.0/lib/rspec/core/reporter.rb:193:in `close_after'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.6.0/lib/rspec/core/reporter.rb:171:in `finish'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.6.0/lib/rspec/core/reporter.rb:81:in `report'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.6.0/lib/rspec/core/runner.rb:112:in `run_specs'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.33.1/lib/inspec/runner_rspec.rb:77:in `run'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.33.1/lib/inspec/runner.rb:116:in `run_tests'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/inspec-1.33.1/lib/inspec/runner.rb:100:in `run'
C:/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:145:in `call'
C:/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:60:in `report'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/handler.rb:259:in `run_report_unsafe'
C:/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:78:in `run_report_safely'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/handler.rb:125:in `block in run_report_handlers'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/handler.rb:123:in `each'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/handler.rb:123:in `run_report_handlers'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/handler.rb:135:in `block in <class:Handler>'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/client.rb:443:in `block in run_completed_successfully'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/client.rb:442:in `each'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/client.rb:442:in `run_completed_successfully'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/client.rb:301:in `run'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application.rb:273:in `run_with_graceful_exit_option'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application.rb:249:in `block in run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/local_mode.rb:44:in `with_server_connectivity'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application.rb:232:in `run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application/client.rb:469:in `sleep_then_run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application/client.rb:458:in `block in interval_run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application/client.rb:457:in `loop'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application/client.rb:457:in `interval_run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application/client.rb:441:in `run_application'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/lib/chef/application.rb:59:in `run'
C:/opscode/chef/embedded/lib/ruby/gems/2.4.0/gems/chef-13.3.42-universal-mingw32/bin/chef-client:26:in `<top (required)>'
C:/opscode/chef/bin/chef-client:58:in `load'
C:/opscode/chef/bin/chef-client:58:in `<main>'
@jerryaldrichiii

This comment has been minimized.

Collaborator

jerryaldrichiii commented Sep 6, 2017

@seththoenen which InSpec profiles are you running with the Audit cookbook?

@seththoenen

This comment has been minimized.

seththoenen commented Sep 6, 2017

@jerryaldrichiii It's a custom profile we've developed in house. Let me switch to something that comes built in with Chef Automate to see if its anything on our end.

@seththoenen

This comment has been minimized.

seththoenen commented Sep 6, 2017

@jerryaldrichiii, good call out on switching profiles. I switched to the built in Windows Security Baseline and there are no issues. So, it looks to be something inside my profile causing this to blow up. I'll report back here when I have more information.

@jerryaldrichiii

This comment has been minimized.

Collaborator

jerryaldrichiii commented Sep 6, 2017

@seththoenen, it was all you friend! Let us know what you find. I'm betting your initial hunch was right:

https://github.com/chef-cookbooks/windows/blob/master/resources/certificate_binding.rb#L126

One of the resources in your profile is probably returning a BOM.

@seththoenen

This comment has been minimized.

seththoenen commented Sep 6, 2017

@jerryaldrichiii, I've done some digging and here's what I've found. This seems to be an issue with the powershell InSpec resource executing on a system that has PowerShell 2.0 or lower. I've verified that systems with PowerShell 3.0, 4.0, or 5.0 have no issues.

@adamleff

This comment has been minimized.

Collaborator

adamleff commented Sep 6, 2017

@seththoenen thanks for the good sleuthing! We do state in the InSpec readme that PowerShell 3.0 or later is required. We could totally make this a better user experience by doing more proactive checking of the PowerShell version in resources that require it. If you'd like to open an issue on the chef/inspec repo for that feature request, we'd be happy to have that.

I'm going to close this issue since it does not appear to be related to the audit cookbook itself. Thanks again for the great detective work!

@adamleff adamleff closed this Sep 6, 2017

@seththoenen

This comment has been minimized.

seththoenen commented Sep 6, 2017

@adamleff no worries. We're going to be bumping the PowerShell version of the affected nodes.

Also, thanks for the assistance @jerryaldrichiii!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment