Encoding::UndefinedConversionError when upgrading to 0.6.2 #127
Comments
|
Full trace, if it helps: /Users/jonsmock/.rvm/gems/ruby-1.9.2-p290/gems/simplecov-0.6.2/lib/simplecov/source_file.rb:81:in |
ghost
assigned 
|
Not sure how this happened. Another encoding-related issue was fixed with #117 and will be part of 0.6.4, which I'll release today. Could you please check against that version (or, if you're very fast, try the current git master in your Gemfile) and report back? |
|
Absolutely - thanks a lot. I should have time later today or early tomorrow to test it out. You rock! |
|
Looks like the new release didn't fix our issue. Anything I can do to isolate this better? Is there a way to figure out which file it's choking on? /Users/jonsmock/.rvm/gems/ruby-1.9.2-p290/gems/simplecov-0.6.4/lib/simplecov/source_file.rb:82:in |
|
Oops the first line of that backtrace should be (still in source.rb): /Users/jonsmock/.rvm/gems/ruby-1.9.2-p290/gems/simplecov-0.6.4/lib/simplecov/source_file.rb:82:in `readlines': U+00C6 from UTF-8 to ASCII-8BIT (Encoding::UndefinedConversionError) |
|
@jonsmock Have you been able to figure out what the problem is? |
|
I'm seeing the same problem. I tried adding simplecov to a project today and I had to back down all the way to 0.5.4 to get around the UndefinedConversionError. I can supply more detailed logs if you need it, but it's in the same readlines method as reported by @jonsmock |
|
I just ran into this issue tonight while working on a new gem. After doing a big of digging around, here's what I found: I'm working on a Fluentd plugin for Azure Table Storage. There's a gem (waz-storage) that I need in order to write to ATS. Due to my use of the waz-storage gem, simplecov tries at some point to both open and to read the /home/obie/.rvm/rubies/ruby-1.9.3-p327/lib/ruby/1.9.1/rexml/rexml.rb file, which contains the copyright symbol. Simplecov balks with the same error documented by this ticket, which is how I ended up here in the first place. In my particular instance, the culprit is the fluentd gem. Within its engine.rb file, it sets both the Encoding.default_external and Encoding.default_internal properties to "ASCII-8BIT". It is the setting of the default_internal property that induces heartburn later on if any file is subsequently opened and ready by Simplecov that contains higher unicode characters, such as a copyright symbol. Ruby provides a mechanism by which we can specify both the default external encoding and the internal encoding for an IO object such as File. So, with respect to source_file.rb, line 82 could be written as either File.open(filename, mode: 'r', external_encoding: Encoding::UTF_8, internal_encoding: Encoding::UTF_8) {|f| @src = f.readlines } or File.open(filename, "r:UTF-8:UTF-8") {|f| @src = f.readlines } The latter is preferable, but there's a catch. The most current ruby version (1.9.3-p327) displays a superfluous warning message for each file opened and read by Simplecov. It's a bug that's already been reported to the ruby team and resolved in the ruby trunk (http://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/35538). NOTE: the current incarnation of line 82 ("r:UTF-8") sets/overrides only the Encoding.default_external property. I hope this helps shed some light on why this issue affects some users and not others. There may be other use cases that throw the same error, but I can't imagine why the two options listed above wouldn't work for them as well. |
- Specify the default_internal encoding as UTF-8 when reading source files. This fixes Issue simplecov-ruby#127.
|
The fix for this by @justfalter is shipped as part of the 0.8 prereleases, 0.8.0 official is coming very soon. I assume this resolves this problem. If not, please report back once 0.8.0 is out. |
v0.8.2, 2013-11-20 ([changes](simplecov-ruby/simplecov@v0.8.1...v0.8.2)) ===================== ## Bugfixes * Replaced the locking behaviour [via lockfile gem](simplecov-ruby/simplecov#185) with plain Ruby explicit file locking when merging results. This should make simplecov merging to behave well on Windows again. See [#258](simplecov-ruby/simplecov#258) and [#223](simplecov-ruby/simplecov#223) (thanks to @tomykaira) v0.8.1, 2013-11-10 ([changes](simplecov-ruby/simplecov@v0.8.0...v0.8.1)) ===================== ## Bugfixes * Fixed a regression introduced in 0.8.0 - the Forwardable STDLIB module is now required explicitly. See [#256](simplecov-ruby/simplecov#256) (thanks to @kylev) v0.8.0, 2013-11-10 ([changes](simplecov-ruby/simplecov@v0.7.1...v0.8.0)) ===================== **Note: Yanked the same day because of the regression that 0.8.1 fixes, see above** ## TL;DR It's been way too long since the last official release 0.7.1, but this was partly due to it proving itself quite stable in most circumstances. This release brings various further stability improvements to result set merging (especially when working with parallel_tests), the configuration, source file encodings, and command name guessing. The 0.8 line is the last one to cooperate with Ruby < 1.9. Starting with 0.9, SimpleCov will assume to be running in Ruby 1.9+, and will not try to detect or bail silently on older Ruby versions. An appropriate deprecation warning has been added. ## Features * Configuration blocks now have access to variables and methods outside of the block's scope. See [#238](simplecov-ruby/simplecov#238) (thanks to @ms-tg) * You can now have a global `~/.simplecov` configuration file. See [#195](simplecov-ruby/simplecov#195) (thanks to @spagalloco) * simplecov-html now uses the MIT-licensed colorbox plugin. Some adjustments when viewing source files, including retaining the currently open file on refresh have been added. See [simplecov-html #15](simplecov-ruby/simplecov-html#15) (thanks to @chetan) * Adds support for Rails 4 command guessing, removes default group `vendor/plugins`. See [#181](simplecov-ruby/simplecov#181) and [#203](simplecov-ruby/simplecov#203) (thanks to @semanticart and @phallstrom) * You can now load simplecov without the default settings by doing `require 'simplecov/no_defaults'` or setting `ENV['SIMPLECOV_NO_DEFAULTS']`. Check `simplecov/defaults` to see what preconfigurations are getting dropped by using this. See [#209](simplecov-ruby/simplecov#209) (thanks to @ileitch) * The result set merging now uses the `lockfile` gem to avoid race conditions. See [#185](simplecov-ruby/simplecov#185) (thanks to @jshraibman-mdsol). * Automatically detect the usage of parallel_tests and adjust the command name with the test env number accordingly, See [#64](simplecov-ruby/simplecov#64) and [#185](simplecov-ruby/simplecov#185) (thanks to @jshraibman-mdsol). ## Enhancements * Rename adapters to "profiles" given that they are bundles of settings. The old adapter methods are deprecated, but remain available for now. See [#207](simplecov-ruby/simplecov#207) (thanks to @mikerobe) * Tweaks to the automatic test suite naming. In particular, `rspec/features` should now be correctly attributed to RSpec, not Cucumber. See [#212](simplecov-ruby/simplecov#212) (thanks to @ersatzryan and @betelgeuse) * MiniTest should now be identified correctly by the command name guesser. See [#244](simplecov-ruby/simplecov#244) (thanks to @envygeeks) * Makes SimpleCov resilient to inclusion of mathn library. See [#175](simplecov-ruby/simplecov#175) and [#140](simplecov-ruby/simplecov#140) (thanks to @scotje) * Allow coverage_dir to be an absolute path. * See [#190](simplecov-ruby/simplecov#190) (thanks to @jshraibman-mdsol) * The internal cucumber test suite now uses Capybara 2. See [#206](simplecov-ruby/simplecov#206) (thanks to @infertux) * Work-arounds for the Coverage library shipped in JRuby 1.6 to behave in line with MRI. See [#174](simplecov-ruby/simplecov#174) (thanks to @grddev) * Fix warning: instance variable @exit_status not initialized. See [#242](simplecov-ruby/simplecov#242) and [#213](simplecov-ruby/simplecov#213) (thanks to @sferik and @infertux) ## Bugfixes * Correct result calculations for people using :nocov: tags. See [#215](simplecov-ruby/simplecov#215) (thanks to @aokolish) * Average hits per line for groups of files is now computed correctly. See [#192](http://github.com/colszowka/simplecov/pull/192) and [#179](http://github.com/colszowka/simplecov/issues/179) (thanks to @Graysonwright) * Compatability with BINARY internal encoding. See [#194](simplecov-ruby/simplecov#194) and [#127](simplecov-ruby/simplecov#127) (thanks to @justfalter) * Special characters in `SimpleCov.root` are now correctly escaped before being used as a RegExp. See [#204](simplecov-ruby/simplecov#204) and [#237](simplecov-ruby/simplecov#237) (thanks to @rli9)
v0.8.2, 2013-11-20 ([changes](simplecov-ruby/simplecov@v0.8.1...v0.8.2)) ===================== ## Bugfixes * Replaced the locking behaviour [via lockfile gem](simplecov-ruby/simplecov#185) with plain Ruby explicit file locking when merging results. This should make simplecov merging to behave well on Windows again. See [#258](simplecov-ruby/simplecov#258) and [#223](simplecov-ruby/simplecov#223) (thanks to @tomykaira) v0.8.1, 2013-11-10 ([changes](simplecov-ruby/simplecov@v0.8.0...v0.8.1)) ===================== ## Bugfixes * Fixed a regression introduced in 0.8.0 - the Forwardable STDLIB module is now required explicitly. See [#256](simplecov-ruby/simplecov#256) (thanks to @kylev) v0.8.0, 2013-11-10 ([changes](simplecov-ruby/simplecov@v0.7.1...v0.8.0)) ===================== **Note: Yanked the same day because of the regression that 0.8.1 fixes, see above** ## TL;DR It's been way too long since the last official release 0.7.1, but this was partly due to it proving itself quite stable in most circumstances. This release brings various further stability improvements to result set merging (especially when working with parallel_tests), the configuration, source file encodings, and command name guessing. The 0.8 line is the last one to cooperate with Ruby < 1.9. Starting with 0.9, SimpleCov will assume to be running in Ruby 1.9+, and will not try to detect or bail silently on older Ruby versions. An appropriate deprecation warning has been added. ## Features * Configuration blocks now have access to variables and methods outside of the block's scope. See [#238](simplecov-ruby/simplecov#238) (thanks to @ms-tg) * You can now have a global `~/.simplecov` configuration file. See [#195](simplecov-ruby/simplecov#195) (thanks to @spagalloco) * simplecov-html now uses the MIT-licensed colorbox plugin. Some adjustments when viewing source files, including retaining the currently open file on refresh have been added. See [simplecov-html #15](simplecov-ruby/simplecov-html#15) (thanks to @chetan) * Adds support for Rails 4 command guessing, removes default group `vendor/plugins`. See [#181](simplecov-ruby/simplecov#181) and [#203](simplecov-ruby/simplecov#203) (thanks to @semanticart and @phallstrom) * You can now load simplecov without the default settings by doing `require 'simplecov/no_defaults'` or setting `ENV['SIMPLECOV_NO_DEFAULTS']`. Check `simplecov/defaults` to see what preconfigurations are getting dropped by using this. See [#209](simplecov-ruby/simplecov#209) (thanks to @ileitch) * The result set merging now uses the `lockfile` gem to avoid race conditions. See [#185](simplecov-ruby/simplecov#185) (thanks to @jshraibman-mdsol). * Automatically detect the usage of parallel_tests and adjust the command name with the test env number accordingly, See [#64](simplecov-ruby/simplecov#64) and [#185](simplecov-ruby/simplecov#185) (thanks to @jshraibman-mdsol). ## Enhancements * Rename adapters to "profiles" given that they are bundles of settings. The old adapter methods are deprecated, but remain available for now. See [#207](simplecov-ruby/simplecov#207) (thanks to @mikerobe) * Tweaks to the automatic test suite naming. In particular, `rspec/features` should now be correctly attributed to RSpec, not Cucumber. See [#212](simplecov-ruby/simplecov#212) (thanks to @ersatzryan and @betelgeuse) * MiniTest should now be identified correctly by the command name guesser. See [#244](simplecov-ruby/simplecov#244) (thanks to @envygeeks) * Makes SimpleCov resilient to inclusion of mathn library. See [#175](simplecov-ruby/simplecov#175) and [#140](simplecov-ruby/simplecov#140) (thanks to @scotje) * Allow coverage_dir to be an absolute path. * See [#190](simplecov-ruby/simplecov#190) (thanks to @jshraibman-mdsol) * The internal cucumber test suite now uses Capybara 2. See [#206](simplecov-ruby/simplecov#206) (thanks to @infertux) * Work-arounds for the Coverage library shipped in JRuby 1.6 to behave in line with MRI. See [#174](simplecov-ruby/simplecov#174) (thanks to @grddev) * Fix warning: instance variable @exit_status not initialized. See [#242](simplecov-ruby/simplecov#242) and [#213](simplecov-ruby/simplecov#213) (thanks to @sferik and @infertux) ## Bugfixes * Correct result calculations for people using :nocov: tags. See [#215](simplecov-ruby/simplecov#215) (thanks to @aokolish) * Average hits per line for groups of files is now computed correctly. See [#192](http://github.com/colszowka/simplecov/pull/192) and [#179](http://github.com/colszowka/simplecov/issues/179) (thanks to @Graysonwright) * Compatability with BINARY internal encoding. See [#194](simplecov-ruby/simplecov#194) and [#127](simplecov-ruby/simplecov#127) (thanks to @justfalter) * Special characters in `SimpleCov.root` are now correctly escaped before being used as a RegExp. See [#204](simplecov-ruby/simplecov#204) and [#237](simplecov-ruby/simplecov#237) (thanks to @rli9)
v0.8.2, 2013-11-20 ([changes](simplecov-ruby/simplecov@v0.8.1...v0.8.2)) ===================== ## Bugfixes * Replaced the locking behaviour [via lockfile gem](simplecov-ruby/simplecov#185) with plain Ruby explicit file locking when merging results. This should make simplecov merging to behave well on Windows again. See [#258](simplecov-ruby/simplecov#258) and [#223](simplecov-ruby/simplecov#223) (thanks to @tomykaira) v0.8.1, 2013-11-10 ([changes](simplecov-ruby/simplecov@v0.8.0...v0.8.1)) ===================== ## Bugfixes * Fixed a regression introduced in 0.8.0 - the Forwardable STDLIB module is now required explicitly. See [#256](simplecov-ruby/simplecov#256) (thanks to @kylev) v0.8.0, 2013-11-10 ([changes](simplecov-ruby/simplecov@v0.7.1...v0.8.0)) ===================== **Note: Yanked the same day because of the regression that 0.8.1 fixes, see above** ## TL;DR It's been way too long since the last official release 0.7.1, but this was partly due to it proving itself quite stable in most circumstances. This release brings various further stability improvements to result set merging (especially when working with parallel_tests), the configuration, source file encodings, and command name guessing. The 0.8 line is the last one to cooperate with Ruby < 1.9. Starting with 0.9, SimpleCov will assume to be running in Ruby 1.9+, and will not try to detect or bail silently on older Ruby versions. An appropriate deprecation warning has been added. ## Features * Configuration blocks now have access to variables and methods outside of the block's scope. See [#238](simplecov-ruby/simplecov#238) (thanks to @ms-tg) * You can now have a global `~/.simplecov` configuration file. See [#195](simplecov-ruby/simplecov#195) (thanks to @spagalloco) * simplecov-html now uses the MIT-licensed colorbox plugin. Some adjustments when viewing source files, including retaining the currently open file on refresh have been added. See [simplecov-html #15](simplecov-ruby/simplecov-html#15) (thanks to @chetan) * Adds support for Rails 4 command guessing, removes default group `vendor/plugins`. See [#181](simplecov-ruby/simplecov#181) and [#203](simplecov-ruby/simplecov#203) (thanks to @semanticart and @phallstrom) * You can now load simplecov without the default settings by doing `require 'simplecov/no_defaults'` or setting `ENV['SIMPLECOV_NO_DEFAULTS']`. Check `simplecov/defaults` to see what preconfigurations are getting dropped by using this. See [#209](simplecov-ruby/simplecov#209) (thanks to @ileitch) * The result set merging now uses the `lockfile` gem to avoid race conditions. See [#185](simplecov-ruby/simplecov#185) (thanks to @jshraibman-mdsol). * Automatically detect the usage of parallel_tests and adjust the command name with the test env number accordingly, See [#64](simplecov-ruby/simplecov#64) and [#185](simplecov-ruby/simplecov#185) (thanks to @jshraibman-mdsol). ## Enhancements * Rename adapters to "profiles" given that they are bundles of settings. The old adapter methods are deprecated, but remain available for now. See [#207](simplecov-ruby/simplecov#207) (thanks to @mikerobe) * Tweaks to the automatic test suite naming. In particular, `rspec/features` should now be correctly attributed to RSpec, not Cucumber. See [#212](simplecov-ruby/simplecov#212) (thanks to @ersatzryan and @betelgeuse) * MiniTest should now be identified correctly by the command name guesser. See [#244](simplecov-ruby/simplecov#244) (thanks to @envygeeks) * Makes SimpleCov resilient to inclusion of mathn library. See [#175](simplecov-ruby/simplecov#175) and [#140](simplecov-ruby/simplecov#140) (thanks to @scotje) * Allow coverage_dir to be an absolute path. * See [#190](simplecov-ruby/simplecov#190) (thanks to @jshraibman-mdsol) * The internal cucumber test suite now uses Capybara 2. See [#206](simplecov-ruby/simplecov#206) (thanks to @infertux) * Work-arounds for the Coverage library shipped in JRuby 1.6 to behave in line with MRI. See [#174](simplecov-ruby/simplecov#174) (thanks to @grddev) * Fix warning: instance variable @exit_status not initialized. See [#242](simplecov-ruby/simplecov#242) and [#213](simplecov-ruby/simplecov#213) (thanks to @sferik and @infertux) ## Bugfixes * Correct result calculations for people using :nocov: tags. See [#215](simplecov-ruby/simplecov#215) (thanks to @aokolish) * Average hits per line for groups of files is now computed correctly. See [#192](http://github.com/colszowka/simplecov/pull/192) and [#179](http://github.com/colszowka/simplecov/issues/179) (thanks to @Graysonwright) * Compatability with BINARY internal encoding. See [#194](simplecov-ruby/simplecov#194) and [#127](simplecov-ruby/simplecov#127) (thanks to @justfalter) * Special characters in `SimpleCov.root` are now correctly escaped before being used as a RegExp. See [#204](simplecov-ruby/simplecov#204) and [#237](simplecov-ruby/simplecov#237) (thanks to @rli9)
We just upgraded from simplecov 0.5.4 to 0.6.2. I started getting this error (shortened) when running our specs:
simplecov/source_file.rb:81:in `readlines': U+00C6 from UTF-8 to ASCII-8BIT (Encoding::UndefinedConversionError)
We're using ruby 1.9.2-p290. Here's
ruby -v:ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-darwin11.0.1]
I see the line was changed here, which looks correct to me (?): https://github.com/colszowka/simplecov/pull/103/files
I see that U+00C6 is http://graphemica.com/%C3%86 . When I run:
grep -r Æ .I only get images and vendor/cache/*.gem files.If I downgrade 0.5.4, simplecov runs fine. If it is something with simplecov, I would like to help fix it, but I don't know what the next step to debug it is. I also don't understand why it's going to ASCII-8BIT in the above error message.
The text was updated successfully, but these errors were encountered: