Encoding::UndefinedConversionError: Input length = 1 #318

Closed
lkfken opened this Issue Sep 26, 2012 · 7 comments

Projects

None yet

4 participants

@lkfken
lkfken commented Sep 26, 2012

SOURCE:

#coding: utf-8
string = %q{你好}

puts Encoding.default_external
puts string.encoding
File.open("test.xml", "w") do |f|
  f.write string
end

With jruby 1.7.0.RC1 (1.9.3p203) 2012-09-25 8e849de on Java HotSpot(TM) Client VM 1.6.0_35-b10 [Windows XP-x86], the output...

Windows-1252
UTF-8
Encoding::UndefinedConversionError: Input length = 1
   write at org/jruby/RubyIO.java:1401
  (root) at X:/ene/RubyScripts/1development/try_msword/app.rb:24
    open at org/jruby/RubyIO.java:1180
  (root) at X:/ene/RubyScripts/1development/try_msword/app.rb:23
    load at org/jruby/RubyKernel.java:1045
  (root) at -e:1

Process finished with exit code 1

With MRI ruby 1.9.3p125 (2012-02-16) [i386-mingw32], the output...

UTF-8
UTF-8

Process finished with exit code 0

Apparently, Encoding.default_external returns different values.
Also, I cannot write to a file if the string is UTF-8 under JRuby. However, the same code works fine under MRI Ruby.

@headius
Member
headius commented Sep 26, 2012

This seems to be lingering issues with Windows-1252 being used as the default external encoding in JRuby when it should not be. Can you try passing -Eutf-8 when running your script, to force external to utf-8?

@enebo
Member
enebo commented Sep 27, 2012

Should we be UTF-8 or should we be file.encoding? We are happy with UTF-8 on other platforms since Java file.coding defaults to that but MRI does not default to that on those platforms? I can see both sides of this.

If we did match defaults as specified by MRI we would probably get less bug reports.

@lkfken
lkfken commented Oct 10, 2012

@headius with -Eutf-8, JRuby is able to generate the file test.xml just fine under Windows XP. No more conversion error. Thank you.

@headius
Member
headius commented Jul 24, 2013

This will likely go away with 2.0 support, where the default encoding is always UTF-8.

@headius headius self-assigned this Nov 12, 2014
@headius
Member
headius commented Nov 12, 2014

Oops, I started to close this before realizing it was a problem on Windows. @lkfken can you test a recent jruby master build?

@lkfken
lkfken commented Nov 14, 2014

jruby 1.7.16.1 (1.9.3p392) 2014-10-28 4e93f31 on Java HotSpot(TM) Client VM 1.7.0_07-b10 +jit [Windows XP-x86]

Same source code.

Even without the flag -Eutf-8, the test file is generated fine. No conversion error.

Thank you.

@enebo enebo modified the milestone: JRuby 9.0.0.0 Jul 14, 2015
@rtyler
rtyler commented Aug 7, 2015

This does appear to be fixed with JRuby 9k, @lkfken if you see that the example case is still broken with 9.0.0.0 please re-open this issue

@rtyler rtyler closed this Aug 7, 2015
@enebo enebo added this to the JRuby 9.0.0.0 milestone Aug 20, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment