Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ArgumentError (not fully converted, 23 bytes left) with String.encode! #5681

Open
bibstha opened this Issue Apr 7, 2019 · 3 comments

Comments

Projects
None yet
3 participants
@bibstha
Copy link

bibstha commented Apr 7, 2019

Environment

Provide at least:

  • JRuby version (jruby -v) and command line (flags, JRUBY_OPTS, etc)
    jruby 9.2.6.0 (2.5.3) 2019-02-11 15ba00b OpenJDK 64-Bit Server VM 12+33 on 12+33 +jit [darwin-x86_64]

  • Operating system and platform (e.g. uname -a)
    MacBookPro Mojave

Problem:

I'm trying to read and convert the following file in utf8
https://github.com/tmm1/ripper-tags/blob/7f31ab7d9009ea2c566e81901cd344b04e6356e1/test/fixtures/encoding.rb

Here is the ruby code that works fine in cruby

# test_encoding.rb
filename = "test/fixtures/encoding.rb"
str = File.open(filename, "r:utf8") { |f| f.read }
str.encode!('utf-16', :invalid => :replace, :undef => :replace)

On jruby it raises an exception

$ ruby test_encoding.rb
ArgumentError: not fully converted, 23 bytes left
  encode! at org/jruby/RubyString.java:6044
   <main> at test.rb:3
@headius

This comment has been minimized.

Copy link
Member

headius commented Apr 8, 2019

Strange...I am also on a MacBook and I cannot get this to fail. I wget the encoding.rb file and run your script against it, and only get a warning:

$ jruby blah.rb
blah.rb:3: warning: Unsupported encoding utf8 ignored

Fixing that to be utf-8 makes it complete without any warnings or errors.

What is the output of this script for you?

puts ENV_JAVA['file.encoding']
@kares

This comment has been minimized.

Copy link
Member

kares commented Apr 17, 2019

works on Linux x86_64 as well, using Java 8/11 also tried OpenJDK 12.0.1+12 (same built as yours)

@headius

This comment has been minimized.

Copy link
Member

headius commented Apr 18, 2019

@bibstha We will need to get some information from you about your environment, since we are unable to reproduce locally. Mostly we need to know if there's a different system encoding, so we can sort out which encoding is not working right.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.