Encoding::Converter#primitive_convert and valid chars appending to the destination string #1835

Open
robin850 opened this Issue Jul 20, 2014 · 0 comments

Projects

None yet

1 participant

@robin850

Hello,

The behavior of Encoding::Converter#primitive_convert doesn't match the one on MRI or Rubinius. This method should normally append the chars that are valid in the destination string until an invalid char is found. Per its documentation:

#primitive_convert drops the converted part of source_buffer. the dropped part
is converted in destination_buffer or buffered in Encoding::Converter object.

However, this is not the case on JRuby. Here's a simple test case to showcase the issue:

# encoding: utf-8
gem 'minitest', '~> 5.4'
require 'minitest/autorun'

class EncodingConverterTest < Minitest::Test
  def test_primitive_convert
    string      = "a\x80a" # a€a
    source      = Encoding::UTF_8
    destination = Encoding::UTF_16LE

    converter = Encoding::Converter.new(source, destination)
    output    = ''.force_encoding(destination)

    # Once an invalid char is found, #primitive_convert should
    # push the chars before this one in the output string
    converter.primitive_convert(string, output)
    assert !output.empty?
  end
end

Tested with JRuby 1.7.13 and MRI 2.1.2 and 1.9.3.

Have a nice day.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment