Does not properly maintain UTF8 encoding

[jeremywadsack]

It appears that UTF8 encoding is being dropped by fastest-csv and that results are converted to ASCII-encoded instead.

  def test_fastest_csv_utf8
    assert_equal(['セカイモン', '5000'], CsvParser.parse_line('セカイモン,"5000"'))
  end

  1) Failure:
TestUTF8Read#test_fastest_csv_utf8:
<["セカイモン", "5000"]> expected but was
<["\xE3\x82\xBB\xE3\x82\xAB\xE3\x82\xA4\xE3\x83\xA2\xE3\x83\xB3", "5000"]>.

[jeremywadsack]

Note that as a workaround, following jasoncodes/filter_io#2, I was able to re-apply encoding after parsing which seems to resolve this.

  encoding = csv_data.encoding
  rows = FastestCSV.parse(csv_data).map do |row|
    row.map do |col|
      col && col.force_encoding(encoding)
    end
  end

[spacemunkay]

I encountered this issue as well. Thanks for the workaround @jeremywadsack!

[jmartty]

After bumping into this issue I decided to fork and make my own version with encoding support @ https://github.com/jmartty/nesquikcsv