Skip to content

Commit

Permalink
Convert invalid strings from binary to utf-8
Browse files Browse the repository at this point in the history
Since changing the encoding from UTF8 to UTF8 doesn't do anything (see
http://bugs.ruby-lang.org/issues/7967#note-1), change the source encoding
to binary (ie. ascii-8bit) and replace all undefined characters.
  • Loading branch information
coffeejunk committed Feb 26, 2013
1 parent abfc798 commit 7f97418
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion lib/cane/encoding_aware_iterator.rb
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ def each(&block)
begin
line =~ /\s/
rescue ArgumentError
line.encode!('UTF-8', 'UTF-8', invalid: :replace)
line.encode!('UTF-8', 'binary', invalid: :replace, undef: :replace)
end

block.call(line)
Expand Down

0 comments on commit 7f97418

Please sign in to comment.