Skip to content

Commit

Permalink
- Better handling of magic encoding comment edge cases (fixes #33).
Browse files Browse the repository at this point in the history
  • Loading branch information
whitequark committed May 15, 2013
1 parent d43338f commit 4ad308a
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 6 deletions.
11 changes: 6 additions & 5 deletions lib/parser/source/buffer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,15 @@ class Buffer

def self.recognize_encoding(string)
if string.empty?
return Encoding::UTF_8
return Encoding::BINARY
end

# TODO: Make this more efficient.
first_line, second_line = string.lines.first(2)
first_line.force_encoding(Encoding::ASCII_8BIT)

[first_line, second_line].each do |line|
line.force_encoding(Encoding::ASCII_8BIT) if line
end

if first_line =~ /\A\xef\xbb\xbf/ # BOM
return Encoding::UTF_8
Expand All @@ -23,9 +26,7 @@ def self.recognize_encoding(string)
encoding_line = first_line
end

encoding_line.force_encoding(Encoding::ASCII_8BIT)

if encoding_line =~ /coding[:=]?\s*([a-z0-9_-]+)/
if encoding_line =~ /^#.*coding[:=]?\s*([a-z0-9_-]+)/
Encoding.find($1)
else
string.encoding
Expand Down
6 changes: 5 additions & 1 deletion test/test_encoding.rb
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,11 @@ def test_shebang
end

def test_empty
assert_equal Encoding::UTF_8, recognize("")
assert_equal Encoding::BINARY, recognize("")
end

def test_false_positive
assert_equal Encoding::BINARY, recognize(%{require 'cane/encoding_aware_iterator'})
end
end
end

0 comments on commit 4ad308a

Please sign in to comment.