Skip to content

Commit

Permalink
Fix character escaping in Perl lexer (#1549)
Browse files Browse the repository at this point in the history
In single- and double-quoted strings, the Perl lexer currently
tokenises `\` as an error if it is not followed by a character in part
of a recognised escape sequence. This is a bug. Perl accepts the use of
`\` in single- and double-quoted strings even if it is not part of a
valid escape sequence.

This commit permits the use of `\` in single-quoted and double-quoted
strings as well as increasing the range of escape sequences that are
recognised in double-quoted strings.
  • Loading branch information
pyrmont committed Jul 5, 2020
1 parent e870b5e commit 9539764
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 2 deletions.
12 changes: 10 additions & 2 deletions lib/rouge/lexers/perl.rb
Expand Up @@ -178,16 +178,24 @@ def self.detect?(text)
end

state :sq do
rule %r/\\[']/, Str::Escape
rule %r/\\[\\']/, Str::Escape
rule %r/[^\\']+/, Str::Single
rule %r/'/, Punctuation, :pop!
rule %r/\\/, Str::Single
end

state :dq do
mixin :string_intp
rule %r/\\[\\tnr"]/, Str::Escape
rule %r/\\[\\tnrabefluLUE"$@]/, Str::Escape
rule %r/\\0\d{2}/, Str::Escape
rule %r/\\o\{\d+\}/, Str::Escape
rule %r/\\x\h{2}/, Str::Escape
rule %r/\\x\{\h+\}/, Str::Escape
rule %r/\\c./, Str::Escape
rule %r/\\N\{[^\}]+\}/, Str::Escape
rule %r/[^\\"]+?/, Str::Double
rule %r/"/, Punctuation, :pop!
rule %r/\\/, Str::Escape
end

state :bq do
Expand Down
8 changes: 8 additions & 0 deletions spec/visual/samples/perl
Expand Up @@ -60,7 +60,15 @@ $s =~ y/[]|/()&/; # "backward" compatible


# quoted strings
my $a = 'foo';
my $a = 'foo\'s bar';
my $a = 'foo\bar';
my $a = 'foo\\bar';
my $a = "foo";
my $a = "foo \" bar";
my $a = "foo\bar";
my $a = "foo \057 \x7f \x{263a} \cC \N{GREEK SMALL LETTER SIGMA} bar";
my $a = "invalid escape \w";

print "a: ";
print $a, " - ";
Expand Down

0 comments on commit 9539764

Please sign in to comment.