Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix backtracking issues, add string interpolation in Perl lexer #1161

Merged
merged 2 commits into from
Jun 9, 2019
Merged

Fix backtracking issues, add string interpolation in Perl lexer #1161

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f2aa62e
Fix backtracking issues, add string interpolation
pyrmont Jun 9, 2019
e29e05b
Fix tokens
pyrmont Jun 9, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 34 additions & 3 deletions lib/rouge/lexers/perl.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,9 +120,9 @@ def self.detect?(text)
rule /\d+(_\d*)*e[+-]?\d+(_\d*)*/i, Num::Float
rule /\d+(_\d+)*/, Num::Integer

rule /'(\\\\|\\'|[^'])*'/, Str
rule /"(\\\\|\\"|[^"])*"/, Str
rule /`(\\\\|\\`|[^`])*`/, Str::Backtick
rule /'/, Punctuation, :sq
rule /"/, Punctuation, :dq
rule /`/, Punctuation, :bq
rule /<([^\s>]+)>/, re_tok
rule /(q|qq|qw|qr|qx)\{/, Str::Other, :cb_string
rule /(q|qq|qw|qr|qx)\(/, Str::Other, :rb_string
Expand Down Expand Up @@ -178,6 +178,26 @@ def self.detect?(text)
rule /;/, Punctuation, :pop!
end

state :sq do
rule /\\[']/, Str::Single
rule /[^\\']+/, Str::Single
rule /'/, Punctuation, :pop!
end

state :dq do
mixin :string_intp
rule /\\[\\tnr"]/, Str::Double
rule /[^\\"]+?/, Str::Double
rule /"/, Punctuation, :pop!
end

state :bq do
mixin :string_intp
rule /\\[\\tnr`]/, Str::Backtick
pyrmont marked this conversation as resolved.
Show resolved Hide resolved
rule /[^\\`]+?/, Str::Backtick
rule /`/, Punctuation, :pop!
end

[[:cb, '\{', '\}'],
[:rb, '\(', '\)'],
[:sb, '\[', '\]'],
Expand All @@ -192,6 +212,17 @@ def self.detect?(text)
end
end

state :in_interp do
rule /}/, Str::Interpol, :pop!
rule /\s+/, Text
rule /[a-z_]\w*/i, Str::Interpol
end

state :string_intp do
rule /[$@][{]/, Str::Interpol, :in_interp
rule /[$@][a-z_]\w*/i, Str::Interpol
end

state :end_part do
# eat the rest of the stream
rule /.+/m, Comment::Preproc, :pop!
Expand Down
6 changes: 6 additions & 0 deletions spec/visual/samples/perl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,12 @@ $p =~ s%
%bar%x;
print $p, "\n";

my $str1 = "This is a ${ string } with fancy interpolation."
my $cmd1 = `So is @{ this } one.`
my $str2 = "This is a $string with plain interpolation."
my $cmd2 = `So is @this one.`
my $str3 = 'A boring $string.'

###### perl_misc ########

# from http://gist.github.com/485595
Expand Down