Added Squirrel lexer #814

mathewmariani · 2017-10-30T21:02:19Z

Added lever for the Squirrel programing language
(http://squirrel-lang.org/).

Added lever for the Squirrel programing language (http://squirrel-lang.org/).

mathewmariani · 2017-10-30T21:17:41Z

For my own curiosity, how does it take newly added lexers to be added into production, and usable?

lib/rouge/lexers/squirrel.rb

vidarh

Hi, I'm trying to help triage old pull requests so we can work down the backlog. It looks like you've addressed the previous comments. Just one more nitpick - '# frozen_string_literal: true' was added project wide since the PR was open. If you please add that, and then squash your commits, I think this one looks ready to go...

lib/rouge/lexers/squirrel.rb

spec/lexers/squirrel_spec.rb

Added lever for the Squirrel programing language (http://squirrel-lang.org/).

mathewmariani · 2019-05-10T22:23:22Z

I'm not sure why all the checks are failing. Can someone help me out?

vidarh · 2019-05-12T20:21:34Z

@mathewmariani It's down to CI issues covered in #1062

Added lever for the Squirrel programing language (http://squirrel-lang.org/).

pyrmont · 2019-06-01T00:23:12Z

@mathewmariani Oh noes, this looks a bit worse for wear. @miparnisari, didn't you have a problem like this? How'd you fix it?

miparnisari · 2019-06-01T00:39:49Z

@pyrmont in my fork I updated the master branch against upstream and then

git checkout <branch>
git rebase -i master
git push --force

😂

pyrmont · 2019-06-01T00:46:43Z

Thanks @miparnisari!

@mathewmariani Might want to give that a try and see if that fixes things :)

mathewmariani · 2019-06-05T20:46:49Z

Thanks @pyrmont and @miparnisari

pyrmont

This is looking pretty good, @mathewmariani :) Let me know if any of the comments are unclear!

lib/rouge/lexers/squirrel.rb

spec/visual/samples/squirrel

mathewmariani · 2019-07-05T22:37:42Z

@pyrmont I have re-written the lexer. The original merge-request is almost two years old, and I felt I have learned more about Ruby since then.

pyrmont · 2019-07-05T22:39:30Z

@mathewmariani Will check the changes out and get back to you soon! :)

pyrmont

Sorry it took a bit longer to get to this than I'd hoped. Overall, it's looking good! I think we're pretty close now :)

pyrmont · 2019-07-18T19:52:10Z

lib/rouge/lexers/squirrel.rb

+
+        # strings
+        rule %r(@"(""|[^"])*")m, Str
+        rule %r("(\\.|.)*?["\n]), Str


Four comments:

What is the reason to have (\\.|.)? Why have the \\. in there?

["\n] is a character class and will match against any character in the class. This regex will match against a string that begins with " and ends in a newline even if there was no matching ". Was that the intention?

This pattern will mean that the lexer won't highlight escape sequences in strings (eg. \n). The Swift lexer is an example of how this can be done:

rouge/lib/rouge/lexers/swift.rb

Lines 163 to 170 in b6411b1

state :dq do

rule %r/\\[\\0tnr'"]/, Str::Escape

rule %r/\\[(]/, Str::Escape, :interp

rule %r/\\u\{\h{1,8}\}/, Str::Escape

rule %r/[^\\"]+/, Str

rule %r/"""/, Str, :pop!

rule %r/"/, Str, :pop!

end

The token for this should be Str::Double.

pyrmont · 2019-07-18T19:53:43Z

lib/rouge/lexers/squirrel.rb

+        # strings
+        rule %r(@"(""|[^"])*")m, Str
+        rule %r("(\\.|.)*?["\n]), Str
+        rule %r('(\\.|.)'), Str::Char


Same issue here as with the above.

pyrmont · 2019-07-18T19:57:35Z

lib/rouge/lexers/squirrel.rb

+        rule %r(/[*].*?[*]/)m, Comment::Multiline
+
+        # strings
+        rule %r(@"(""|[^"])*")m, Str


I think the token for this should be Str::Heredoc.

pyrmont · 2019-07-18T19:58:41Z

lib/rouge/lexers/squirrel.rb

+        rule %r((?:class)\b), Keyword::Declaration, :class
+
+        # operators and punctuation
+        rule %r([~!%^&*()+=|\[\]{}:;,.<>\/?-]), Punctuation


When used in a character class, - should either be escaped or be the first character in the class.

pyrmont · 2019-07-18T20:00:12Z

lib/rouge/lexers/squirrel.rb

+        rule %r([~!%^&*()+=|\[\]{}:;,.<>\/?-]), Punctuation
+
+        rule %r([0-9]+(([.]([0-9]+)?)(e[-]?[0-9]+)?)), Num::Float
+        rule %r(0x[0-9a-fA-F]+|0[0-7]+|[0-9]+), Num::Integer


The first part of this pattern should be broken out into a separate rule to match against the hexadecimal numbers. The token for that would be Num::Hex.

I decided to concatenate them since Squirrel reads Hexadecimals as Integers. But, I can separate them.

Yeah, the names of the tokens are really a little misleading.

All languages I know (all languages maybe?) use different numeric bases only for input. Once a number is input, it's whatever it is: an integer, a float, etc. The token Num::Integer should really be Num::Integer::Decimal and Num::Hex should be Num::Integer::Hex. Alas, Pygments chose the tokens that they did.

Could you separate the hexadecimal representation pattern into a new rule with a new token? That will be consistent with the other lexers.

pyrmont · 2019-07-18T20:00:56Z

spec/visual/samples/squirrel

+print(fib(5) + "\n");
+
+
+#


Could you add an example (possibly here) of the multiline comment?

mathewmariani added 2 commits October 30, 2017 17:02

Added Squirrel lexer

1557a60

Added lever for the Squirrel programing language (http://squirrel-lang.org/).

Fixed block alignment

f506986

miparnisari suggested changes Jun 17, 2018

View reviewed changes

lib/rouge/lexers/squirrel.rb Outdated Show resolved Hide resolved

lib/rouge/lexers/squirrel.rb Outdated Show resolved Hide resolved

lib/rouge/lexers/squirrel.rb Outdated Show resolved Hide resolved

lib/rouge/lexers/squirrel.rb Outdated Show resolved Hide resolved

Updated lexers/squirrel.rb

c63ea00

vidarh suggested changes Jan 13, 2019

View reviewed changes

lib/rouge/lexers/squirrel.rb Show resolved Hide resolved

spec/lexers/squirrel_spec.rb Show resolved Hide resolved

vidarh mentioned this pull request Jan 17, 2019

Cleaning up pull requests - Reviewed lexer PRs #1063

Closed

mathewmariani and others added 6 commits February 17, 2019 13:40

added frozen_string_literal comment

8722ff4

Added Squirrel lexer

f28424d

Added lever for the Squirrel programing language (http://squirrel-lang.org/).

Fixed block alignment

86f4b72

Updated lexers/squirrel.rb

7724c1c

added frozen_string_literal comment

22aa592

Merge branch 'master' of https://github.com/mathewmariani/rouge

950ec1c

mathewmariani and others added 8 commits May 31, 2019 18:32

Added Squirrel lexer

f83c1d8

Added lever for the Squirrel programing language (http://squirrel-lang.org/).

Fixed block alignment

34fc3a0

Updated lexers/squirrel.rb

b97a038

added frozen_string_literal comment

d292ce3

Added Squirrel lexer

6f9bd4f

Added lever for the Squirrel programing language (http://squirrel-lang.org/).

Fixed block alignment

9607e75

Updated lexers/squirrel.rb

d6e0fa0

Merge branch 'master' of https://github.com/mathewmariani/rouge

9295ef9

pyrmont added the author-action The PR has been reviewed but action by the author is needed label Jun 1, 2019

Merge remote-tracking branch 'upstream/master'

71296eb

pyrmont removed the author-action The PR has been reviewed but action by the author is needed label Jun 5, 2019

pyrmont added the needs-review The PR needs to be reviewed label Jun 5, 2019

pyrmont suggested changes Jun 6, 2019

View reviewed changes

pyrmont added author-action The PR has been reviewed but action by the author is needed and removed needs-review The PR needs to be reviewed labels Jun 6, 2019

mathewmariani added 2 commits July 3, 2019 22:47

updated lexer and samples

24e68af

changed regex syntax

5e7c375

pyrmont added needs-review The PR needs to be reviewed and removed author-action The PR has been reviewed but action by the author is needed labels Jul 8, 2019

pyrmont suggested changes Jul 18, 2019

View reviewed changes

pyrmont added author-action The PR has been reviewed but action by the author is needed and removed needs-review The PR needs to be reviewed labels Jul 18, 2019

pyrmont self-assigned this Sep 3, 2019

pyrmont removed their assignment Dec 9, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added Squirrel lexer #814

Added Squirrel lexer #814

mathewmariani commented Oct 30, 2017

mathewmariani commented Oct 30, 2017

vidarh left a comment

mathewmariani commented May 10, 2019

vidarh commented May 12, 2019

pyrmont commented Jun 1, 2019

miparnisari commented Jun 1, 2019

pyrmont commented Jun 1, 2019

mathewmariani commented Jun 5, 2019

pyrmont left a comment

mathewmariani commented Jul 5, 2019

pyrmont commented Jul 5, 2019

pyrmont left a comment

pyrmont Jul 18, 2019 •

edited

Loading

pyrmont Jul 18, 2019

pyrmont Jul 18, 2019 •

edited

Loading

pyrmont Jul 18, 2019

pyrmont Jul 18, 2019

mathewmariani Jul 19, 2019 •

edited

Loading

pyrmont Jul 19, 2019

pyrmont Jul 18, 2019

	state :dq do
	rule %r/\\[\\0tnr'"]/, Str::Escape
	rule %r/\\[(]/, Str::Escape, :interp
	rule %r/\\u\{\h{1,8}\}/, Str::Escape
	rule %r/[^\\"]+/, Str
	rule %r/"""/, Str, :pop!
	rule %r/"/, Str, :pop!
	end

		print(fib(5) + "\n");


		#

Added Squirrel lexer #814

Are you sure you want to change the base?

Added Squirrel lexer #814

Conversation

mathewmariani commented Oct 30, 2017

mathewmariani commented Oct 30, 2017

vidarh left a comment

Choose a reason for hiding this comment

mathewmariani commented May 10, 2019

vidarh commented May 12, 2019

pyrmont commented Jun 1, 2019

miparnisari commented Jun 1, 2019

pyrmont commented Jun 1, 2019

mathewmariani commented Jun 5, 2019

pyrmont left a comment

Choose a reason for hiding this comment

mathewmariani commented Jul 5, 2019

pyrmont commented Jul 5, 2019

pyrmont left a comment

Choose a reason for hiding this comment

pyrmont Jul 18, 2019 • edited Loading

Choose a reason for hiding this comment

pyrmont Jul 18, 2019

Choose a reason for hiding this comment

pyrmont Jul 18, 2019 • edited Loading

Choose a reason for hiding this comment

pyrmont Jul 18, 2019

Choose a reason for hiding this comment

pyrmont Jul 18, 2019

Choose a reason for hiding this comment

mathewmariani Jul 19, 2019 • edited Loading

Choose a reason for hiding this comment

pyrmont Jul 19, 2019

Choose a reason for hiding this comment

pyrmont Jul 18, 2019

Choose a reason for hiding this comment

pyrmont Jul 18, 2019 •

edited

Loading

pyrmont Jul 18, 2019 •

edited

Loading

mathewmariani Jul 19, 2019 •

edited

Loading