Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added HOCON lexer. #1253

Merged
merged 2 commits into from Jul 11, 2019

Conversation

@davidtwco
Copy link
Contributor

commented Jul 8, 2019

This PR adds a lexer for HOCON. I'm not sure that exhaustively covers all of HOCON, but I believe it covers the majority of the language.

I'm happy to make any modifications, improvements or changes as requested. Thanks.

This is a resubmission of #836.

@davidtwco davidtwco referenced this pull request Jul 8, 2019
@ashmaroli

This comment has been minimized.

Copy link
Contributor

commented Jul 8, 2019

The variables keywords, keys, values seem to be used just once across the entire lexer. You've two choices:

  • You can either instruct the Ruby interpreter to interpolate a regex just once (for performance reasons) by using the o flag: %r/#{foo}/o.
  • Or if you don't foresee a possibility for a new lexer being subclassed from the HOCON class, you may obliterate those variables entirely and have static regexps right away:
    # values
    rule %r/[^\$\"{}\[\]:=,\+#`\^\?!@\*&]+?/, Literal
@davidtwco

This comment has been minimized.

Copy link
Contributor Author

commented Jul 8, 2019

I've removed the variables.

@pyrmont pyrmont added the needs-review label Jul 8, 2019

@pyrmont

This comment has been minimized.

Copy link
Member

commented Jul 8, 2019

@davidtwco Thanks for submitting this again and I'm sorry you've had to wait so long :(

I'm looking at the informal spec and, given HOCON is a superset of JSON, it looks like this lexer would be best implemented as a subclass of Rouge::Lexers::JSON. You'd avoid needing to reimplement all the stuff that carries across from JSON and just be able to focus on the stuff that's different.

In addition to simply redefining a rule that you need to operate differently, Rouge comes with the methods prepend and append that you can use to manipulate an existing rule. You can see an example of using the JSON lexer as a superclass in the JSONDOC lexer.

@pyrmont pyrmont added author-action and removed needs-review labels Jul 8, 2019

@davidtwco

This comment has been minimized.

Copy link
Contributor Author

commented Jul 9, 2019

@pyrmont I've implemented this as a subclass of JSON, I think I've eliminated all the redundant rules and those that remain are necessary.

@pyrmont

This comment has been minimized.

Copy link
Member

commented Jul 9, 2019

Wow, that's fast! I'll have a look over it and let you know if I spot anything :)

@pyrmont pyrmont added needs-review and removed author-action labels Jul 9, 2019

@davidtwco

This comment has been minimized.

Copy link
Contributor Author

commented Jul 9, 2019

Pushed again to (hopefully) fix Travis failures and fix a missing word in a comment.

lib/rouge/lexers/hocon.rb Outdated Show resolved Hide resolved
@pyrmont
Copy link
Member

left a comment

Got a few questions:

lib/rouge/lexers/hocon.rb Show resolved Hide resolved
lib/rouge/lexers/hocon.rb Outdated Show resolved Hide resolved
lib/rouge/lexers/hocon.rb Show resolved Hide resolved
lib/rouge/lexers/hocon.rb Outdated Show resolved Hide resolved

@pyrmont pyrmont added author-action and removed needs-review labels Jul 9, 2019

@pyrmont pyrmont merged commit cc8ffa1 into rouge-ruby:master Jul 11, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@davidtwco davidtwco deleted the davidtwco:add-hocon-lexer branch Jul 12, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.