Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added HOCON lexer. #1253

Merged
merged 2 commits into from
Jul 11, 2019
Merged

Added HOCON lexer. #1253

merged 2 commits into from
Jul 11, 2019

Conversation

davidtwco
Copy link
Contributor

This PR adds a lexer for HOCON. I'm not sure that exhaustively covers all of HOCON, but I believe it covers the majority of the language.

I'm happy to make any modifications, improvements or changes as requested. Thanks.

This is a resubmission of #836.

@davidtwco davidtwco mentioned this pull request Jul 8, 2019
@ashmaroli
Copy link
Contributor

The variables keywords, keys, values seem to be used just once across the entire lexer. You've two choices:

  • You can either instruct the Ruby interpreter to interpolate a regex just once (for performance reasons) by using the o flag: %r/#{foo}/o.
  • Or if you don't foresee a possibility for a new lexer being subclassed from the HOCON class, you may obliterate those variables entirely and have static regexps right away:
    # values
    rule %r/[^\$\"{}\[\]:=,\+#`\^\?!@\*&]+?/, Literal

@davidtwco
Copy link
Contributor Author

I've removed the variables.

@pyrmont pyrmont added the needs-review The PR needs to be reviewed label Jul 8, 2019
@pyrmont
Copy link
Contributor

pyrmont commented Jul 8, 2019

@davidtwco Thanks for submitting this again and I'm sorry you've had to wait so long :(

I'm looking at the informal spec and, given HOCON is a superset of JSON, it looks like this lexer would be best implemented as a subclass of Rouge::Lexers::JSON. You'd avoid needing to reimplement all the stuff that carries across from JSON and just be able to focus on the stuff that's different.

In addition to simply redefining a rule that you need to operate differently, Rouge comes with the methods prepend and append that you can use to manipulate an existing rule. You can see an example of using the JSON lexer as a superclass in the JSONDOC lexer.

@pyrmont pyrmont added author-action The PR has been reviewed but action by the author is needed and removed needs-review The PR needs to be reviewed labels Jul 8, 2019
@davidtwco
Copy link
Contributor Author

@pyrmont I've implemented this as a subclass of JSON, I think I've eliminated all the redundant rules and those that remain are necessary.

@pyrmont
Copy link
Contributor

pyrmont commented Jul 9, 2019

Wow, that's fast! I'll have a look over it and let you know if I spot anything :)

@pyrmont pyrmont added needs-review The PR needs to be reviewed and removed author-action The PR has been reviewed but action by the author is needed labels Jul 9, 2019
@davidtwco
Copy link
Contributor Author

Pushed again to (hopefully) fix Travis failures and fix a missing word in a comment.

lib/rouge/lexers/hocon.rb Outdated Show resolved Hide resolved
Copy link
Contributor

@pyrmont pyrmont left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got a few questions:

lib/rouge/lexers/hocon.rb Show resolved Hide resolved
lib/rouge/lexers/hocon.rb Outdated Show resolved Hide resolved
lib/rouge/lexers/hocon.rb Show resolved Hide resolved
lib/rouge/lexers/hocon.rb Outdated Show resolved Hide resolved
@pyrmont pyrmont added author-action The PR has been reviewed but action by the author is needed and removed needs-review The PR needs to be reviewed labels Jul 9, 2019
@pyrmont pyrmont merged commit cc8ffa1 into rouge-ruby:master Jul 11, 2019
@pyrmont pyrmont removed author-action The PR has been reviewed but action by the author is needed labels Jul 11, 2019
@davidtwco davidtwco deleted the add-hocon-lexer branch July 12, 2019 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants