Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow escaped quote literals in the SyntaxScopeMap #19336

Merged
merged 1 commit into from May 15, 2019

Conversation

Projects
None yet
2 participants
@nathansobo
Copy link
Contributor

commented May 14, 2019

As raised by @Ben3eeE in atom/language-json#68 (comment), we are currently unable to apply decorative syntax scopes to " literals for TreeSitter grammars, which is forcing us to break user expectations around how syntax themes apply to strings.

In this PR, I add some special handling for escaped " literals to the SyntaxScopeMap. Internally, the SyntaxScopeMap uses the postcss-selector-parser library to parse any scope it is provided. This library allows " literals inside of selectors, but they need to be escaped in the selector. So a valid post CSS selector looks like this:

string > "\""

Unfortunately, these selectors are themselves expressed as strings within CSON or JSON files in our grammars. That means the escaping gets a bit intense, and looks like this:

'string > "\\""': 'puncuation.quote.double.whatever'

Notice that I include 2 backslashes when escaping the quote so that they end up getting passed through to the parser as a single \. If I only include one backslash it ends up looking like an escaped quote at the interpretation level of CSON rather than the level of the selector language. 馃槗

Anyway, once properly escaped, the quote literal is handled by the parser, but the value it spits out continues to include the escaping backslash. In order to match the text of the anonymous nodes returned by TreeSitter, we need to remove the escaping after trimming off the outer quotes.

@Ben3eeE hopefully this gives you what you need to style quotes properly in various grammars.

cc @50Wliu @maxbrunsfeld

Allow escaped " literals in the SyntaxScopeMap
In order to pass a quote literal to the postcss-selector-parser, it 
needs to be escaped. However, this escaping is not removed by the parser 
in the yielded value token, so in this commit I replace any escaped 
quote literals with unescaped quotes after we trim the outer quotes off 
of the string.

@nathansobo nathansobo referenced this pull request May 14, 2019

Merged

Add tree-sitter json grammar #68

2 of 2 tasks complete
@maxbrunsfeld
Copy link
Contributor

left a comment

馃憤 Makes sense!

It's not surprising that there are some gotchas with the CSS-within-CSON approach I went with here 馃槵. Glad to see you were able to solve this issue though.

As an aside - We've since developed a system where you write this stuff in regular css files, and the CSS files get compiled ahead-of-time into JSON state machines that let you match the properties even more efficiently than the SyntaxScopeMap. I don't know if it would be worth porting this to work in Atom though. It'd be a fun project, but it'd mean another API change.

@nathansobo

This comment has been minimized.

Copy link
Contributor Author

commented May 14, 2019

That CSS syntax is pretty sweet. Curious about this and the state machine, but it's probably low on the list of priorities right now for integration into Atom.

@nathansobo

This comment has been minimized.

Copy link
Contributor Author

commented May 15, 2019

Flaky test in GitHub package that's definitely unrelated. Merging.

@nathansobo nathansobo merged commit 4e5e2e5 into master May 15, 2019

1 of 2 checks passed

Atom Pull Requests #20190514.10 failed
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details

@nathansobo nathansobo deleted the ns/allow-quote-selectors branch May 15, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can鈥檛 perform that action at this time.