#downcase! directive to use with ruby injections #1190

DerekStride · 2021-04-15T18:25:50Z

What

This is a follow-up to neovim/neovim#14152 which was superseded by neovim/neovim#14046 which allows setting the injected parser language on the metadata field of directives.

Add a `downcase!` directive

I moved the implementation of the #downcase! directive I proposed in neovim/neovim#14152 to this plugin as it seems to be a better home for it.

Often times the text matched by an @language node may not exactly resemble the name of the parser for that language. More often than not the parser name will be lowercase, i.e. html instead of HTML. In the below example ruby code a string constant is defined using ruby's heredoc syntax. The coding conventions for heredocs are to surround the string content with the name of the language in all uppercase letters ie. HTML, SQL, JAVASCRIPT, etc.

class A
  MY_HTML = <<~HTML
    <title>This is a title</title>
    <div class="example-class">
      <span>This is a span</span>
    </div>
  HTML
end

The downcase! directive is used in an queries/ruby/injections.scm file like so to convert the HTML value in the @language node to a valid lowercase html name of the parser.

(heredoc_body
 (heredoc_content) @content
 (heredoc_end) @language
 (#set! "language" @language)
 (#downcase! "language"))

Screenshot

stsewd · 2021-04-15T18:28:46Z

Thanks for the PR, I think we could do something more general like #812

theHamsta · 2021-04-15T19:43:39Z

@stsewd but isn't #812 capturing a node irrespective of case and this PR about processing a result of a capture? When we capture the string HTML/PYTHON we need to convert it to html/python only those are parser languages. We could also make the behavior of downcase! the default for language injection

stsewd · 2021-04-16T02:11:29Z

Both are about capturing a string from a node and matching it to a set of languages, for example the HTML parser would have an injection regex that's case-insensitive. But I agree we can just make it default to make it lowercase.

vigoux · 2021-04-16T06:01:28Z

lua/nvim-treesitter/query_predicates.lua

+query.add_directive('downcase!', function(match, _, bufnr, pred, metadata)
+  local node = match[pred[2]]
+  local text = query.get_node_text(node, bufnr)
+  local language = string.lower(text)
+  metadata.language = language
+end)


Although I agree with the implementation, it may be better to make it a bit more generic, maybe by providing the key to set instead of just using language this would make it look a bit like set, and may feel a little better.

@DerekStride vigoux is right. Could you add an argument that determines which metadata to downcase?

It could even be a modifier for set! In a future PR (set! "language" @foo downcase!) or directly apply on a key (set! "language" @foo) (downcase! "language") to chain multiple functions

Thanks for the feedback, I liked the suggestion to allow the directives to be chained, and made the corresponding changes.

(#set! "language" @foo) (#downcase! "language")

queries/ruby/injections.scm

vigoux · 2021-04-19T19:26:28Z

Hehe you'll say I am asking many things, but I'd rather go for a mimic of the signature of the set predicate.

The reason is that we don't have any guarantees on the order of evaluation of the directives, and thus making them standalone is better.
Does it sound reasonable to you?

DerekStride · 2021-04-19T19:30:17Z

I'd rather go for a mimic of the signature of the set predicate

I assume you mean to mimic the difference of when there are 3 vs 2 arguments, like what's listed below:

  ["set!"] = function(_, _, _, pred, metadata)
    if #pred == 4 then
      -- (#set! @capture "key" "value")
      metadata[pred[2]][pred[3]] = pred[4]
    else
      -- (#set! "key" "value")
      metadata[pred[2]] = pred[3]
    end

Then we can support (#downcase! @capture "key") too?

@node

``` (#downcase! "language") ``` downcase! will ensure the metadata value for the specified key will be downcased. If the value is a node, it will downcase the text specified by the node. ``` (#downcase! @node "key") ``` You can also namespace the key with a specific capture, similar to how you can call `(#set! @node "key" "value")`

DerekStride · 2021-04-21T13:10:39Z

lua/nvim-treesitter/query_predicates.lua

+  if #pred == 3 then
+    -- (#downcase! @capture "key")
+    key = pred[3]
+    value = metadata[pred[2]][key]
+  else
+    -- (#downcase! "key")
+    key = pred[2]
+    value = metadata[key]
+  end


@vigoux I made the changes that we discussed, I think I found a bug with how set was implemented in neovim. I'll be opening a PR with a fix.

Fix is up here: neovim/neovim#14418

steelsojka · 2021-04-23T12:40:07Z

I think a better solution would be something more generic that could simply look up the language based on a set of rules. For example, looking it up by a table. Since we can configure the language through a directive we could make it look it up any way we want to, whether that's by matching a regex pattern, a simple table lookup, or a transformation function of the language.

EDIT: Maybe we'll just save the more advanced stuff for another time...

steelsojka · 2021-04-23T12:41:45Z

@vigoux Is the order of the directives not guaranteed? I have some work that relies on that... 😢

Closes nvim-treesitter#812 Supersedes nvim-treesitter#1190

stsewd · 2021-04-23T18:23:56Z

I have implemented what I mean in #1190 (comment) #1238.

vigoux · 2021-04-24T05:19:05Z

@vigoux Is the order of the directives not guaranteed? I have some work that relies on that... 😢

I'll dig in tree-sitter at some point, but I am not sure it is...

theHamsta · 2021-06-03T21:45:41Z

@vigoux is this ready to merge?

vigoux · 2021-06-04T05:47:13Z

Yup, looks ready indeed.

vigoux reviewed Apr 16, 2021

View reviewed changes

theHamsta approved these changes Apr 16, 2021

View reviewed changes

theHamsta requested a review from vigoux April 19, 2021 18:03

vigoux reviewed Apr 19, 2021

View reviewed changes

queries/ruby/injections.scm Outdated Show resolved Hide resolved

DerekStride commented Apr 21, 2021

View reviewed changes

DerekStride mentioned this pull request Apr 21, 2021

[RDY] tree-sitter ensure there is a nested table allocated for #set! neovim/neovim#14418

Merged

vigoux added this to Review in progress in Core Apr 23, 2021

stsewd added a commit to stsewd/nvim-treesitter that referenced this pull request Apr 23, 2021

Use config.injection.pattern to match language to be injected

227bd2f

Closes nvim-treesitter#812 Supersedes nvim-treesitter#1190

stsewd mentioned this pull request Apr 23, 2021

[RFC] Use config.injections.aliases to inject a language #1238

Closed

vigoux merged commit e98e2ea into nvim-treesitter:master Jun 4, 2021

Core automation moved this from Review in progress to Done Jun 4, 2021

DerekStride deleted the downcase-directive branch June 7, 2021 15:15

calebdw mentioned this pull request Feb 17, 2023

Unable to dynamically inject languages into PHP #4333

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

#downcase! directive to use with ruby injections #1190

#downcase! directive to use with ruby injections #1190

DerekStride commented Apr 15, 2021 •

edited

stsewd commented Apr 15, 2021

theHamsta commented Apr 15, 2021

stsewd commented Apr 16, 2021

vigoux Apr 16, 2021

theHamsta Apr 16, 2021

theHamsta Apr 16, 2021 •

edited

DerekStride Apr 19, 2021

vigoux commented Apr 19, 2021

DerekStride commented Apr 19, 2021 •

edited

DerekStride Apr 21, 2021

DerekStride Apr 21, 2021

steelsojka commented Apr 23, 2021 •

edited

steelsojka commented Apr 23, 2021

stsewd commented Apr 23, 2021

vigoux commented Apr 24, 2021

theHamsta commented Jun 3, 2021

vigoux commented Jun 4, 2021

#downcase! directive to use with ruby injections #1190

#downcase! directive to use with ruby injections #1190

Conversation

DerekStride commented Apr 15, 2021 • edited

What

Add a downcase! directive

Screenshot

stsewd commented Apr 15, 2021

theHamsta commented Apr 15, 2021

stsewd commented Apr 16, 2021

vigoux Apr 16, 2021

Choose a reason for hiding this comment

theHamsta Apr 16, 2021

Choose a reason for hiding this comment

theHamsta Apr 16, 2021 • edited

Choose a reason for hiding this comment

DerekStride Apr 19, 2021

Choose a reason for hiding this comment

vigoux commented Apr 19, 2021

DerekStride commented Apr 19, 2021 • edited

DerekStride Apr 21, 2021

Choose a reason for hiding this comment

DerekStride Apr 21, 2021

Choose a reason for hiding this comment

steelsojka commented Apr 23, 2021 • edited

steelsojka commented Apr 23, 2021

stsewd commented Apr 23, 2021

vigoux commented Apr 24, 2021

theHamsta commented Jun 3, 2021

vigoux commented Jun 4, 2021

DerekStride commented Apr 15, 2021 •

edited

Add a `downcase!` directive

theHamsta Apr 16, 2021 •

edited

DerekStride commented Apr 19, 2021 •

edited

steelsojka commented Apr 23, 2021 •

edited