Emacs Snippets not working anymore with Tree Sitter Major Modules #1169

nicemaker · 2023-07-28T04:49:04Z

I started to use only the new build in tree-sitter for my languages.
So for example instead of typescipt-mode I am now using typescipt-ts-mode.
Since I switched yasnippets are not showing up anymore. I guess yasnippet doesn't understand that typescript-ts-mode needs the same snippets as typescript mode.

I was basically following instructions here:
https://www.masteringemacs.org/article/how-to-get-started-tree-sitter

I was hoping that the major-mode-remap list would take care of that mapping, but that doesn't seem to work for yasnippet.
Any ideas?

(setq major-mode-remap-alist
 '((typescript-mode . typescript-ts-mode)
   (python-mode . python-ts-mode)))

The text was updated successfully, but these errors were encountered:

cybergrind · 2023-08-10T11:07:19Z

For you you can get in working by adding (yas-activate-extra-mode 'python-mode) to python-ts-mode-hook

twlz0ne · 2023-08-10T13:41:49Z

My workaround:

(advice-add 'yas--modes-to-activate :around
 (defun yas--get-snippet-tables@tree-sitter (orig-fn &optional mode)
   (funcall orig-fn
            (or (car (rassq (or mode major-mode) major-mode-remap-alist))
                mode))))

jvillasante · 2023-11-22T21:48:12Z

@twlz0ne This is not working for me, at least for C++. Any other workaround?

jvillasante · 2023-11-23T21:59:36Z

In case somebody needs a workaround to work today, I found this that seems to work

twlz0ne · 2023-11-24T07:24:12Z

@jvillasante Don't know why not working for you, the advice above just tells yas to use /path/to/snippets/lang-mode instead of /path/to/snippets/lang-ts-mode. Have you checked the major-mode-remap-alist and snippet dirs?

(car (rassq 'c++-ts-mode major-mode-remap-alist))
;; => c++-mode

(and (yas--get-snippet-tables 'c++-mode) t)
;; => t

aspiers · 2023-12-30T20:00:26Z

In case somebody needs a workaround to work today, I found this that seems to work

Thanks a lot @jvillasante, yasnippet-treesitter-shim looks like a great workaround - although IMHO really this should work out of the box with yasnippet rather than needing a separate package as a bandaid 😞

Bandaid to fix yasnippet not working with treesit modes out of the box: joaotavora/yasnippet#1169 fbrosda/yasnippet-treesitter-shim#1

aspiers · 2023-12-30T22:32:40Z

I got the shim working with use-package and straight.el and submitted fbrosda/yasnippet-treesitter-shim#1 to document this so that others can install it more easily.

Should hopefully fix issue #1169.

monnier · 2024-01-01T21:19:50Z

looks like a great workaround - although IMHO really this should work out of the box with yasnippet rather than needing a separate package as a bandaid 😞

I just pushed a patch to `master` which should hopefully help. Can someone confirm it fixes the problem for them?

aspiers · 2024-01-02T21:04:58Z

@monnier Thanks so much for working on this! I have just tested, and can confirm it works 🚀

BTW I tested it by setting major-mode-remap-alist with the help of treesit-auto.el, although I note that that package only sets major-mode-remap-alist locally by advising set-auto-mode-0:

https://github.com/renzmann/treesit-auto/blob/07a8f924cd4f020a2eb32b45d8543af9556f355d/treesit-auto.el#L466-L484

So I had to set it globally via:

M-: (setq major-mode-remap-alist (treesit-auto--build-major-mode-remap-alist))

This means that while your commit is working as intended, it won't work out of the box with treesit-auto.el yet. I don't yet understand why the latter is only setting major-mode-remap-alist locally not globally, so I'm not sure what the best full solution would be, but I guess the responsibility for fixing that is more likely to lie with treesit-auto.el rather than yasnippet.

aspiers · 2024-01-02T21:16:09Z

I wrote:

This means that while your commit is working as intended, it won't work out of the box with treesit-auto.el yet. I don't yet understand why the latter is only setting major-mode-remap-alist locally not globally, so I'm not sure what the best full solution would be, but I guess the responsibility for fixing that is more likely to lie with treesit-auto.el rather than yasnippet.

I've filed renzmann/treesit-auto#76 to track this.

monnier · 2024-01-03T21:57:32Z

This means that while your commit is working as intended, it won't work out of the box with `treesit-auto.el` yet.

Hmm... I wonder why my code isn't seeing the buffer-local value that `treesit-auto` sets for `major-mode-remap-alist`. AFAICT the yasnippet code should be run inside the buffer and after `treesit-auto` intervened, so it should see the relevant value. Could you `M-x trace-function RET` on the following functions: yas--modes-to-activate yas-minor-mode yas--get-snippet-tables yas--load-pending-jits treesit-auto--set-major-remap set-auto-mode-0 and then see what the trace looks like?

aspiers · 2024-01-04T13:06:28Z

I will do that when I get a moment (how did I use emacs for 30 years and only just hear about trace-function?!), but I have already seen a weird thing which might explain this:

Somehow treesit-auto correctly executes the setq-local when the file is first visited (proven via edebug), but then when the buffer is opened, major-mode-remap-alist somehow reverts to nil.

aspiers · 2024-01-04T13:10:17Z

I've commented the same point in renzmann/treesit-auto#76 (comment), so perhaps that would be a better place to continue this conversation for now, since it currently looks like more of an issue with treesitter-auto rather than yasnippet.

monnier · 2024-01-04T15:17:28Z

*Where* do you see that it has value nil? Since it's a buffer-local setting, the buffer in which you are when you look at the value can make all the difference. You might also want to use `M-x debug-on-variable-change` on `major-mode-remap-alist` (which reminds me that we should also add `trace-variable` to `trace.el`).

aspiers · 2024-01-04T15:29:16Z

Yep, I appreciate the difference and did check that carefully: I see it as nil in the buffer of the (Typescript) file where I want yasnippet activated. To clarify, this is what I did:

Instrument treesit-auto--set-major-remap with Edebug.
Load the Typescript file
Step through treesit-auto--set-major-remap and verify that major-mode-remap-alist is being set correctly. (At this point, pressing e to evaluate (buffer-name) in the context of the debugger proves that it's being set on the buffer of the Typescript file. In contrast, doing M-: (buffer-name) at this point returns treesit-auto.el as expected.)
Allow treesit-auto--set-major-remap to complete, so that the Typescript file's buffer is now open.
Observe that C-h v major-mode-remap-alist now shows nil.

It's strange, but perhaps something else is resetting the variable to nil. Or more likely, I'm just being an idiot somehow.

Anyhow, I will try debug-on-variable-change which is another great trick I somehow never discovered before! Perhaps I should find more issues so that I can continue learning from you ;-)

aspiers · 2024-01-04T15:49:10Z

OK, this is interesting and seems relevant! I tried debug-on-variable-change. Sure enough, it triggered the debugger as expected:

Debugger entered--setting major-mode-remap-alist in buffer Options.tsx to ((yaml-mode . yaml-ts-mode) (typescript-mode . typescript-ts-mode) (js2-mode . js-ts-mode) (javascript-mode . js-ts-mode) (js-mode . js-ts-mode) (sh-mode . bash-ts-mode)): 
  debug--implement-debug-watch(major-mode-remap-alist ((yaml-mode . yaml-ts-mode) (typescript-mode . typescript-ts-mode) (js2-mode . js-ts-mode) (javascript-mode . js-ts-mode) (js-mode . js-ts-mode) (sh-mode . bash-ts-mode)) set #<buffer Options.tsx>)
  set(major-mode-remap-alist ((yaml-mode . yaml-ts-mode) (typescript-mode . typescript-ts-mode) (js2-mode . js-ts-mode) (javascript-mode . js-ts-mode) (js-mode . js-ts-mode) (sh-mode . bash-ts-mode)))
  treesit-auto--set-major-remap(tsx-ts-mode nil)
  apply(treesit-auto--set-major-remap (tsx-ts-mode nil))
  set-auto-mode-0(tsx-ts-mode nil)
  set-auto-mode--apply-alist((("\\(\\.js[mx]\\|\\.har\\)\\'" . js-ts-mode) ("\\.ya?ml\\'" . yaml-ts-mode) ("\\.ts\\'" . typescript-ts-mode) ("\\.tsx\\'" . tsx-ts-mode) ("\\.toml\\'" . toml-ts-mode) ("\\.rs\\'" . rust-ts-mode) ("\\(?:\\.\\(?:rbw?\\|ru\\|rake\\|thor\\|jbuilder\\|rabl\\|ge..." . ruby-ts-mode) ("\\.py[iw]?\\'" . python-ts-mode) ("\\.json\\'" . json-ts-mode) ("\\.js\\'" . js-ts-mode) ("\\.java\\'" . java-ts-mode) ("go\\.mod\\'" . go-mod-ts-mode) ("\\.go\\'" . go-ts-mode) ("\\Dockerfile\\'" . dockerfile-ts-mode) ("\\.css\\'" . css-ts-mode) ("\\.cpp\\'" . c++-ts-mode) ("\\.cmake\\'" . cmake-ts-mode) ("\\.cs\\'" . csharp-ts-mode) ("\\.c\\'" . c-ts-mode) ("\\.sh\\'" . bash-ts-mode) ("\\.odc\\'" . archive-mode) ("\\.odf\\'" . archive-mode) ("\\.odi\\'" . archive-mode) ("\\.otp\\'" . archive-mode) ("\\.odp\\'" . archive-mode) ("\\.otg\\'" . archive-mode) ("\\.odg\\'" . archive-mode) ("\\.ots\\'" . archive-mode) ("\\.ods\\'" . archive-mode) ("\\.odm\\'" . archive-mode) ("\\.ott\\'" . archive-mode) ("\\.odt\\'" . archive-mode) ("\\(?:\\(?:\\.\\(?:b\\(?:\\(?:abel\\|ower\\)rc\\)\\|json\\(?:l..." . json-mode) ("\\.scss\\'" . sass-mode) ("\\.sass\\'" . sass-mode) ("\\.haml\\'" . haml-mode) ("\\.\\(phtml\\|tpl\\.php\\|jsp\\|as[cp]x\\|erb\\|mustache\\|..." . web-mode) ("\\.json\\'" . json-mode) ("\\.ya?ml\\'" . yaml-mode) ("\\.\\(e?ya?\\|ra\\)ml\\'" . yaml-mode) ("\\.feature\\'" . feature-mode) ("/\\.zsh\\(env\\|rc\\|/functions/\\)\\|\\.stp$" . sh-mode) ("\\.sc\\'" . sclang-mode) ("\\.gem\\'" . tar-mode) ("\\(\\.\\(e?rb\\|rjs\\|rake\\)\\|Rakefile\\|Guardfile\\)\\'" . ruby-mode) ("\\.py\\'" . python-mode) ("\\.php\\'" . php-mode) ("\\.\\(?:php[s345]?\\|phtml\\)\\'" . php-mode-maybe) ("\\.\\(?:php\\.inc\\|stub\\)\\'" . php-mode) ("/\\.php_cs\\(?:\\.dist\\)?\\'" . php-mode) ...) nil nil)
  #<subr set-auto-mode>()
  so-long--set-auto-mode(#<subr set-auto-mode>)
  apply(so-long--set-auto-mode #<subr set-auto-mode> nil)
  set-auto-mode()
  normal-mode(t)
  after-find-file(nil nil t nil nil)

However after continuing from this, the debugger reactivated again with this:

Debugger entered--killing local value of major-mode-remap-alist in buffer Options.tsx: 
  debug--implement-debug-watch(major-mode-remap-alist nil makunbound #<buffer Options.tsx>)
  prog-mode()
  typescript-ts-base-mode()
  tsx-ts-mode()
  #<subr set-auto-mode-0>(tsx-ts-mode nil)
  apply(#<subr set-auto-mode-0> (tsx-ts-mode nil))
  set-auto-mode-0(tsx-ts-mode nil)
  set-auto-mode--apply-alist((("\\(\\.js[mx]\\|\\.har\\)\\'" . js-ts-mode) ("\\.ya?ml\\'" . yaml-ts-mode) ("\\.ts\\'" . typescript-ts-mode) ("\\.tsx\\'" . tsx-ts-mode) ("\\.toml\\'" . toml-ts-mode) ("\\.rs\\'" . rust-ts-mode) ("\\(?:\\.\\(?:rbw?\\|ru\\|rake\\|thor\\|jbuilder\\|rabl\\|ge..." . ruby-ts-mode) ("\\.py[iw]?\\'" . python-ts-mode) ("\\.json\\'" . json-ts-mode) ("\\.js\\'" . js-ts-mode) ("\\.java\\'" . java-ts-mode) ("go\\.mod\\'" . go-mod-ts-mode) ("\\.go\\'" . go-ts-mode) ("\\Dockerfile\\'" . dockerfile-ts-mode) ("\\.css\\'" . css-ts-mode) ("\\.cpp\\'" . c++-ts-mode) ("\\.cmake\\'" . cmake-ts-mode) ("\\.cs\\'" . csharp-ts-mode) ("\\.c\\'" . c-ts-mode) ("\\.sh\\'" . bash-ts-mode) ("\\.odc\\'" . archive-mode) ("\\.odf\\'" . archive-mode) ("\\.odi\\'" . archive-mode) ("\\.otp\\'" . archive-mode) ("\\.odp\\'" . archive-mode) ("\\.otg\\'" . archive-mode) ("\\.odg\\'" . archive-mode) ("\\.ots\\'" . archive-mode) ("\\.ods\\'" . archive-mode) ("\\.odm\\'" . archive-mode) ("\\.ott\\'" . archive-mode) ("\\.odt\\'" . archive-mode) ("\\(?:\\(?:\\.\\(?:b\\(?:\\(?:abel\\|ower\\)rc\\)\\|json\\(?:l..." . json-mode) ("\\.scss\\'" . sass-mode) ("\\.sass\\'" . sass-mode) ("\\.haml\\'" . haml-mode) ("\\.\\(phtml\\|tpl\\.php\\|jsp\\|as[cp]x\\|erb\\|mustache\\|..." . web-mode) ("\\.json\\'" . json-mode) ("\\.ya?ml\\'" . yaml-mode) ("\\.\\(e?ya?\\|ra\\)ml\\'" . yaml-mode) ("\\.feature\\'" . feature-mode) ("/\\.zsh\\(env\\|rc\\|/functions/\\)\\|\\.stp$" . sh-mode) ("\\.sc\\'" . sclang-mode) ("\\.gem\\'" . tar-mode) ("\\(\\.\\(e?rb\\|rjs\\|rake\\)\\|Rakefile\\|Guardfile\\)\\'" . ruby-mode) ("\\.py\\'" . python-mode) ("\\.php\\'" . php-mode) ("\\.\\(?:php[s345]?\\|phtml\\)\\'" . php-mode-maybe) ("\\.\\(?:php\\.inc\\|stub\\)\\'" . php-mode) ("/\\.php_cs\\(?:\\.dist\\)?\\'" . php-mode) ...) nil nil)
  #<subr set-auto-mode>()
  so-long--set-auto-mode(#<subr set-auto-mode>)
  apply(so-long--set-auto-mode #<subr set-auto-mode> nil)
  set-auto-mode()
  normal-mode(t)
  after-find-file(nil nil t nil nil)

I can't figure out why prog-mode would cause the local value of major-mode-remap-alist to be killed, but I'm guessing you'll know the reason immediately :)

monnier · 2024-01-04T16:23:51Z

However after continuing from this, the debugger reactivated again with this: ``` Debugger entered--killing local value of major-mode-remap-alist in buffer Options.tsx: debug--implement-debug-watch(major-mode-remap-alist nil makunbound #<buffer Options.tsx>) prog-mode()

Ah, yes, of course: the first thing that a major mode does is kill "all" buffer-local variables (basically, to "reset" the settings installed by any previous major mode).

aspiers · 2024-01-04T17:09:12Z

Hrm, so does this mean that treesit-auto's approach is fundamentally flawed?

monnier · 2024-01-04T17:43:50Z

Hrm, so does this mean that `treesit-auto`'s approach is fundamentally flawed?

Don't know, but it does mean that Yasnippet can't get treesit-auto's information via `major-mode-remap-alist` for free. Stefan

aspiers · 2024-01-04T17:49:39Z

OK thanks, so for now the workarounds are either to use https://github.com/fbrosda/yasnippet-treesitter-shim or my hack of setting it globally:

(setq major-mode-remap-alist (treesit-auto--build-major-mode-remap-alist))

I think any further discussion on the treesit-auto side of things belongs in renzmann/treesit-auto#76.

monnier · 2024-01-04T20:23:40Z

OK thanks, so for now the workarounds are either to use https://github.com/fbrosda/yasnippet-treesitter-shim or [my hack](aspiers/emacs@b579a4c) of setting it globally: ```elisp (setq major-mode-remap-alist (treesit-auto--build-major-mode-remap-alist)) ```

Yes. `treesit-auto` (and `major-mode-remap-alist`) is designed to redirect FOO-mode to FOO-ts-mode, whereas YASnippet needs something different, which is to know that FOO-ts-mode is a "kind of" FOO-mode. This should be used both for the users that want to redirect FOO-mode to FOO-ts-mode and for those that don't do that but which invoked FOO-ts-mode some other way (such as manually). So I think the canonical way should be for the major mode to use `derived-mode-add-parents` to declare its relationship. I just pushed to `master` a patch which makes YASnippet use `derived-mode-all-parents` so as to obey such settings. Since this functionality is new in Emacs-30, the code still tries to use `major-mode-remap-alist`, but that's just a crutch which will cover only *some* configs 🙁. BTW, instead of having "dummy" snippets with `.yas-parents` pointing to the real snippets, like `yasnippet-treesitter-shim` does, maybe we should allow snippets to have not only `.yas-parents` but also `.yas-children`.

aspiers · 2024-01-05T12:35:22Z

Yep, that all makes sense.

aspiers added a commit to aspiers/emacs that referenced this issue Dec 30, 2023

add yasnippet-treesitter-shim

f36855f

Bandaid to fix yasnippet not working with treesit modes out of the box: joaotavora/yasnippet#1169 fbrosda/yasnippet-treesitter-shim#1

monnier added a commit that referenced this issue Jan 1, 2024

(yas--modes-to-activate): Pay attention to major-mode-remap-alist

b86b44c

Should hopefully fix issue #1169.

aspiers mentioned this issue Jan 2, 2024

Doesn't work with yasnippet renzmann/treesit-auto#76

Open

monnier closed this as completed Feb 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Emacs Snippets not working anymore with Tree Sitter Major Modules #1169

Emacs Snippets not working anymore with Tree Sitter Major Modules #1169

nicemaker commented Jul 28, 2023

cybergrind commented Aug 10, 2023

twlz0ne commented Aug 10, 2023

jvillasante commented Nov 22, 2023

jvillasante commented Nov 23, 2023

twlz0ne commented Nov 24, 2023 •

edited

Loading

aspiers commented Dec 30, 2023

aspiers commented Dec 30, 2023

monnier commented Jan 1, 2024 via email

aspiers commented Jan 2, 2024

aspiers commented Jan 2, 2024

monnier commented Jan 3, 2024 via email

aspiers commented Jan 4, 2024

aspiers commented Jan 4, 2024

monnier commented Jan 4, 2024 via email

aspiers commented Jan 4, 2024

aspiers commented Jan 4, 2024

monnier commented Jan 4, 2024 via email •

edited

Loading

aspiers commented Jan 4, 2024

monnier commented Jan 4, 2024 via email

aspiers commented Jan 4, 2024

monnier commented Jan 4, 2024 via email

aspiers commented Jan 5, 2024

Emacs Snippets not working anymore with Tree Sitter Major Modules #1169

Emacs Snippets not working anymore with Tree Sitter Major Modules #1169

Comments

nicemaker commented Jul 28, 2023

cybergrind commented Aug 10, 2023

twlz0ne commented Aug 10, 2023

jvillasante commented Nov 22, 2023

jvillasante commented Nov 23, 2023

twlz0ne commented Nov 24, 2023 • edited Loading

aspiers commented Dec 30, 2023

aspiers commented Dec 30, 2023

monnier commented Jan 1, 2024 via email

aspiers commented Jan 2, 2024

aspiers commented Jan 2, 2024

monnier commented Jan 3, 2024 via email

aspiers commented Jan 4, 2024

aspiers commented Jan 4, 2024

monnier commented Jan 4, 2024 via email

aspiers commented Jan 4, 2024

aspiers commented Jan 4, 2024

monnier commented Jan 4, 2024 via email • edited Loading

aspiers commented Jan 4, 2024

monnier commented Jan 4, 2024 via email

aspiers commented Jan 4, 2024

monnier commented Jan 4, 2024 via email

aspiers commented Jan 5, 2024

twlz0ne commented Nov 24, 2023 •

edited

Loading

monnier commented Jan 4, 2024 via email •

edited

Loading