Cycling among different highlight files?

[sogaiu]

TLDR:

I've been working on highlights.scm files for tree-sitter-clojure and have begun to think that being able to cycle through some different highlighting schemes might be quite useful (please see below for background of the idea).

I scanned the current emacs lisp files a bit and the impression I got was that the idea is to have basically have one type of highlighting (possibly with customizations?). Does that seem correct?

If the idea of cycling through some different highlighting schemes sounds palatable, any suggestions / pointers on implementing such functionality?

Thanks for the consideration.

Background:

I watched a talk (https://youtu.be/l1b7Da2DnPo?t=831) by @tonsky recently where he discusses his alabaster theme: https://github.com/tonsky/sublime-scheme-alabaster

I implemented an approximation to it, gave it a try and found that it might be helpful for me sometimes (e.g. primarily when first examining unfamiliar code), but I doubt I would find it useful all of the time.

To give a few examples, it makes comments stand out, which I find helpful sometimes but not usually. It also does not highlight "built-in" functions / macros / special forms. When I am not yet familiar with a particular language, I would prefer to have such things highlighted and if possible be able to tell built-ins apart from "user-defined". Even if I am familiar, I thought I might be helped to be able to temporarily turn on such highlighting.

FWIW, the aforementioned repository has some details on the rationale behind some of his decisions and I found this to be interesting reading. I don't happen to agree with multiple issues, but I would guess that whatever I were to express as a list of important points, someone else would likely have their own differing opinion.

These experiences suggest to me that perhaps it would be useful to have a few different highlighting schemes one can switch among for different purposes. Perhaps one can think of them as different "glasses" for different occasions.

The query syntax seems to be evolving to make more things possible (e.g. possibly some form of not quantifier may show up at some point: tree-sitter/tree-sitter#705) and my experience with it so far suggests it may be easier to put together (as well as modify) different schemes than some previous valiant efforts (https://github.com/clojure-emacs/clojure-mode/blob/master/clojure-mode.el#L750-L926).

[sogaiu]

Below is a quick proof-of-concept.

There is one interactive function treesitter-cycle-hl -- the idea is to keep invoking it to get different highlighting applied.

(defvar treesitter-cycle-hl--current-index-alist
  '((clojure . 0)))

(defun treesitter-cycle-hl--current-index ()
  (when-let ((lang-sym (alist-get major-mode tree-sitter-major-mode-language-alist)))
    (alist-get lang-sym treesitter-cycle-hl--current-index-alist)))

(defun treesitter-cycle-hl--set-current-index (new-val)
  (when-let ((lang-sym (alist-get major-mode tree-sitter-major-mode-language-alist)))
    (when (treesitter-cycle-hl--current-index)
      (setf (alist-get lang-sym treesitter-cycle-hl--current-index-alist)
            new-val))))

(defvar treesitter-cycle-hl--file-paths-alist
  '((clojure .
             ("~/.emacs.d/straight/repos/emacs-tree-sitter/langs/queries/clojure/highlights.scm"
              "~/src/emacs-tree-sitter.sogaiu/langs/queries/clojure/highlights.scm"))))

(defun treesitter-cycle-hl--current-file-paths ()
  (when-let ((lang-sym (alist-get major-mode tree-sitter-major-mode-language-alist)))
    (alist-get lang-sym treesitter-cycle-hl--file-paths-alist)))

(defun treesitter-cycle-hl ()
  "Cycle to logically next highlight patterns and highlight based on result."
  (interactive)
  (when tree-sitter-hl-mode
    (tree-sitter-hl-mode -1))
  (treesitter-cycle-hl--set-current-index (1+ (treesitter-cycle-hl--current-index)))
  (when (>= (treesitter-cycle-hl--current-index) (length (treesitter-cycle-hl--current-file-paths)))
    (treesitter-cycle-hl--set-current-index 0))
  ;; tree-sitter-langs--hl-default-patterns
  (let ((index (treesitter-cycle-hl--current-index))
        (file-paths (treesitter-cycle-hl--current-file-paths)))
    (setq tree-sitter-hl-default-patterns
          (condition-case nil
              (with-temp-buffer
                (insert-file-contents (nth index file-paths))
                (goto-char (point-max))
                (insert "\n")
                (buffer-string))
            (file-missing nil))))
  (tree-sitter-hl-mode))

[ubolonton]

I scanned the current emacs lisp files a bit and the impression I got was that the idea is to have basically have one type of highlighting (possibly with customizations?). Does that seem correct?

tree-sitter-hl's interface is the variable tree-sitter-hl-default-patterns, to be set by major modes, which can implement any pattern-cycling scheme they see fit. (Its role is similar to that of font-lock-defaults.) The "one type of highlighting" limitation is from the package tree-sitter-langs, which acts as a temporary surrogate for future tree-sitter-based language-specific modes.

If the idea of cycling through some different highlighting schemes sounds palatable, any suggestions / pointers on implementing such functionality?

Your example shows how to do it currently: disable tree-sitter-hl-mode, set tree-sitter-hl-default-patterns, and re-enable tree-sitter-hl-mode. It should work well, apart from the minor inefficiency of recreating the query cursor.

To give a few examples, it makes comments stand out, which I find helpful sometimes but not usually. It also does not highlight "built-in" functions / macros / special forms. When I am not yet familiar with a particular language, I would prefer to have such things highlighted and if possible be able to tell built-ins apart from "user-defined". Even if I am familiar, I thought I might be helped to be able to temporarily turn on such highlighting.

FWIW, the aforementioned repository has some details on the rationale behind some of his decisions and I found this to be interesting reading. I don't happen to agree with multiple issues, but I would guess that whatever I were to express as a list of important points, someone else would likely have their own differing opinion.

These experiences suggest to me that perhaps it would be useful to have a few different highlighting schemes one can switch among for different purposes. Perhaps one can think of them as different "glasses" for different occasions.

I agree that being able to quickly change highlighting scheme on-the-fly is very useful. I just don't know what's the reasonable API tree-sitter-langs can provide for that, yet. It would take some design work. As a temporary solution, I don't want tree-sitter-langs to dictate too much of an API too early (e.g. global vs local toggles, customization options, whether the schemes should be sorted, or named...)

P.S. (alist-get major-mode tree-sitter-major-mode-language-alist) can be replaced with (tsc--lang-symbol tree-sitter-language). (I'll probably make that function non-internal soon.)

[sogaiu]

Thank you very much for the detailed response, including the explanation, code tips, and heads up about possibly upcoming changes 👍

I hope over time some good / reasonable API is arrived at.

[sogaiu]

On a related note, @pedrorgirardi told me about the following repository: https://github.com/rougier/nano-emacs

There is an associated paper that has the following text within it:

The most striking example is certainly the syntax colorization that seems to go against every good design principles in a majority of text editors and the motto guiding design could be summarized by "Let's add mor colors" (using regex).

There are other interesting observations in the paper too that are not necessarily relevant here, but the paper does have a section titled "Colorization" that might be of interest to some.