Some modes require a second mode activation

[blaenk]

I'm not quite sure how to describe this, but for example with sql-mode buffers, when I first open the buffer, TODOs in comments aren't highlighted. If I disable the mode, then enable it again, then the keywords are highlighted. I checked and sql-mode derives prog-mode.

[dschrempf]

I am having the same problem with latex-mode which is not derived from prog-mode but I have hl-todo explicitely enabled. The TODO keywords are found with, e.g., hl-todo-next (but not highlighted). Deactivation with subsequent activation enables highlighting.

[tarsius]

Yeah me too. It's quite annoying but I never got around to investigate enough to actually fix it because I suspect doing so would involve gaining a deep understanding of how font-locking works and implementing functionality which ought to already exist.

Back in the days package authors used font-lock-fontify-buffer which wasn't intended for this purpose of making new keywords take affect but by accident accomplished to do so (sometimes).

(defun font-lock-fontify-buffer (&optional interactively)
  "Fontify the current buffer the way the function `font-lock-mode' would."
  (declare
   ;; When called from Lisp, this function is a big mess.  The caller usually
   ;; expects one of the following behaviors:
   ;; - refresh the highlighting (because the font-lock-keywords have been
   ;;   changed).
   ;; - apply font-lock highlighting even if font-lock-mode is not enabled.
   ;; - reset the highlighting rules because font-lock-defaults
   ;;   has been changed (and then rehighlight everything).
   ;; Of course, this function doesn't do all of the above in all situations
   ;; (e.g. depending on whether jit-lock is in use) and it can't guess what
   ;; the caller wants.
   (interactive-only "use `font-lock-ensure' or `font-lock-flush' instead."))
  (interactive "p")
  (font-lock-set-defaults)
  (let ((font-lock-verbose (or font-lock-verbose interactively)))
    (funcall font-lock-fontify-buffer-function)))

The commit that added the above comment, emacs-mirror/emacs@6711a21, also added font-lock-ensure and font-lock-flush.

I tried using the former instead for a while because its doc-string says that (when called without any arguments) it makes sure that the whole buffer is fontified, but apparently that only means that ensures that the buffer has been fontified at some point and not that it ensures that all currently defined keywords actually are in effect.

So I started to also call font-lock-flush before font-lock-ensure, the former "declares" that the fontification is out-of-date, and the latter "ensures" that fontification is in effect, though apparently (in some cases only?) this still isn't enough to actually put the newly added keywords into effect.

(define-minor-mode hl-todo-mode
  "Highlight TODO and similar keywords in comments and strings."
  :lighter ""
  :keymap hl-todo-mode-map
  :group 'hl-todo
  (if hl-todo-mode
      (hl-todo--setup)
    (font-lock-remove-keywords nil hl-todo--keywords))
  (when font-lock-mode
    (if (and (fboundp 'font-lock-flush)
             (fboundp 'font-lock-ensure))
        (save-restriction
          (widen)
          (font-lock-flush)
          (font-lock-ensure))
      (with-no-warnings
        (font-lock-fontify-buffer)))))

But those are the functions added to do this sort of thing and I don't know what else I could do to make the new keywords take effect without implementing it myself. Even if these functions reliably did what I think they were designed to do, they would still be problematic because they refontify the whole buffer (when they actually do, which, as we have learned, doesn't appear to be always the case), then that would be wasteful (leading to issues such as #22).

What I would really like to be able to use is (font-lock-flush-and-ensure-additional-keywords-on-areas-which-have-already-been-fontified NEWLY-ADDED-KEYWORDS).

[tarsius]

Please try out #29.

[blaenk]

Didn't seem to fix the issue. Tested with sql-mode.

Care to try it @dschrempf?

[tarsius]

I am unable to locate an sql-mode.el in Emacs or the Emacsmirror. Did you mean sql.el, which defines a mode sql-mode? Else please tell me where I can obtain sql-mode.el.

The version of sql-mode.el that comes with Emacs 26.1 contains four instances of XXX (and no other keywords), all of which are highlighted for me after hl-todo-mode is turned on using prog-mode (indirectly, using global-hl-todo-mode, without customizing hl-todo-activate-in-modes. Do you use another hook to activate hl-todo-mode? If so, please show me your complete setup for this feature.

[tarsius]

Oh! If you say "tested with sql-mode" you probably mean a buffer whose major-mode is sql-mode, right? If so please post such a file that I could use for testing.

@dschrempf I assume you were talking about latex-mode buffers, not a buffer visiting tex-mode.el. Did you customize hl-todo-activate-in-modes? latex-mode derives from tex-mode, which derives from text-mode, which is not a member of hl-todo-activate-in-modes by default.

[dschrempf]

Yes, I was talking about buffers using latex-mode. The help about the mode actually says: LaTeX/MPS mode defined in ‘tex-mode.el’.

As far as I can see the problem has not been fixed. I am using hl-todo-20181003.1521 from MELPA, I hope this is the correct version. Let me know, if I should specificly test it with this repository.

The value of hl-todo-avtivate-in-modes is (latex-mode prog-mode). I tried to change it to (TeX-latex-mode prog-mode), but that did not help either (also text-mode doesn't help).

Also hl-mode is reporting to be activated (but the keywords are only highlighted after deactivating it with subsequence activation):

hl-todo-mode is a variable defined in ‘hl-todo.el’.
Its value is t
Local in buffer MBE-PoMo.tex; global value is nil

  Automatically becomes buffer-local when set.

Documentation:
Non-nil if Hl-Todo mode is enabled.
Use the command ‘hl-todo-mode’ to change this variable.

[back]

I don't know why this is not working, it is working fine with emacs-lisp-mode and also other modes if I remember correctly. Maybe LaTeX mode is doing some highlighting itself? Thanks a lot for trying to fix this issue!

[tarsius]

I found the issue and fixed it in cdc2266.

[blaenk]

Confirmed. Great work as usual! 👍

[dschrempf]

I am a little bit at a loss, because this still affects me in latex-mode. hl-todo-mode works in all other buffers as far as I can see. I latex-mode, a TODO keyword in a comment line has the following text properties

There are text properties here:
  face                 font-lock-comment-face
  font-lock-multiline  t
  fontified            t

The text properties of the same TODO keyword in a CPP buffer, where the highlighting works perfectly:

There are text properties here:
  c-in-sws             t
  face                 (:inherit hl-todo :foreground "#dc752f")
  fontified            t

If I disable and enable hl-todo-mode in the LaTeX buffer, the hihglighting works. However, when I save the file, the highlights vanish again!

[sachdevaprash]

I observe the same issue in latex-mode. After saving, I need to disable and re-enable hl-todo-mode.