XP-mode freezes conflicting with auto-composition-mode

[greenfork]

Reproduce:

1. Copy this to file and save

#lang racket

(define (neighbours cell)
  (for*/set ([dx #(-1 0 1) dy #(-1 0 1)])))

2. Open again in emacs
3. Wait for XP to finish processing and RktX symbol appear
4. Make a cursor move where | is cursor from:

#lang racket

(define (neighbours cell)
  (for*/set ([dx #(-1 0 1) dy #(-1 0 1)])))|

to:

#lang racket

(define (neighbours cell)
  (for*/set ([dx #(-1 0 1) dy #(-1 0 1)]))|)

It completely freezes on my machine.

((alist-get 'racket-mode package-alist))
((emacs-version "27.1")
 (system-type gnu/linux)
 (x-gtk-use-system-tooltips t)
 (major-mode racket-mode)
 (racket--el-source-dir "/home/grfork/.emacs.d/straight/build/racket-mode/")
 (racket--rkt-source-dir "/home/grfork/.emacs.d/straight/build/racket-mode/racket/")
 (racket-program "racket")
 (racket-command-timeout 10)
 (racket-xp-after-change-refresh-delay 1)
 (racket-xp-highlight-unused-regexp "^[^_]")
 (racket-repl-buffer-name-function nil)
 (racket-memory-limit 2048)
 (racket-error-context medium)
 (racket-history-filter-regexp "\\`\\s *\\'")
 (racket-images-inline t)
 (racket-images-keep-last 100)
 (racket-images-system-viewer "display")
 (racket-images-system-viewer "display")
 (racket-use-repl-submit-predicate nil)
 (racket-pretty-print t)
 (racket-indent-curly-as-sequence t)
 (racket-indent-sequence-depth 0)
 (racket-pretty-lambda nil)
 (racket-smart-open-bracket-enable nil)
 (racket-module-forms "\\s(\\(?:module[*+]?\\|library\\)")
 (racket-logger-config
  ((cm-accomplice . warning)
   (GC . info)
   (module-prefetch . warning)
   (optimizer . info)
   (racket/contract . error)
   (sequence-specialization . info)
   (* . fatal)))
 (racket-show-functions
  (racket-show-pseudo-tooltip)))
(enabled-minor-modes
 (amx-mode)
 (auto-composition-mode)
 (auto-compression-mode)
 (auto-encryption-mode)
 (auto-fill-mode)
 (auto-save-mode)
 (column-number-mode)
 (company-mode)
 (counsel-mode)
 (counsel-projectile-mode)
 (delete-selection-mode)
 (diff-hl-mode)
 (display-fill-column-indicator-mode)
 (electric-indent-mode)
 (electric-pair-mode)
 (file-name-shadow-mode)
 (flycheck-mode)
 (font-lock-mode)
 (global-auto-revert-mode)
 (global-company-mode)
 (global-diff-hl-mode)
 (global-eldoc-mode)
 (global-flycheck-mode)
 (global-font-lock-mode)
 (global-so-long-mode)
 (global-subword-mode)
 (global-undo-tree-mode)
 (hs-minor-mode)
 (ivy-mode)
 (ivy-rich-mode)
 (key-chord-mode)
 (line-number-mode)
 (lispy-mode)
 (mouse-wheel-mode)
 (override-global-mode)
 (projectile-mode)
 (racket-xp-mode)
 (rainbow-delimiters-mode)
 (recentf-mode)
 (save-place-mode)
 (savehist-mode)
 (semantic-minor-modes-format)
 (shell-dirtrack-mode)
 (show-paren-mode)
 (straight-package-neutering-mode)
 (straight-use-package-mode)
 (subword-mode)
 (tooltip-mode)
 (transient-mark-mode)
 (undo-tree-mode)
 (which-key-mode)
 (window-divider-mode)
 (winner-mode)
 (ws-butler-mode))
(disabled-minor-modes
 (abbrev-mode)
 (amx-debug-mode)
 (auto-fill-function)
 (auto-revert-mode)
 (auto-revert-tail-mode)
 (auto-save-visited-mode)
 (avy-linum-mode)
 (blink-cursor-mode)
 (buffer-read-only)
 (cl-old-struct-compat-mode)
 (company-search-mode)
 (compilation-minor-mode)
 (compilation-shell-minor-mode)
 (completion-in-region-mode)
 (crux-reopen-as-root-mode)
 (dash-fontify-mode)
 (defining-kbd-macro)
 (diff-auto-refine-mode)
 (diff-hl-dir-mode)
 (diff-minor-mode)
 (dired-hide-details-mode)
 (edebug-mode)
 (eldoc-mode)
 (electric-layout-mode)
 (electric-quote-mode)
 (global-dash-fontify-mode)
 (global-display-fill-column-indicator-mode)
 (global-prettify-symbols-mode)
 (global-semantic-highlight-edits-mode)
 (global-semantic-highlight-func-mode)
 (global-semantic-show-parser-state-mode)
 (global-semantic-show-unmatched-syntax-mode)
 (global-semantic-stickyfunc-mode)
 (global-superword-mode)
 (global-visual-line-mode)
 (horizontal-scroll-bar-mode)
 (ibuffer-auto-mode)
 (ido-everywhere)
 (isearch-mode)
 (ivy-rich-project-root-cache-mode)
 (jit-lock-debug-mode)
 (lispy-goto-mode)
 (lispy-other-mode)
 (menu-bar-mode)
 (next-error-follow-minor-mode)
 (org-cdlatex-mode)
 (org-src-mode)
 (org-table-follow-field-mode)
 (orgtbl-mode)
 (outline-minor-mode)
 (overwrite-mode)
 (paragraph-indent-minor-mode)
 (prettify-symbols-mode)
 (racket-smart-open-bracket-mode)
 (semantic-highlight-edits-mode)
 (semantic-highlight-func-mode)
 (semantic-mode)
 (semantic-show-parser-state-mode)
 (semantic-show-unmatched-syntax-mode)
 (semantic-stickyfunc-mode)
 (sh-electric-here-document-mode)
 (size-indication-mode)
 (so-long-minor-mode)
 (straight-live-modifications-mode)
 (straight-symlink-emulation-mode)
 (superword-mode)
 (tab-bar-history-mode)
 (tab-bar-mode)
 (temp-buffer-resize-mode)
 (tool-bar-mode)
 (undo-tree-visualizer-selection-mode)
 (unify-8859-on-decoding-mode)
 (unify-8859-on-encoding-mode)
 (url-handler-mode)
 (use-hard-newlines)
 (vc-parent-buffer)
 (view-mode)
 (visible-mode)
 (visual-line-mode)
 (ws-butler-global-mode)
 (xref-etags-mode))

[greghendershott]

That doesn't sound good.

Unfortunately I can't seem to reproduce this.

In fact I frequently do the same steps, when writing various for forms: I have a missing body clause, I back up inside the closing paren, and hit ENTER. Like, yesterday I did this at least a half-dozen times.

In my case paredit supplies the closing paren. In your case, electric-pair-mode.

1. Speaking of which: I wonder if some minor-mode is causing a problem.

   I tried enabling various minor-modes that your report shows to be active -- especially things like flycheck, and things that handle parens. No luck.

   Is it possible you could try disabling some, and see if that makes the problem go away? (If I know which one, I can try to figure out why.)

   [Having both flycheck and racket-xp-mode enabled in the same buffer feels a little "heavy" and redundant? Maybe your version of flycheck elicits the problem, even though mine didn't?]

2. Speaking of versions, I see you're using straight. Are you on Racket Mode's latest commit 48084a4 or on something older and if so what?

[greghendershott]

p.s. Another thing you could try is:

• M-x toggle-debug-on-quit to enable that.
• Do your steps.
• See if you can press C-g until you get Emacs to quit and show you a buffer -- then please copy/paste that here.

Note: C-g might work right away, but you might need to press it multiple times over a few seconds until it has an effect.

[greenfork]

Thanks for suggestions, I really had a lot of redundant modes there, e.g. lispy already does indenting and pairs; as well as flycheck which you mentioned.

I updated all packages before submitting the issue, I have the latest commit you mentioned.

I found the problem, disabling auto-composition-mode solves this issue.

I have this in my config:

      ;; On MacOS due to retina display font should be bigger.
      (if (eq system-type 'darwin)
      (set-face-attribute 'default nil :font "Fantasque Sans Mono" :height 120)
        (set-face-attribute 'default nil :font "Fantasque Sans Mono" :height 110))

      (let ((alist `((?& . ,(regexp-opt '("&&")))
                     (?* . ,(regexp-opt '("*/")))
                     (?| . ,(regexp-opt '("|||>" "||>" "||" "|>")))
                     (?: . ,(regexp-opt '("::")))
                     (?= . ,(regexp-opt '("===" "==>" "==" "=>>" "=>" "=<<" "=/=")))
                     (?! . ,(regexp-opt '("!==" "!=")))
                     (?> . ,(regexp-opt '(">=>" ">=" ">>=" ">>-" ">>" ">->" ">-")))
                     (?- . ,(regexp-opt '("->>" "->" "-->" "-<<" "-<")))
                     (?< . ,(regexp-opt '("<|||" "<||" "<|>" "<|" "<==" "<=>" "<=<" "<=" "<!--" "<>" "<->"
                                          "<--" "<-<" "<-" "<<=" "<<-" "<<" "<~>" "<~" "<~~")))
                     (?/ . ,(regexp-opt '("/**/" "/*" "//")))
                     (?~ . ,(regexp-opt '("~>" "~~>" "~~"))))))
        (dolist (char-regexp alist)
      (set-char-table-range composition-function-table (car char-regexp)
                    `([,(cdr char-regexp) 0 font-shape-gstring]))))

As a workaround to use ligatures in emacs. Ligatures are not supported out of the box, this seemed as one of the cleanest workarounds I could find.

Exactly this line is causing the problem:

(?* . ,(regexp-opt '("*/")))

presumably because of the for*/set function. I don't think you need a specific font to reproduce this problem, you could try this setting, and auto-composition-mode should be enabled by default:

      (let ((alist `((?* . ,(regexp-opt '("*/"))))))
        (dolist (char-regexp alist)
      (set-char-table-range composition-function-table (car char-regexp)
                    `([,(cdr char-regexp) 0 font-shape-gstring]))))

[greghendershott]

Thanks for the detective work!

I looked at composite.el, the source for auto-composition-mode. Mostly it seems to be "housekeeping" for the data structures. The core work seems done by compose-region-internal, which is defined in C source.

Before I go down that rabbit hole, I have a guess. Could you try one more experiment, please?

What if you change the value of racket-show-functions from '(racket-show-pseudo-tooltip) to something boring like '(racket-show-echo-area). Does it still freeze?

[greenfork]

That's a great guess! It stops freezing after I change it to '(racket-show-echo-area).
Also tried all other options, in summary:

• racket-show-header-line - freezes as with racket-show-pseudo-tooltip
• racket-show-pos-tip - around 300 ms freeze (doesn't seem to depend on auto-composition-mode)
• racket-show-echo-area - no freezing at all

[greenfork]

And I want to mention that it is not a problem for me right now. I like more racket-show-echo-area as it appears less "noisy" for me because half of the time my code is in inconsistent state and popping warning (with 2 lines or more) right at the cursor obscure the other code below that.

[greghendershott]

Thanks for trying that experiment and I'm glad you have a configuration that works well for you.

I'll leave this open for now (maybe re-title it). I'm curious to understand the conflict between auto-composition-mode and racket-show-pseudo-tooltip. It's interesting that you also see a freeze using racket-show-header-line --- I didn't expect that. So although it might not be for awhile I will eventually want to revisit this.

Thanks again for your help!

[greenfork]

A small update on the issue:
I switched to ligature support using ligature.el library which is supposed to be the new true way of doing ligatures. The problem remains same.
There's a mu4e issue which may have some additional information: djcb/mu#1281

[greghendershott]

Thanks for the follow up. That comment linked to some mu4e code doing (overlay-put overlay 'display targetstr). That is, setting a value for the display property of an overlay. The Racket Mode pseudo tooltips do this, too.

The Display Property is a pretty powerful low-level Emacs display engine feature.

It seems this is a problem with that feature being able to handle ligature character composition.

It seems unlikely I could fix this in Racket Mode.

I'm not even sure how to attempt to work around, or avoid it. I think I'd need to detect such a char sequence (but not sure how, reliably) and display something else (but not sure what, and how to know what).

Having said that, I hate saying "well that's not my problem" and I'm curious to learn more. So I'll try to explore more as/when I have time.

Meanwhile, do you think I should:

• update the README with a warning about this?
• have racket-xp-mode check for the auto-composition-mode or ligature-mode minor modes being enabled, and
  • issue a warning?
  • offer to disable those modes in the racket-xp-mode buffer)?

[greenfork]

There are 5 ways to enable ligatures in Emacs, no particular one seems completely bugless. I think it would be hard to maintain detection of whether "ligature mode" is enabled for all of them. I would just add a warning to "Troubleshooting" or similar section as you mentioned.

[greghendershott]

Here's the most-minimal recipe I've been able to find so far:

(progn
  (set-char-table-range composition-function-table
                        ?*
                        `([,(regexp-opt '("*/")) 0 font-shape-gstring]))
  (let ((ov (make-overlay (point) (1+ (point)))))
    (overlay-put ov 'after-string "for*/list")))

This reliably freezes e.g. Emacs 25.2.2. [Although I test a variety of Emacs versions in CI, for day-to-day normal work I'm boring and just use the Emacs 25.2.2 default in Ubuntu 18.0.4.]

From what I've read this will freeze even Emacs 27.2 -- but it might work fine in the current Emacs snapshot 28.0.50, which will become Emacs 28.1.

---

Meanwhile I agree the best I think I can do is warn about this in the configuration documentation.

(In theory I could scan composition-function-table to look for "suspicious" entries. But I don't know how to define "suspicious". I'm willing to go to some lengths if it makes "just works" experiences, but I don't see any reasonable way here.)

[greghendershott]

I built Emacs from source (commit ee3a4e3d1be656cd0df71ed197dc5f102556f0e0 from a few hours ago), ran it with emacs -Q, and confirmed that executing the minimal recipe does NOT cause a freeze.

So some good news is that it looks like this will be fixed for Emacs 28.1.

[greghendershott]

Docs updated for the info file supplied with the package, as well as the HTML version: https://www.racket-mode.com/#Ligatures

I guess I'll leave this issue open, for now, and in fact "pin" it here in GitHub issues in the hope that helps other people find it.

[greghendershott]

I realized I should close this issue. I did document the situation, and, the only possible fix is really for folks to use Emacs 28.

But I'll still leave it "pinned" for visibility.