Coexistence with vertical-border display table slot?

[rnkn]

Dear Steve,

In console Emacs, given that in many fonts the | vertical line character does not create a solid vertical line between line breaks, I’ve added the below code to my init to make the window vertical border a solid line.

(defun prettify-vertical-border (&optional dummy)
  (ignore dummy)
  (set-display-table-slot (or buffer-display-table
                              standard-display-table)
                          'vertical-border ?│))

(when (not window-system)
  (add-hook 'window-configuration-change-hook
            'prettify-vertical-border))

But I’ve found that this seems to conflict with global-page-break-lines-mode, and so when spawning a new window that creates a vertical split (with global-page-break-lines-mode enabled), I get the ugly old broken vertical border, and not my prettified vertical border. Specifically this happens when spawning a help-mode window on the left (i.e. from the right window).

I’m not entirely sure if root of this troubled coexistence lies with page-break-lines, given that I can only manifest the appearance of the old, broken vertical border when spawning a help-mode window (and with global-page-break-lines-mode) with C-h o SYMBOL RET.

Please help! I don’t think I can live without the aesthetic soothingness of a solid vertical border and solid page-break-lines!

Many thanks.

• GNU Emacs 27.0.60 (build 1, x86_64-apple-darwin19.2.0, NS appkit-1894.20 Version 10.15.2 (Build 19C57)) of 2020-01-18
• page-break-lines 20190519.2238

[purcell]

This is a fiddly case. Your code probably ends up modifying standard-display-table. page-break-lines modifies buffer-display-table, creating a fresh one when necessary using make-display-table. The problem is, the fresh one does not contain any entries you'be added to standard-display table.

My guess is that you can (add-hook 'page-break-lines-mode-hook 'prettify-vertical-border) to be sure that your custom entry also gets added to the new buffer-display-table.

P.S. Perhaps prefer unless to when + not.

[purcell]

Well, in fact, the problem might be that sometimes your function runs in a context where it affects buffer-display-table, and sometimes it affects standard-display-table. Probably what you want is to explicitly modify standard-display-table once at start-up, and then only modify buffer-display-table from window-configuration-change-hook.

[purcell]

ie.

(defun prettify-vertical-border (&optional dummy)
  (ignore dummy)
  (when buffer-display-table
    (set-display-table-slot buffer-display-table 'vertical-border ?│)))

(unless window-system
  (set-display-table-slot standard-display-table 'vertical-border ?│)
  (add-hook 'window-configuration-change-hook
            'prettify-vertical-border))

[purcell]

Even more than that, you should never have top-level code which is conditional upon window-system, because you might run the startup code in an emacs daemon and later start a GUI frame, or start a GUI emacs and then later start a TTY frame. So in this case you should check window-system when the hook function runs, or just see if it's safe to always update the display table unconditionally: presumably GUI frames don't use the display table when rendering the vertical border anyway.

[rnkn]

I really appreciate the quick and thorough response — thanks!

Unfortunately, that didn’t do the trick... What I have now:

(defun prettify-vertical-border (&optional dummy)
  (ignore dummy)
  (when buffer-display-table
    (set-display-table-slot buffer-display-table 'vertical-border ?│)))

(when (not window-system)
  (set-display-table-slot standard-display-table 'vertical-border ?│)
  (add-hook 'window-configuration-change-hook 'prettify-vertical-border)
  (when (featurep 'page-break-lines)
    (add-hook 'page-break-lines-mode-hook 'prettify-vertical-border)))

(I like the semantics of “when there’s not this, do this” vs “unless there’s this, do this”; it feels more like natural language, wheras I'd only say “unless” after the condition, “do this unless this.”)

What seems especially odd is that this is only happening with windows spawned with modes derived from special-mode; if I open a dired buffer in the right window then dired-find-file-other-window on a file, I get a nice solid line, which means global-page-break-lines-mode and my hook are getting along fine, and make one suspect special-mode as the culprit, but then only with disabling global-page-break-lines-mode enabled. I’ve looked at help-mode and special-mode and they don’t seem to be doing anything with the display table.

I’ve asked the emacs-devel list if we can have vertical-border be a defcustum a la this new display-fill-column-indicator, but you probably know how hard it is for an “outsider” to have any effect there...

Also, thanks for the hint re window-system, I did not know it was possible to create a GUI frame from an Emacs daemon. That is cool!

[purcell]

I imagine this is an issue with hook orders. What are the contents of window-configuration-change-hook and page-break-lines-mode-hook? Examples of the latter in both affected and unaffected buffers would be helpful.

[rnkn]

Ah sorry I should have mentioned that I did try reordering the hooks and did get the same results; with the code above, evaluating window-configuration-change-hook and then evaluating page-break-lines-mode-hook in an affected buffer/window gives:

window-configuration-change-hook ->
  (prettify-vertical-border page-break-lines--update-display-tables window--adjust-process-windows)
page-break-lines-mode-hook ->
  (prettify-vertical-border page-break-lines-mode-set-explicitly)

Then once adding the APPEND flag to each call to add-hook (which is now DEPTH in Emacs 27.x):

window-configuration-change-hook ->
  (page-break-lines--update-display-tables window--adjust-process-windows prettify-vertical-border)
page-break-lines-mode-hook ->
  (page-break-lines-mode-set-explicitly prettify-vertical-border)

So even when the prettify function is well and truly last, something is getting in the way.

Another tidbit of possibly useful info: after interacting with the minibuffer (e.g. M-: C-g of course window-configuration-change-hook is called, and the vertical border is successfully solid, so it would seem that window-configuration-change-hook is doing its job eventually.