Printing a tibble removes all text faces

[novusshu]

It seems that anytime I print a tibble, the text highlights, coloring are shut off for the rest of the session.

I'm using a cyberpunk theme in the screenshots, but even if I disable the theme, the text faces will still change after the tibble printing. This change has nothing to do with the theme.

I'm using:
MacOS 12.3
Emacs 28.0
ESS:20220225.1523

Any help would be appreciated!!

[swhalemwo]

I'm having the same issue. I had just before updated to emacs 28.1 (on Manjaro 21.2.6), so perhaps changes there are responsible for this bug.

Edit: After downgrading to 27.2 tibbles are printed correctly again.

[jsahrmann]

I'm having the same problem on Windows 10, Emacs 28.1.

It also seems to happen after printing error messages written by rlang:

what-cursor-position with prefix argument (C-u C-x =) indicates that there's an overlay on top of the shadowed text. Here's the output for the 'p' in the first print line:

             position: 645 of 870 (74%), column: 2
            character: p (displayed as p) (codepoint 112, #o160, #x70)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x70
               script: latin
               syntax: w 	which means: word
             category: .:Base, L:Strong L2R, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 70" or "C-x 8 RET LATIN SMALL LETTER P"
          buffer code: #x70
            file code: #x70 (encoded by coding system utf-8-dos)
              display: by this font (glyph code):
    harfbuzz:-outline-Fira Code-bold-normal-normal-mono-17-*-*-*-c-*-iso8859-1 (#xE1)

Character code properties: customize what to show
  name: LATIN SMALL LETTER P
  general-category: Ll (Letter, Lowercase)
  decomposition: (112) ('p')

There are text properties here:
  font-lock-face       comint-highlight-input
  fontified            t
  front-sticky         t

And here's output from the second print line:

             position: 846 of 870 (97%), column: 2
            character: p (displayed as p) (codepoint 112, #o160, #x70)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x70
               script: latin
               syntax: w 	which means: word
             category: .:Base, L:Strong L2R, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 70" or "C-x 8 RET LATIN SMALL LETTER P"
          buffer code: #x70
            file code: #x70 (encoded by coding system utf-8-dos)
              display: by this font (glyph code):
    harfbuzz:-outline-Fira Code-bold-normal-normal-mono-17-*-*-*-c-*-iso8859-1 (#xE1)

Character code properties: customize what to show
  name: LATIN SMALL LETTER P
  general-category: Ll (Letter, Lowercase)
  decomposition: (112) ('p')

There are 3 overlays here:
 From 843 to 859
  evaporate            t
  face                 (:foreground "gray30")
  insert-behind-hooks  (ansi-color-freeze-overlay)
  modification-hooks   (ansi-color-freeze-overlay)
 From 843 to 869
  evaporate            t
  face                 (:foreground "gray30")
  insert-behind-hooks  (ansi-color-freeze-overlay)
  modification-hooks   (ansi-color-freeze-overlay)
 From 843 to 871
  evaporate            t
  face                 (:foreground "gray30")
  insert-behind-hooks  (ansi-color-freeze-overlay)
  modification-hooks   (ansi-color-freeze-overlay)


There are text properties here:
  font-lock-face       comint-highlight-input
  fontified            t
  front-sticky         t

[wardfont]

I'm having the same issue for tibble and rlang output. the "ansi-color-freeze-overlay" is also present for me.

Emacs version 28.1
Manjaro 21.2.6
ESS 20220225.1523

[maxecharel]

what-cursor-position with prefix argument (C-u C-x =) indicates that there's an overlay on top of the shadowed text.

Same for me; and thx @jsahrma for the diagnostic procedure :)

Emacs 28.1
ESS 20220225.1523

[maxecharel]

Note that a workaround is to change the tibble print options:

options(pillar.subtle = FALSE)

The pillar package is the one which handles printing as of tibble 3.1.0. This hack is clearly not the ultimate solution to the problem, but at least it makes the console output readable.

[xqz-u]

@maxecharel Thank you for the solution you found! I have it working for the case of printing a tibble.
However, as soon as a warning is displayed, the whole text face changes to black again. Referring to
@jsahrma's example:

Any idea how to hack away this issue too?

emacs-version "28.1"
ess-version "18.10.3snapshot"
ess-20220225.1523

[wardfont]

Setting options(rlang_backtrace_on_error = "none") solves the issue caused by rlang reminders.

I set both hacks in my .Rprofile file on the home directory so they take effect by default.

@maxecharel Thanks for the pointer!

[swhalemwo]

thanks for the solutions so far! I've come across this behavior in another occasion, namely warnings that don't throw an error:

does someone know which option to change to fix this as well?

Edit: I posted this point because the issue (removal of text faces) was the same as in the original post, and since this issue had been discussed in relation to other aspects than printing tibbles (errors) I thought text face removal in the case of warnings was relevant here as well. Should I make a separate issue for it?

[swhalemwo]

@lionel- sorry if it wasn't clear, by fixing I didn't mean avoiding the warning itself, but that the warning removes faces for all subsequent text.

[lionel-]

By the way as a stopgap you can clone this branch of xterm-color locally: https://github.com/lionel-/xterm-color/tree/patches

Here is my config:

(use-package xterm-color
  :load-path "PATH-TO-xterm-color"

  :init
  (setq comint-output-filter-functions
        (remove 'ansi-color-process-output comint-output-filter-functions))

  (add-hook 'inferior-ess-mode-hook
            (lambda () (add-hook 'comint-preoutput-filter-functions #'xterm-color-filter nil t)))

  :config
  (setq xterm-color-use-bold t))

[maxecharel]

@lionel- thx for the workaround, will certainly try it. Does it mean that the issue comes from ansi-color.el and thus cannot be solved by patching ESS?

[lionel-]

I haven't looked into it (I don't have any time to work on ESS lately, I'm hoping next year will be less busy) but this would be my guess.

[maxecharel]

By the way as a stopgap you can clone this branch of xterm-color locally: https://github.com/lionel-/xterm-color/tree/patches
Here is my config:

Works like a charm (at least when printing tibble), thanks again @lionel- .

Out of curiosity, does xterm-color improve other aspects of text rendering in iESS comint buffer?

[lionel-]

IIRC it supports more ANSI escapes.

[Fuco1]

Simplest fix seems to be to run (setq-local ansi-color-for-comint-mode 'filter) in the comint buffer. You can add this to 'inferior-ess-mode-hook, my config is

    (defun my-inferior-ess-init ()
      (setq-local ansi-color-for-comint-mode 'filter)
      (smartparens-mode 1))
    (add-hook 'inferior-ess-mode-hook 'my-inferior-ess-init)

[mmaechler]

Thank you, @Fuco1. From what I read here (and in #1199) it seems we (i.e. ESS) should address this itself already, and that becomes more urgent because of Emacs 28 behavior (and the use of ANSI escape printing by some R packages).
OTOH, I dont' feel familiar with the current elisp in ess-inf.el anymore. Notably I cannot even easily find where inferior-ess-mode-hook is run, as it is not found by grep in the *.el sources .. (??)

[Fuco1]

This hook is generated automatically by define-derived-mode

after macro-expand

It is called at the end of the of the expanded form

[lionel-]

@Fuco1 This strips all ANSI escapes instead of decorating the text right? I'm not sure this should be the default in ESS, this seems like this should be set in user configs.

So I would not call this a fix for Emacs 28, but rather a temporary stopgap until a proper fix is found.

[Fuco1]

Yea, it strips the ansi escapes, but it seems the inferior ess mode does its own font-locking. But I have no idea what a good solution would be.

Without this, the buffer is quite unusable

But yes, previously these were colored, now they are not

[lionel-]

I have no idea what a good solution would be.

Probably the historical behaviour is the right solution. It used to be that the font-locking isn't applied on ansi-colourised output. This is still the case in my local config.

[DarwinAwardWinner]

This isn't a complete solution, but you can tell R to print an ANSI reset code after every command, before the next prompt:

invisible(addTaskCallback(function(...) {
    if (interactive()) {
        # Remember to install crayon
        try(cat(crayon::reset("")), silent = TRUE)
    }
    TRUE
}, name = "ansi_reset"))

This doesn't fix the coloring of the output of the offending commands, but it does prevent it from leaking into subsequent commands, at least in my Emacs.

[Atreyagaurav]

This isn't a complete solution, but you can tell R to print an ANSI reset code after every command, before the next prompt:

invisible(addTaskCallback(function(...) {
    if (interactive()) {
        # Remember to install crayon
        try(cat(crayon::reset("")), silent = TRUE)
    }
    TRUE
}, name = "ansi_reset"))

This doesn't fix the coloring of the output of the offending commands, but it does prevent it from leaking into subsequent commands, at least in my Emacs.

Shouldn't this be the default setting in R itself? Like I can't think of any reason anscii reset isn't done in a prompt causing previous output to leak into the next one.

Also, where do you put that code? Sorry I'm new to R, is there a config file like file that you can place that'll be loaded for all R sessions?

[DarwinAwardWinner]

You put it in ~/.Rprofile (or whatever the Windows equivalent is). I believe R on the console does do this. It's just the not-quite-a-terminal environment of the ESS buffer that has this issue.

[cddesja]

This isn't a complete solution, but you can tell R to print an ANSI reset code after every command, before the next prompt:

invisible(addTaskCallback(function(...) {
    if (interactive()) {
        # Remember to install crayon
        try(cat(crayon::reset("")), silent = TRUE)
    }
    TRUE
}, name = "ansi_reset"))

This doesn't fix the coloring of the output of the offending commands, but it does prevent it from leaking into subsequent commands, at least in my Emacs.

FYI, if you use org-mode code blocks and LaTeX output this prevents tables from being written to a .tex file and subsequently rendered in a PDF.

[DarwinAwardWinner]

@cddesja It should be possible to add an extra check for this, e.g. only printing the reset code if standard output is a terminal or something like that.

[jackkamm]

For org-mode users: this issue also affects output blocks from interactive ob-R sessions.

Unfortunately, @Fuco1 's solution to set ansi-color-for-comint-mode to filter doesn't fix the org-mode output.

Luckily, @lionel- 's solution to use xterm-color does work for org-mode.

It's not necessary to use @lionel- 's patched fork; I found the upstream xterm-color on MELPA works as well (though it may be missing other features from @lionel- 's fork, namely bold text).

Here's my config for this:

(defun my-inferior-ess-init ()
  "Workaround for https://github.com/emacs-ess/ESS/issues/1193"
  (add-hook 'comint-preoutput-filter-functions #'xterm-color-filter -90 t)
  (setq-local ansi-color-for-comint-mode nil))

(add-hook 'inferior-ess-mode-hook #'my-inferior-ess-init)

[PavoDive]

I have nothing to add to the solution (the one provided by @jackkamm worked very well for me), but I wanted to express my gratitude to the team maintaining ESS. I want to commend not only a great software, but also an amazingly helpful community.

[solna86]

This bug was also affecting me but I can no longer reproduce it when upgrading to Emacs 29.1 from Emacs 28.2.

[walmes]

I've observed the same problems with tibbles, tidymodels outputs, and messages and warnings that your already mentioned. So, as suggested by @lionel-, I added xterm-color in my Doom Emacs configs and it seems solved all problems related to illegible font faces.

;; package.el
(package! xterm-color)

;; config.el
(use-package! xterm-color
  :init
  (setq comint-output-filter-functions
        (remove 'ansi-color-process-output comint-output-filter-functions))
  (add-hook 'inferior-ess-mode-hook
            (lambda () (add-hook 'comint-preoutput-filter-functions #'xterm-color-filter nil t)))
  :config
  (setq xterm-color-use-bold t))

Now all outputs are in appropriate font face.