O_NERD and O_EMOJI creates duplicate letters in filename

[FelixKratz]

Environment details (Put x in the checkbox along with the information)

• Operating System: macOS Ventura
• Desktop Environment: Aqua
• Terminal Emulator: kitty, Alacritty
• Shell: zsh
• Custom desktop opener (if applicable):
• Program options used: none
• Configuration options set: O_NERD
• Plugins are installed
• Issue exists on nnn master

Exact steps to reproduce the issue

Clone the repo, enable nerdfonts in the makefile

O_NERD := 1  # support icons-nerdfont

and compile with LDLIBS="-L/opt/homebrew/opt/ncurses/lib/" make.

Some files (with certain icons it appears) have duplicate initial letters as seen in the screenshot below:

as soon as I select the makefile it will also have the duplicate initial letter:

I have done some testing, this seems to also happen with O_EMOJI is set to enabled:

but not with O_ICONS enabled:

I have peeked into the source code to find what is different between those options, but did not find a convincing clue.

I have also checked with kitty and alacritty, they both face the same problem.

These are the dynamic libs used by nnn:

 x otool -L ./nnn
./nnn:
	/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
	/opt/homebrew/opt/ncurses/lib/libncursesw.6.dylib (compatibility version 6.0.0, current version 6.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)

[N-R-K]

Previous discussion: #1662

According to #1662 (reply in thread) v4.7 didn't have this problem (can you confirm this too? @FelixKratz) but I went through the diff between 4.7 and 4.8 and didn't notice anything that might cause it.

Also another person who reported the issue was on macos as well (#1662 (reply in thread)) so maybe something in macos is causing it?

[FelixKratz]

Sorry, I did not search the discussion, only the issues.

I have done a quick bisect and this is the commit where it goes wrong: 20e944f

This commit changed the codepoints, how would such a change lead to the duplication of filname letters?

[FelixKratz]

Interestingly, O_EMOJI shows this problem since the introduction in b45be9b.

Edit: The problem appears when calling addstr(icon.icon);.

All instances where this is a problem wcswidth reports a length of 2 as seen in the screenshot below, where I display the wcswidth of the icon to the left of the actual item:

It seems that this messes up some internal counting logic.

[jarun]

I use emojis on xfce4-terminal and don't see this issue.
Can you try to fix this and raise a PR? I will try on my setup and see if it works for me as well?
The change should be in print_icon() and/or adjust_cols(). See if changing the padding works.

[FelixKratz]

I believe this problem has no trivial fix.

It seems that removing

	if (sel)
		attrs |= A_REVERSE;

in printent fixes the duplication problem. This suggests to me that there is a problem with the ncurses internal redraw and damage logic. This is possibly connected to a disagreement in the terminals perceived width for the character and the ncurses calculated width of the character. But to be honest, I have no clue about terminal rendering.

The problematic new nerdfont chars seem to be 4byte wide, while they where 3bytes before the update.
The same is true for the emojis.

[jarun]

@luukvbaal would you able to check this?

[jarun]

@FelixKratz what happens if you replace A_REVERSE with A_STANDOUT?

[FelixKratz]

@FelixKratz what happens if you replace A_REVERSE with A_STANDOUT?

As soon as any attrs are set that would cause the filename to be redrawn it starts to happen

No attrs:

A_STANDOUT:

A_BOLD:

[jarun]

@N-R-K @luukvbaal what's the best course of action here?

We can't make the next release without fixing this.

I see something here that I don't fully understand: #1661

[N-R-K]

All instances where this is a problem wcswidth reports a length of 2 as seen in the screenshot below, where I display the wcswidth of the icon to the left of the actual item:

Interesting. This does seem to be what's causing the issue, because on my system (linux/glibc), all those icons have a width of 1 when called in wcswidth.

Could this be a libc bug in macos? Can you share the output of locale? Also what do you see when you run LC_ALL="C.UTF8" nnn?

[FelixKratz]

Could this be a libc bug in macos? Can you share the output of locale? Also what do you see when you run LC_ALL="C.UTF8" nnn?

My locale output is:

Running LC_ALL="C.UTF8" ./nnn looks completely wild:

I will happily try out any ideas, I am a bit lost regarding this problem as I have no experience with ncurses.

[lawnowner]

I don't think your problem is due to a bug in nnn. Unless you can reproduce this on Linux, maybe your OS is using a different text normalization for unicode encoding (e.g. NFD instead of NFC), so the width of some characters are 2 instead of 1 for you due to decomposition.

The locale category values in double quotes are not set explicitly, they are implied values derived from LANG or LC_ALL, but in your output all values are in double quotes, so none of them are explicitly set in your environment. You can try LC_CTYPE="en_US.UTF-8" ./nnn or LC_ALL="en_US.UTF-8" ./nnn on the command line, or setting the LC_ variables in your shell startup file and/or terminal emulator config. If you got a weird utf-8-mac variant of the locale, you can try that as well. Also see setlocale has no effect on macOS.

[cyaconi]

I just tried both LC_CTYPE="en_US.UTF-8" ./nnn and LC_ALL="en_US.UTF-8" ./nnn on macOS and the problem continues. Hope it helps.

[FelixKratz]

I seem to have found a fix for this: #1694

[quantonganh]

I confirmed that wcswidth returns the different value for the same Unicode code point between macOS and Linux.

macOS:

LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=

---

Linux:

LANG=en_GB.UTF-8
LANGUAGE=
LC_CTYPE="en_GB.UTF-8"
LC_NUMERIC="en_GB.UTF-8"
LC_TIME="en_GB.UTF-8"
LC_COLLATE="en_GB.UTF-8"
LC_MONETARY="en_GB.UTF-8"
LC_MESSAGES="en_GB.UTF-8"
LC_PAPER="en_GB.UTF-8"
LC_NAME="en_GB.UTF-8"
LC_ADDRESS="en_GB.UTF-8"
LC_TELEPHONE="en_GB.UTF-8"
LC_MEASUREMENT="en_GB.UTF-8"
LC_IDENTIFICATION="en_GB.UTF-8"
LC_ALL=

---

In version 4.8, the code point is different, width is 1 but the icon is missing:

---

I don't think your problem is due to a bug in nnn. Unless you can reproduce this on Linux, maybe your OS is using a different text normalization for unicode encoding (e.g. NFD instead of NFC), so the width of some characters are 2 instead of 1 for you due to decomposition.

It could be the result of a combination of nnn and the Nerd fonts, locale settings, unicode version and the C library implementation.