statusline has bug with unicode emoji

[ttys3]

This bug confused me for a rather long time, until today I finally got the time to dig into it.

• nvim --version:

NVIM v0.5.0-dev+1178-gf0ace6d41
Build type: RelWithDebInfo
LuaJIT 2.1.0-beta3
Compilation: /usr/bin/cc -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -O2 -g -Og -g -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wmissing-prototypes -Wimplicit-fallthrough -Wvla
 -fstack-protector-strong -fno-common -fdiagnostics-color=always -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -DMIN_LOG_LEVEL=3 -I/home/ttys3/.config/nvim/neovim/build/co
nfig -I/home/ttys3/.config/nvim/neovim/src -I/home/ttys3/.config/nvim/neovim/.deps/usr/include -I/usr/include -I/home/ttys3/.config/nvim/neovim/build/src/nvim/auto -I/home/ttys3/.config/nvim/neovim/build/include
Compiled by ttys3@ttys3

Features: +acl +iconv +tui
See ":help feature-compile"

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/usr/local/share/nvim"

Run :checkhealth for more info

• Operating system/version: Ubuntu 20.10 x86_64
• Terminal name/version:

both GNOME Terminal 3.38.0 using VTE 0.62.0 +BIDI +GNUTLS +ICU +SYSTEMD
and alacritty 0;7.2

• $TERM: xterm-256color

Steps to reproduce using nvim -u NORC

minimal rc: statusline_issue.vim

set nu
set updatetime=100

lua << EOF
math.randomseed(os.time())
EOF

function! AleStatus() abort
    let ok = 'ok'
        let prefix =  '🇦 '
    let num = luaeval('math.random(1,2)')
    if num == 1 
        return prefix
    else    
        return prefix.  ok
    endif
endfunction

set statusline=%=%{AleStatus()}
set statusline+=[0x%B]

" 0123456789 

nvim -u ./statusline_issue.vim ./statusline_issue.vim

# move chars from `0` to `9`, and you'll find the wrong code shows up.

Actual behaviour

char hex randomly show chars 0xXX and 0xXXX (yes, three XXX chars)

we all know that for 4, its ASCII code is 0x34, but neovim shows as 0x343, and the ] been eaten by neovim

Expected behavior

char hex works ok, should show chars (0xXX)

Misc

we use 0 to 9 because their ASCII code is very easy to remember.

so that we can found the problem very easily

use 0x%B is just for example.

the actual screenshot in my case will like this:

and this:

the char 🇦 codepoints is U+1F1E6 https://emojipedia.org/emoji/%F0%9F%87%A6/

if we replace it with another emoji, say Melon 🍈 https://emojipedia.org/melon/, everything works fine. see below gif

[ttys3]

[jdhao]

Can reproduce this issue on macOS with Alacritty.

kitty on macOS does not have this issue. Maybe it is a terminal issue?

[ttys3]

@jdhao
until today, I tested it under kitty on Linux, the same: does not have this issue.

I agree, maybe it is a terminal issue. I'm not sure.

the issue can be reproduced on:
Gnome Terminal
Alacritty
Terminator

[zeertzjq]

May be another character width issue.

[zeertzjq]

Likely fixed by #19686