Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

invalid charset name #366

Closed
M4n5ter opened this issue Sep 27, 2022 · 30 comments
Closed

invalid charset name #366

M4n5ter opened this issue Sep 27, 2022 · 30 comments

Comments

@M4n5ter
Copy link

M4n5ter commented Sep 27, 2022

FAIL:

~
❯ procs
invalid charset name

SUCCESS:

~echo $LANG                                                                                   
zh_CN.UTF-8
                                                                                                               
~
❯ LANG=en_US.UTF-8 procs                                    
@dalance
Copy link
Owner

dalance commented Sep 28, 2022

The "invalid charset name" is not shown by procs directly.
I think it may be shown by less which is called by procs.

In your environment, does less work fine with LANG=zh_CN.UTF-8?

@M4n5ter
Copy link
Author

M4n5ter commented Sep 28, 2022

The "invalid charset name" is not shown by procs directly. I think it may be shown by less which is called by procs.

In your environment, does less work fine with LANG=zh_CN.UTF-8?

less works well.

~echo $LANG                     
zh_CN.UTF-8
                                                                                                               
~   
❯ less cv_debug.log
                                                                                                               
~

@dalance
Copy link
Owner

dalance commented Sep 28, 2022

How about procs --pager disable?

@M4n5ter
Copy link
Author

M4n5ter commented Sep 28, 2022

How about procs --pager disable?

procs --pager disable can work .

@dalance
Copy link
Owner

dalance commented Sep 28, 2022

How about LESSCHARSET=utf-8 LANG=zh_CN.UTF-8 less -SR cv_debug.log ?

@M4n5ter
Copy link
Author

M4n5ter commented Sep 28, 2022

How about LESSCHARSET=utf-8 LANG=zh_CN.UTF-8 less -SR cv_debug.log ?

LESSCHARSET=utf-8 LANG=zh_CN.UTF-8 less -SR cv_debug.log can also work,

~   
❯ LESSCHARSET=utf-8 LANG=zh_CN.UTF-8 less -SR cv_debug.log
                                                                                                               
~

@dalance
Copy link
Owner

dalance commented Sep 28, 2022

Could you provide the information about your environment?

@M4n5ter
Copy link
Author

M4n5ter commented Sep 28, 2022

Could you provide the information about your environment?

~   
❯ neofetch
             ............                i78@i78-PC 
         .';;;;;.       .,;,.            ---------- 
      .,;;;;;;;.       ';;;;;;;.         OS: Deepin 20.7 x86_64 
    .;::::::::'     .,::;;,''''',.       Host: 82MS Yoga 14sACH 2021 
   ,'.::::::::    .;;'.          ';      Kernel: 5.18.4-amd64-desktop-hwe 
  ;'  'cccccc,   ,' :: '..        .:     Uptime: 18 mins 
 ,,    :ccccc.  ;: .c, '' :.       ,;    Packages: 2455 (dpkg) 
.l.     cllll' ., .lc  :; .l'       l.   Shell: zsh 5.7.1 
.c       :lllc  ;cl:  .l' .ll.      :'   Resolution: 2880x1800 
.l        'looc. .   ,o:  'oo'      c,   DE: Deepin 20.7 
.o.         .:ool::coc'  .ooo'      o.   WM: KWin 
 ::            .....   .;dddo      ;c    Theme: deepin-dark [GTK2/3] 
  l:...            .';lddddo.     ,o     Icons: bloom-dark [GTK2/3] 
   lxxxxxdoolllodxxxxxxxxxc      :l      Terminal: deepin-terminal 
    ,dxxxxxxxxxxxxxxxxxxl.     'o,       CPU: AMD Ryzen 7 5800H with Radeon Graphics (16) @ 3.200GHz 
      ,dkkkkkkkkkkkkko;.    .;o;         GPU: AMD ATI 03:00.0 Cezanne 
        .;okkkkkdl;.    .,cl:.           Memory: 5657MiB / 13837MiB 
            .,:cccccccc:,.
                                                                 


                                                                                                               
~   
❯ env
CLUTTER_IM_MODULE=fcitx
COLORTERM=truecolor
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
DEEPIN_WINE_SCALE=2.00
DESKTOP_SESSION=deepin
DISPLAY=:0
D_DISABLE_RT_SCREEN_SCALE=1
D_DXCB_FORCE_OVERRIDE_HIDPI=1
GDMSESSION=deepin
GIO_LAUNCHED_DESKTOP_FILE=/usr/share/applications/deepin-terminal.desktop
GIO_LAUNCHED_DESKTOP_FILE_PID=14979
GNOME_DESKTOP_SESSION_ID=this-is-deprecated
GPG_AGENT_INFO=/run/user/1000/gnupg/S.gpg-agent:0:1
GTK_IM_MODULE=fcitx
HOME=/home/i78
LANG=zh_CN.UTF-8
LANGUAGE=zh_CN
LOGNAME=i78
PATH=/home/i78/GOPATH/bin:/home/i78/sdk/go/bin:/home/i78/IDEs/GoLand-2022.2/bin:/home/i78/Fleet/lib/app/bin:/home/i78/software/qemu/qemu-7.0.0/build:/home/i78/software/riscv64-unknown-elf-gcc-8.3.0-2020.04.1-x86_64-linux-ubuntu14/bin:/home/i78/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin
PWD=/home/i78
QT_ACCESSIBILITY=1
QT_DBL_CLICK_DIST=30
QT_IM_MODULE=fcitx
QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1
QT_QPA_PLATFORM=
QT_SCALE_FACTOR_ROUNDING_POLICY=PassThrough
SDL_IM_MODULE=fcitx
SHELL=/bin/zsh
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
TERM=xterm-256color
USER=i78
WINDOWID=0
XAUTHORITY=/home/i78/.Xauthority
XDG_CACHE_HOME=/home/i78/.cache
XDG_CONFIG_HOME=/home/i78/.config
XDG_CURRENT_DESKTOP=Deepin
XDG_DATA_HOME=/home/i78/.local/share
XDG_GREETER_DATA_DIR=/var/lib/lightdm/data/i78
XDG_RUNTIME_DIR=/run/user/1000
XDG_SEAT=seat0
XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0
XDG_SESSION_CLASS=user
XDG_SESSION_DESKTOP=deepin
XDG_SESSION_ID=2
XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0
XDG_SESSION_TYPE=x11
XDG_VTNR=1
XMODIFIERS=@im=fcitx
SHLVL=1
OLDPWD=/home/i78
ZSH=/home/i78/.oh-my-zsh
PAGER=less
LESS=-R
LSCOLORS=Gxfxcxdxbxegedabagacad
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
GOLAND=/home/i78/IDEs/GoLand-2022.2
GOPATH=/home/i78/GOPATH
GOPROXY=https://goproxy.cn,direct
RUSTUP_DIST_SERVER=https://rsproxy.cn
RUSTUP_UPDATE_ROOT=https://rsproxy.cn/rustup
_=/usr/bin/env
                                                                                                               
~

@M4n5ter
Copy link
Author

M4n5ter commented Sep 28, 2022

oh, I found this to be a random event.
image

image

I opened 8, 3 of which failed with invalid charset name.

@dalance
Copy link
Owner

dalance commented Sep 28, 2022

Could you provide your less version?

@M4n5ter
Copy link
Author

M4n5ter commented Sep 28, 2022

These tabs for successful execution of procs also failed when they were executed again later.
image

@M4n5ter
Copy link
Author

M4n5ter commented Sep 28, 2022

Could you provide your less version?

~   
❯ less -V
less 487 (GNU regular expressions)
Copyright (C) 1984-2016  Mark Nudelman

less comes with NO WARRANTY, to the extent permitted by law.
For information about the terms of redistribution,
see the file named README in the less distribution.
Homepage: http://www.greenwoodsoftware.com/less

@M4n5ter
Copy link
Author

M4n5ter commented Sep 28, 2022

The invalid charset name seems to be random

~   
❯ LANG=en_US.UTF-8 procs     
invalid charset name
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
invalid charset name
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
invalid charset name
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
invalid charset name
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
invalid charset name
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
invalid charset name
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
invalid charset name
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
invalid charset name
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
invalid charset name
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
invalid charset name
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
invalid charset name
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
invalid charset name
                                                                                                                                                   
~   
❯ LANG=en_US.UTF-8 procs     
                                                                                                                                                   
~

@dalance
Copy link
Owner

dalance commented Sep 28, 2022

procs calls less with LESSCHARSET=utf-8 internally.
If the value "utf-8" is broken by any reason, less show the message like below.
It may not be related with LANG.

$ LESSCHARSET=utf- less
invalid charset name

@M4n5ter
Copy link
Author

M4n5ter commented Sep 28, 2022

procs calls less with LESSCHARSET=utf-8 internally. If the value "utf-8" is broken by any reason, less show the message like below. It may not be related with LANG.

$ LESSCHARSET=utf- less
invalid charset name

But it works fine when less works alone in my environment.

~   
❯ less temp1.txt 
                                                                                           
~   
❯ less temp2.txt 
                                                                                           
~   
❯ less temp3.txt
                                                                                           
~   
❯ LESSCHARSET=utf less temp4.txt   
invalid charset name
                                                                                           
~   
❯ LESSCHARSET=utf-8 less temp4.txt
                                                                                           
~

dalance added a commit that referenced this issue Sep 28, 2022
@dalance
Copy link
Owner

dalance commented Sep 28, 2022

Could you try cargo install --git https://github.com/dalance/procs ?
The sequence to call less is fixed.

@M4n5ter
Copy link
Author

M4n5ter commented Sep 28, 2022

Could you try cargo install --git https://github.com/dalance/procs ?
The sequence to call less is fixed.

image
image

invalid charset name appeared again.

@dalance
Copy link
Owner

dalance commented Sep 28, 2022

less-487.zip

Could you try the attached less which is show the passed value of LESSCHARSET?

$ unzip less-487.zip
$ cd less-487
$ ./configure
$ make

.config/procs/config.toml

[pager]
command = "path to the built binary"

@M4n5ter
Copy link
Author

M4n5ter commented Sep 28, 2022

less-487.zip

Could you try the attached less which is show the passed value of LESSCHARSET?

$ unzip less-487.zip
$ cd less-487
$ ./configure
$ make

.config/procs/config.toml

[pager]
command = "path to the built binary"

It's solved(try 50 times and succeed 50 times),but the sample file doesn't suit me.

~/less/less-487   
❯ procs --config > ~/.config/procs/config.toml
                                                                                                                                                   
~/less/less-487   
❯ echo command = '"......."' >> ~/.config/procs/config.toml

image

@M4n5ter
Copy link
Author

M4n5ter commented Sep 28, 2022

@dalance Thank you for your help :)

@dalance
Copy link
Owner

dalance commented Sep 28, 2022

I only added the following debug message to less-487.

197a198
>       PARG parg;
224c225,226
<               error("invalid charset name", NULL_PARG);
---
>               parg.p_string = name;
>               error("invalid charset name: %s", &parg);

So the cause may be the installed less binary.

@MithicSpirit
Copy link

I am experiencing the same issue (building from latest git commit: 928ff46, using LANG=en_US.UTF-8).

By creating an executable file in my PATH called less that logs the environment variables it was called with, I have determined that procs is setting LESSCHARSET=utf-8al/bin. This seems to be due to pager_envs expecting a c-string rather than just a rust slice (i.e. it needs to be null-terminated, regardless of length). The following diff has fixed it for me:

diff --git a/src/view.rs b/src/view.rs
index 7953c36..11ad82e 100644
--- a/src/view.rs
+++ b/src/view.rs
@@ -597,11 +597,11 @@ impl View {
         if let Some(ref pager) = config.pager.command {
             Pager::with_pager(pager)
                 // workaround for default less charset is "ascii" on some environments (ex. Ubuntu)
-                .pager_envs(&["LESSCHARSET=utf-8"])
+                .pager_envs(&["LESSCHARSET=utf-8\0"])
                 .setup();
         } else if which::which("less").is_ok() {
             Pager::with_pager("less -SR")
-                .pager_envs(&["LESSCHARSET=utf-8"])
+                .pager_envs(&["LESSCHARSET=utf-8\0"])
                 .setup();
         } else {
             Pager::with_pager("more -f").setup();

@MithicSpirit
Copy link

MithicSpirit commented Oct 2, 2022

Digging deeper into it, it seems like pager_envs wants an impl IntoIterator<Item = impl Into<OsString>>. The key here is that, when rust converts a &str into a OsString, it does not add the necessary null byte at the end.

EDIT: grammar

@dalance
Copy link
Owner

dalance commented Oct 3, 2022

@MithicSpirit OsString is not null-terminated. The cause of this issue seems to be in pager crate, I sent the PR to fix it.

https://gitlab.com/imp/pager-rs/-/merge_requests/8

@rbutoi
Copy link

rbutoi commented Oct 4, 2022

Can this issue be renamed? I'm hitting it too, seeing the exact same nondeterministic behaviour, without the zh_CN.UTF-8. And from reading discussion, looks like it's not specific to that env.

To help anyone else: short of dropping in binaries, the workaround is to disable paging: --pager disable. This would be good at the top as well.

@M4n5ter M4n5ter changed the title procs does not support zh_CN.UTF-8 invalid charset name Oct 4, 2022
@M4n5ter
Copy link
Author

M4n5ter commented Oct 4, 2022

I only added the following debug message to less-487.

197a198
>       PARG parg;
224c225,226
<               error("invalid charset name", NULL_PARG);
---
>               parg.p_string = name;
>               error("invalid charset name: %s", &parg);

So the cause may be the installed less binary.

less may not be related with this issue, the problem appeared again in my environment.

dalance added a commit that referenced this issue Oct 5, 2022
@dalance
Copy link
Owner

dalance commented Oct 5, 2022

I added @MithicSpirit 's workaround.
@M4n5ter How about the latest commit?

@MithicSpirit
Copy link

Latest commit works for me.

@M4n5ter
Copy link
Author

M4n5ter commented Oct 5, 2022

I am using f3518056f7a74abd2509080547c769f960ec6f39 and there is no problem at present.

@dalance
Copy link
Owner

dalance commented Oct 5, 2022

Thanks all.
I'll release v0.13.2 with the workaround.

@M4n5ter M4n5ter closed this as completed Oct 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants