Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

can't rename file in non-English #997

Closed
paulcarroty opened this issue May 14, 2024 · 12 comments
Closed

can't rename file in non-English #997

paulcarroty opened this issue May 14, 2024 · 12 comments

Comments

@paulcarroty
Copy link

paulcarroty commented May 14, 2024

Tested using cw and cW, got something weird: 123
Version: 0.13

@xaizek
Copy link
Member

xaizek commented May 14, 2024

OS? Original name is in Cyrillic and looks correct? What's the output of locale in a shell?

@paulcarroty
Copy link
Author

paulcarroty commented May 14, 2024

name is in Cyrillic and looks correct?

Original file can be in Latin/Cyr, the same weird characters appear after typing in Cyr.

$ uname -o
GNU/Linux
$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

@xaizek
Copy link
Member

xaizek commented May 14, 2024

locale looks fine. What's the filesystem and is it mounted with some locale/encoding-related options? You could also try checking whether the behaviour differs in something like /tmp.

I also assume that you're starting vifm from a shell with that locale output, otherwise it could be an issue of environment setup by, say, your DE when spawning a terminal (e.g., if locale is set in shell's config instead of on login or system-wide).

@paulcarroty
Copy link
Author

paulcarroty commented May 14, 2024

/dev/sda2 on / type ext4 (rw,relatime,stripe=256)
/dev/sda4 on /home type ext4 (rw,nodev,relatime,discard,stripe=256)

you're starting vifm from a shell with that locale output

Correct.

You could also try checking whether the behaviour differs in something like /tmp

Same issue. Also tested on Xterm and Ubuntu 24.04 LTS.

@xaizek
Copy link
Member

xaizek commented May 14, 2024

I'm running out of things that could affect the outcome. There is a conversion from wide to narrow encoding because input is processed as Unicode, but is then converted to local encoding, which is why locale could make a difference. One thing you could try is renaming a file in Vifm to абв (to have successive code point numbers) for a name and posting how ls prints that file after the rename. This might suggest how character values get transformed because ls now uses escapes for weird byte sequences.

I'll also need to check in a VM if I get the same on 24.04 LTS. What's the other distribution where you see the issue?

@paulcarroty
Copy link
Author

paulcarroty commented May 14, 2024

абв in terminal appeared as :
abc

printf '%s\n'  | od -t x1 -a
0000000  ee  80  80  ee  80  80  ee  80  80  0a
          n nul nul   n nul nul   n nul nul  nl
0000012

@xaizek
Copy link
Member

xaizek commented May 14, 2024

Thanks, that is not what I expected.

@xaizek
Copy link
Member

xaizek commented May 19, 2024

So far I wasn't able to reproduce this inside docker or a VM with Ubuntu 24.04 LTS.

@paulcarroty
Copy link
Author

OK, tested on Linux Mint and it works. Does Vifm needs any extra build dependency for non-EN lang support?
This is my distro .spec file.

@xaizek
Copy link
Member

xaizek commented May 29, 2024

No, if locales work, the rest should work. I just tried clearlinux docker container and do see the issue there. Will need to do some tests to see what might be the reason behind it.

@xaizek
Copy link
Member

xaizek commented Jun 1, 2024

wget_wch() from ncurses doesn't return non-latin input on Clear Linux. The input result looks like Ctrl+Space combination which internally represented as 0xe000, the character code you get. Don't know why this happens on Clear Linux though, https://github.com/clearlinux-pkgs/ncurses/blob/main/ncurses.spec looks fine. I made a test program:

Test program

// 1. Save as test.c
// 2. Compile:
//      gcc -Wall -g -o test test. -lcursesw
// 3. Run:
//      ./test
// 4. Type in something
// 5. Quit by pressing Escape key or Ctrl+C

#define _XOPEN_SOURCE_EXTENDED

#include <curses.h>

#include <locale.h>
#include <stdlib.h>
#include <wchar.h>
#include <wctype.h>

static WINDOW *win;

static void init(void)
{
    printf("MB_CUR_MAX: %ld\n", MB_CUR_MAX);
    setlocale(LC_ALL, "");
    printf("MB_CUR_MAX: %ld\n", MB_CUR_MAX);

    initscr();
    win = subwin(stdscr, 1, getmaxx(stdscr), 0, 0);
    refresh();
}

static void setup(void)
{
    noecho();
    wtimeout(win, 1000);
}

static void loop(void)
{
    wint_t c = 1;
    int result;

    result = wget_wch(win, &c);
    while (c != 27) {
        if (result == OK) {
            if (iswprint(c)) {
                wchar_t buf[] = { c, 0 };
                waddwstr(win, buf);
            } else {
                wprintw(win, "[0x%04x]", c);
            }
            wrefresh(win);
        }

        c = 1;
        result = wget_wch(win, &c);
    }
}

static void clean(void)
{
    delwin(win);
    endwin();
}

int main(void)
{
    init();
    setup();
    loop();
    clean();
    return 0;
}

It exposes the same behaviour as Vifm which makes ncurses build on Clear Linux a likely cause of the issue. An alternative is that the program has the same issue as Vifm which I just don't see. However if I build ncurses 6.4 manually like here

wget http://ftp.gnu.org/gnu/ncurses/ncurses-6.4.tar.gz
tar -xf ncurses-6.4.tar.gz
NCURSES_DIR="$PWD/ncurses-6.4"
pushd "$NCURSES_DIR"
./configure --without-shared --enable-widec --prefix=/ \
--without-normal --without-debug --without-cxx --without-cxx-binding \
--without-ada --without-manpages --without-tests
make -j4
make DESTDIR="$PWD/build" install
popd

and link that test program or Vifm to it, I see no issues, so it seems to be related to packaging of ncurses. It's possible there is something that Vifm could do in this regard, but at the moment I don't think what that could be.

ncurses-6.4-20230708.tgz isn't the same as ncurses-6.4.tar.gz, maybe that's just a buggy version (the URL for downloading doesn't even work, because that was a development version).

@paulcarroty
Copy link
Author

Wow, thanks for digging. Seems like it's related to strictly non-English characters: tested français got franais.

@paulcarroty paulcarroty changed the title can't rename file in Cyrillic can't rename file in non-English Jun 1, 2024
xaizek added a commit to xaizek/vifm that referenced this issue Jun 11, 2024
iswprint(key) won't handle it properly because Ctrl+Space is '\0' and
it's translated to a custom WC_C_SPACE to be able to process it given
that it normally signifies ends of a string.

See vifm#997 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants