FZF broken on archlinux (ncurses library version changes) #350

kowalskey · 2015-09-18T08:53:52Z

Hello!

I noticed today that fzf stopped working on my Archlinux install.
Looks like ncurses got upgraded and libncursesw.so.5 is not available anymore.

I recompiled FZF to confirm that this was the problem.

Here's ldd output for packaged binary

% ldd fzf-0.10.3-linux_amd64                                                                                                                                    
        linux-vdso.so.1 (0x00007ffdfffb1000)
        libncursesw.so.5 => not found
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f2db6790000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f2db63ec000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f2db69ad000)

And here's ldd output for self-compiled binary:

% ldd fzf-linux_amd64                                                                                                                                           
        linux-vdso.so.1 (0x00007fff427d8000)
        libncursesw.so.6 => /usr/lib/libncursesw.so.6 (0x00007f157e171000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f157df54000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f157dbb0000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f157e3de000)

I think that fzf may benefit from shipping arch-specific binary or pointing in docs that it requires compilation on latest archlinux.

Or maybe it might perform/suggest compilation in install script ?

If you need any help/testing with this please let me know.

The text was updated successfully, but these errors were encountered:

junegunn · 2015-09-18T09:39:44Z

Can you try git pull and rerun the install script? It will download static binary if the default one does not work.

Related: 1de4cc3

jonaz · 2015-09-18T09:42:03Z

I have also problem on arch linux today after pulling latest from master.

I manually compiled fzf and that binary works.

Using the linked one:
./fzf-0.10.5-linux_amd64: error while loading shared libraries: libncursesw.so.5: cannot open shared object file: No such file or directory

Using the static binary:

fzf: loadlocale.c:130: _nl_intern_locale_data: Assertion `cnt < (sizeof (_nl_value_type_LC_COLLATE) / sizeof (_nl_value_type_LC_COLLATE[0]))' failed.
SIGABRT: abort
PC=0x57fce9
signal arrived during cgo execution

goroutine 8 [syscall, locked to thread]:
runtime.cgocall_errno(0x401e10, 0xc2080998e8, 0x0)
    /go1.4/src/runtime/cgocall.go:130 +0xf5 fp=0xc2080998c8 sp=0xc2080998a0
github.com/junegunn/fzf/src/curses._Cfunc_setlocale(0x6, 0x7f925c0008c0, 0x0)
    /go/src/github.com/junegunn/fzf/src/curses/:146 +0x44 fp=0xc2080998e8 sp=0xc2080998c8
github.com/junegunn/fzf/src/curses.Init(0xc20801e2a0, 0xc208090100)
    /go/src/github.com/junegunn/fzf/src/curses/curses.go:260 +0x104 fp=0xc2080999d8 sp=0xc2080998e8
github.com/junegunn/fzf/src.func·024()
    /go/src/github.com/junegunn/fzf/src/terminal.go:221 +0x46 fp=0xc2080999f0 sp=0xc2080999d8
github.com/junegunn/fzf/src.(*Terminal).Loop(0xc208034000)
    /go/src/github.com/junegunn/fzf/src/terminal.go:707 +0x143 fp=0xc208099fd8 sp=0xc2080999f0
runtime.goexit()
    /go1.4/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc208099fe0 sp=0xc208099fd8
created by github.com/junegunn/fzf/src.Run
    /go/src/github.com/junegunn/fzf/src/core.go:196 +0xb95

goroutine 1 [semacquire]:
sync.(*Cond).Wait(0xc20802a040)
    /go1.4/src/sync/cond.go:62 +0x9e
github.com/junegunn/fzf/src/util.(*EventBox).Wait(0xc20801e000, 0xc208050f38)
    /go/src/github.com/junegunn/fzf/src/util/eventbox.go:32 +0xaf
github.com/junegunn/fzf/src.Run(0xc208062000)
    /go/src/github.com/junegunn/fzf/src/core.go:266 +0xd11
main.main()
    /go/src/github.com/junegunn/fzf/src/fzf/main.go:6 +0x2c

goroutine 5 [syscall]:
os/signal.loop()
    /go1.4/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
    /go1.4/src/os/signal/signal_unix.go:27 +0x35

goroutine 6 [runnable]:
github.com/junegunn/fzf/src.(*Reader).readFromCommand(0xc20801e060, 0x680770, 0x5a)
    /go/src/github.com/junegunn/fzf/src/reader.go:72 +0x187
github.com/junegunn/fzf/src.(*Reader).ReadSource(0xc20801e060)
    /go/src/github.com/junegunn/fzf/src/reader.go:26 +0x86
created by github.com/junegunn/fzf/src.Run
    /go/src/github.com/junegunn/fzf/src/core.go:140 +0x718

goroutine 7 [semacquire]:
sync.(*Cond).Wait(0xc20802a080)
    /go1.4/src/sync/cond.go:62 +0x9e
github.com/junegunn/fzf/src/util.(*EventBox).Wait(0xc20801e080, 0xc208019780)
    /go/src/github.com/junegunn/fzf/src/util/eventbox.go:32 +0xaf
github.com/junegunn/fzf/src.(*Matcher).Loop(0xc20803a660)
    /go/src/github.com/junegunn/fzf/src/matcher.go:67 +0x75
created by github.com/junegunn/fzf/src.Run
    /go/src/github.com/junegunn/fzf/src/core.go:191 +0xb1a

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /go1.4/src/runtime/asm_amd64.s:2232 +0x1

rax     0x0
rbx     0x7f9268a85000
rcx     0x57fce9
rdx     0x6
rdi     0x7c56
rsi     0x7c5a
rbp     0x698e40
rsp     0x7f9269d45768
r8      0x7f925c0011b0
r9      0x0
r10     0x8
r11     0x206
r12     0x697350
r13     0x82
r14     0x697418
r15     0x3
rip     0x57fce9
rflags  0x206
cs      0x33
fs      0x0
gs      0x0

junegunn · 2015-09-18T09:48:28Z

@jonaz ~~Please rerun the install script and see if it helps.~~ Oops. Sorry, I'll have to look into it. Hmm.

jonaz · 2015-09-18T09:49:38Z

@junegunn i just manually compiled fzf and that binary works. I already tried removing ~/.fzf and cloing and running ./install again without success.

Check my last comment above. I saw different errors when using your static and linked binary.

junegunn · 2015-09-18T10:04:31Z

@kowalskey reported that the static binary works on his machine.

1de4cc3#commitcomment-13306953

I wonder what makes the difference.

kowalskey · 2015-09-18T10:17:28Z

Assertion `cnt < (sizeof (_nl_value_type_LC_COLLATE) / sizeof (_nl_value_type_LC_COLLATE[0]))' failed.

@jonaz I'm not sure but assertion above may suggest some locale problems.
please check your locale settings
for reference here are my LC_*:

env | egrep LC\|LOCALE\|LANG                                                                                                                                      
LANG=pl_PL.utf8
LC_COLLATE=C
LC_CTYPE=pl_PL.utf8
LC_MEASUREMENT=pl_PL.UTF-8
LC_MONETARY=pl_PL.UTF-8
LC_NUMERIC=pl_PL.UTF-8
LC_PAPER=pl_PL.UTF-8
LC_TIME=pl_PL.UTF-8

I removed binaries and called install once again:

% ./install
Downloading bin/fzf ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   404    0   404    0     0    513      0 --:--:-- --:--:-- --:--:--   512
100  993k  100  993k    0     0   231k      0  0:00:04  0:00:04 --:--:--  333k
  - Creating symlink: bin/fzf-0.10.5-linux_amd64 -> bin/fzf
  - Checking fzf executable ... Error: /home/suawek/.fzf/bin/fzf: error while loading shared libraries: libncursesw.so.5: cannot open shared object file: No such file or directory
Downloading bin/fzf ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   411    0   411    0     0    524      0 --:--:-- --:--:-- --:--:--   524
100 1536k  100 1536k    0     0   317k      0  0:00:04  0:00:04 --:--:--  403k
  - Creating symlink: bin/fzf-0.10.5-linux_amd64-static -> bin/fzf
  - Checking fzf executable ... 0.10.5
Do you want to add auto-completion support? ([y]/n) y
Do you want to add key bindings? ([y]/n)


Generate ~/.fzf.bash ... OK
Generate ~/.fzf.zsh ... OK
Update fish_user_paths ... OK
Symlink ~/.config/fish/functions/fzf_key_bindings.fish ... OK

Update /home/suawek/.bashrc:
  - [ -f ~/.fzf.bash ] && source ~/.fzf.bash
    - Already exists: line #110

Update /home/suawek/.zshrc:
  - [ -f ~/.fzf.zsh ] && source ~/.fzf.zsh
    - Already exists: line #87

Update /home/suawek/.config/fish/functions/fish_user_key_bindings.fish:
  - fzf_key_bindings
    - Already exists: line #2

Finished. Restart your shell or reload config file.
   source ~/.bashrc  # bash
   source ~/.zshrc   # zsh
   fzf_key_bindings  # fish

Use uninstall script to remove fzf.

For more information, see: https://github.com/junegunn/fzf

kowalskey · 2015-09-18T10:18:54Z

after unsetting LC_COLLATE I reproduced problem:

fzf: loadlocale.c:130: _nl_intern_locale_data: Assertion `cnt < (sizeof (_nl_value_type_LC_COLLATE) / sizeof (_nl_value_type_LC_COLLATE[0]))' failed.
SIGABRT: abort
PC=0x57fce9
signal arrived during cgo execution

goroutine 8 [syscall, locked to thread]:
runtime.cgocall_errno(0x401e10, 0xc2080518e8, 0x0)
        /go1.4/src/runtime/cgocall.go:130 +0xf5 fp=0xc2080518c8 sp=0xc2080518a0
github.com/junegunn/fzf/src/curses._Cfunc_setlocale(0x6, 0x1060bc0, 0x0)
        /go/src/github.com/junegunn/fzf/src/curses/:146 +0x44 fp=0xc2080518e8 sp=0xc2080518c8
github.com/junegunn/fzf/src/curses.Init(0xc208020260, 0xc208030100)
        /go/src/github.com/junegunn/fzf/src/curses/curses.go:260 +0x104 fp=0xc2080519d8 sp=0xc2080518e8
github.com/junegunn/fzf/src.func·024()
        /go/src/github.com/junegunn/fzf/src/terminal.go:221 +0x46 fp=0xc2080519f0 sp=0xc2080519d8
github.com/junegunn/fzf/src.(*Terminal).Loop(0xc208036000)
        /go/src/github.com/junegunn/fzf/src/terminal.go:707 +0x143 fp=0xc208051fd8 sp=0xc2080519f0
runtime.goexit()
        /go1.4/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc208051fe0 sp=0xc208051fd8
created by github.com/junegunn/fzf/src.Run
        /go/src/github.com/junegunn/fzf/src/core.go:196 +0xb95

goroutine 1 [semacquire]:
sync.(*Cond).Wait(0xc208038140)
        /go1.4/src/sync/cond.go:62 +0x9e
github.com/junegunn/fzf/src/util.(*EventBox).Wait(0xc208020300, 0xc208052f38)
        /go/src/github.com/junegunn/fzf/src/util/eventbox.go:32 +0xaf
github.com/junegunn/fzf/src.Run(0xc208060000)
        /go/src/github.com/junegunn/fzf/src/core.go:266 +0xd11
main.main()
        /go/src/github.com/junegunn/fzf/src/fzf/main.go:6 +0x2c

goroutine 5 [syscall]:
os/signal.loop()
        /go1.4/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
        /go1.4/src/os/signal/signal_unix.go:27 +0x35

goroutine 6 [runnable]:
os.Pipe(0x8, 0x7ffe7533da18, 0x0, 0x0)
        /go1.4/src/os/pipe_linux.go:17 +0xf2
os/exec.(*Cmd).StdoutPipe(0xc208096000, 0x0, 0x0, 0x0, 0x0)
        /go1.4/src/os/exec/exec.go:467 +0x27c
github.com/junegunn/fzf/src.(*Reader).readFromCommand(0xc208020340, 0x7ffe7533cb82, 0x6e)
        /go/src/github.com/junegunn/fzf/src/reader.go:63 +0xd2
github.com/junegunn/fzf/src.(*Reader).ReadSource(0xc208020340)
        /go/src/github.com/junegunn/fzf/src/reader.go:26 +0x86
created by github.com/junegunn/fzf/src.Run
        /go/src/github.com/junegunn/fzf/src/core.go:140 +0x718

goroutine 7 [semacquire]:
sync.(*Cond).Wait(0xc208038180)
        /go1.4/src/sync/cond.go:62 +0x9e
github.com/junegunn/fzf/src/util.(*EventBox).Wait(0xc208020360, 0xc20801bf80)
        /go/src/github.com/junegunn/fzf/src/util/eventbox.go:32 +0xaf
github.com/junegunn/fzf/src.(*Matcher).Loop(0xc20803c6f0)
        /go/src/github.com/junegunn/fzf/src/matcher.go:67 +0x75
created by github.com/junegunn/fzf/src.Run
        /go/src/github.com/junegunn/fzf/src/core.go:191 +0xb1a

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
        /go1.4/src/runtime/asm_amd64.s:2232 +0x1

rax     0x0
rbx     0x7f487e12e000
rcx     0x57fce9
rdx     0x6
rdi     0x1aa7
rsi     0x1aa7
rbp     0x698e40
rsp     0x7ffe7533a7f8
r8      0xffffffff
r9      0x0
r10     0x8
r11     0x202
r12     0x697350
r13     0x82
r14     0x697418
r15     0x3
rip     0x57fce9
rflags  0x202
cs      0x33
fs      0x0
gs      0x0

So this is triggered by LC_COLLATE being unset or otherwise unavailable.

junegunn · 2015-09-18T10:23:38Z

@kowalskey Thanks for the investigation! I'll see what I can do.

jonaz · 2015-09-18T10:33:28Z

This is my LOCALE/LC/LANG:

$ env | egrep LC|LOCALE|LANG
LANG=en_US.UTF-8

junegunn · 2015-09-18T10:36:54Z

@jonaz I could also reproduce the problem:

With $LANG set, the static binary crashes
Without $LANG, it works: LANG= fzf

I'm trying to find the right solution to the problem.

junegunn · 2015-09-18T10:55:01Z

Possibly related: rstudio/rmarkdown#498
Could be an issue of Arch, but I'm not sure.

kowalskey · 2015-09-18T12:20:21Z

It seems to be issue with Archlinux. There are some topics on forums, however AFAIK no practical (working) solutions are present.
Some people reported successes with their solutions, however I was unable to confirm that.

Few topics I encountered are:

I tested a little and noticed that when LANG is set to locale generated by locale-gen fzf crashes.
To be more specific:

If LANG is set to empty or meaningless value e.g LANG=foo - fzf works
If generated locales are deleted (/usr/lib/locale/locale-archive is not present) - fzf works
If LANG is set to locale generate by locale-gen and locale is present in ``/usr/lib/locale/locale-archive` - fzf crashes

at least this is what I observe on my system.

I'll try to find some other packages which are affected by this and if there aren't any bugs at http://bugs.archlinux.org regarding glibc / locale problems - I'll report one and leave URL here for reference.

kowalskey · 2015-09-18T17:55:21Z

Aaand I think I pinpointed what is the problem.

both me and @jonaz use statically compiled binary which is probably compiled against older glibc version.
In glibc 2.22 locale format changed a bit.

So upgrading to glibc 2.22 on arch will automatically call locale-gen and thus update locales.
And because fzf is probably compiled against older version (@junegunn can probably confirm that) it will fail when trying to load file saved in newer format.

I think that static recompilation against newer glibc will fix that problem on Arch. Or at least it should help to confirm whether this is a problem.

junegunn · 2015-09-19T01:10:18Z

@kowalskey Awesome, thanks for looking into it. I was unsuccessful to build static binary on Arch, so currently I'm building it on Ubuntu. (I got the impression that Arch community was against this notion of static linking in general: https://www.archlinux.org/todo/remove-static-libraries/)

linux: docker-arch
    docker run -i -t -v $(GOPATH):/go junegunn/arch-sandbox \
        /bin/bash -ci 'cd /go/src/github.com/junegunn/fzf/src; make'

linux-static: docker-ubuntu
    docker run -i -t -v $(GOPATH):/go junegunn/ubuntu-sandbox \
        /bin/bash -ci 'cd /go/src/github.com/junegunn/fzf/src; make STATIC=1'

So we can either:

Find a way to build static binaries on Arch
or to upgrade Ubuntu and see if it helps

junegunn · 2015-09-19T01:47:47Z

Okay, I was able to build a static binary on Arch by building ncurses6 and gpm from source (and with LDFLAGS := --ldflags '-extldflags "-static -ldl -lgpm"'), and the resulting binary doesn't suffer the problem, however it does not work on Ubuntu. Maybe it's okay since the non-static version works there, but we need to check how it works on other platforms as well.

Related: #322

Instead of building a separate statically-linked binary, build partially-static binary that only contains ncurses to avoid compatibility issues in libc.

junegunn · 2015-09-19T09:40:24Z

I think I got this. Statically linking only ncurses seems to fix the compatibility issue. I'll release 0.10.6 binaries with the change and let you know. Thank you very much for your help.

junegunn · 2015-09-19T13:20:31Z

Just released 0.10.6. Please update the git repo and re-run the install script. The problem should be gone now. Thanks.

[root@ca9ed173557c fzf]# ldd bin/fzf
        linux-vdso.so.1 (0x00007fff03be6000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f0aa960e000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f0aa926a000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f0aa982b000)

junegunn · 2015-09-19T13:28:51Z

Hmm, realized that the new binary does not run on old linux boxes (RHEL5) so I reverted the update.

junegunn · 2015-09-19T15:28:49Z

Phew, I changed the makefile to build the binary on Centos 6 which has old glibc 2.12, and finally it works fine on Ubuntu 14, Centos 6, RHEL 5, RHEL 6, and the latest Arch linux.

kowalskey · 2015-09-19T16:04:50Z

Great job, thanks!

lyeoh · 2015-09-20T01:58:37Z

On Arch x86_64, I had to change line 8 of curses.go

#cgo linux,amd64 LDFLAGS: -l:libncurses.a -l:libtinfo.a -l:libgpm.a -ldl

to

#cgo linux,amd64 LDFLAGS: -lncurses

to get it to compile. Otherwise, the build halts with the following.

/usr/bin/ld: cannot find -l:libncurses.a
/usr/bin/ld: cannot find -l:libtinfo.a
/usr/bin/ld: cannot find -l:libgpm.a
collect2: error: ld returned 1 exit status
==> ERROR: A failure occurred in build().
    Aborting...

junegunn · 2015-09-20T04:02:29Z

@lyeoh Yeah, that's because on Arch, libraries are not shipped with static libraries (.a). You'll have to manually compile and install ncurses and gpm to build with the directives. Leaving only -lncurses will build a binary that dynamically links to ncurses, it'll work fine on the machine but it will not run on system with ncurses 5. I decided to build the official binary on Centos 6 as noted above to make it compatible on most systems.

alerque · 2015-09-20T06:29:14Z

If you want to compile a static binary release that's fine for those that want it, but the source code repository should be setup to compile for their host systems. Most of us compiling from the source are going to want to run on the system it was compiled for. A statically linked build could be a build time option but it should not be the default.

junegunn · 2015-09-20T06:36:54Z

@alerque Yeah I see the point, please send me a PR, so I can still build statically compiled binary with make linux and you can build dynamic binary without having to edit the source, then we'll all be happy :)

junegunn added this to the 0.10.6 milestone Sep 19, 2015

junegunn added a commit that referenced this issue Sep 19, 2015

Build partially-static binary for linux (#350)

6bc3fe6

Instead of building a separate statically-linked binary, build partially-static binary that only contains ncurses to avoid compatibility issues in libc.

junegunn closed this as completed Sep 19, 2015

junegunn reopened this Sep 19, 2015

junegunn closed this as completed Sep 19, 2015

netei mentioned this issue Sep 24, 2015

loadlocale.c:130: _nl_intern_locale_data: Assertion failed #356

Closed

smrg-lm mentioned this issue Apr 25, 2018

Cannot start Carla, loadlocale.c related error falkTX/Carla#623

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FZF broken on archlinux (ncurses library version changes) #350

FZF broken on archlinux (ncurses library version changes) #350

kowalskey commented Sep 18, 2015

junegunn commented Sep 18, 2015

jonaz commented Sep 18, 2015

junegunn commented Sep 18, 2015

jonaz commented Sep 18, 2015

junegunn commented Sep 18, 2015

kowalskey commented Sep 18, 2015

kowalskey commented Sep 18, 2015

junegunn commented Sep 18, 2015

jonaz commented Sep 18, 2015

junegunn commented Sep 18, 2015

junegunn commented Sep 18, 2015

kowalskey commented Sep 18, 2015

kowalskey commented Sep 18, 2015

junegunn commented Sep 19, 2015

junegunn commented Sep 19, 2015

junegunn commented Sep 19, 2015

junegunn commented Sep 19, 2015

junegunn commented Sep 19, 2015

junegunn commented Sep 19, 2015

kowalskey commented Sep 19, 2015

lyeoh commented Sep 20, 2015

junegunn commented Sep 20, 2015

alerque commented Sep 20, 2015

junegunn commented Sep 20, 2015

FZF broken on archlinux (ncurses library version changes) #350

FZF broken on archlinux (ncurses library version changes) #350

Comments

kowalskey commented Sep 18, 2015

junegunn commented Sep 18, 2015

jonaz commented Sep 18, 2015

junegunn commented Sep 18, 2015

jonaz commented Sep 18, 2015

junegunn commented Sep 18, 2015

kowalskey commented Sep 18, 2015

kowalskey commented Sep 18, 2015

junegunn commented Sep 18, 2015

jonaz commented Sep 18, 2015

junegunn commented Sep 18, 2015

junegunn commented Sep 18, 2015

kowalskey commented Sep 18, 2015

kowalskey commented Sep 18, 2015

junegunn commented Sep 19, 2015

junegunn commented Sep 19, 2015

junegunn commented Sep 19, 2015

junegunn commented Sep 19, 2015

junegunn commented Sep 19, 2015

junegunn commented Sep 19, 2015

kowalskey commented Sep 19, 2015

lyeoh commented Sep 20, 2015

junegunn commented Sep 20, 2015

alerque commented Sep 20, 2015

junegunn commented Sep 20, 2015