Cannot opam install bap-llvm when system only has ncurses

[Pamplemousse]

Looking to install bap using opam, it fails when trying to install bap-llvm, with the following:

[nix-shell:~/Workspace/bap]$ opam install bap-llvm
The following actions will be performed:
  ∗ install bap-llvm 2.2.0

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[bap-llvm.2.2.0] found in cache

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[ERROR] The compilation of bap-llvm failed at "/home/pamplemousse/.opam/opam-init/hooks/sandbox.sh build make".

#=== ERROR while compiling bap-llvm.2.2.0 =====================================#
# context     2.0.7 | linux/x86_64 | ocaml-base-compiler.4.09.0 | https://opam.ocaml.org#72d9d430
# path        ~/.opam/4.09.0/.opam-switch/build/bap-llvm.2.2.0
# command     ~/.opam/opam-init/hooks/sandbox.sh build make
# exit-code   2
# env-file    ~/.opam/log/bap-llvm-28423-d2d111.env
# output-file ~/.opam/log/bap-llvm-28423-d2d111.out
### output ###
# [...]
# 2 warnings generated.
# `which clang++` -O2 -std=c++11 -fPIC -I/home/pamplemousse/.opam/4.09.0/lib/ocaml -I/home/pamplemousse/.opam/4.09.0/lib/bap -Ilib/bap_disasm -I/nix/store/hjbsxvj8az4s02aai49mn742aly98w3k-llvm-7.1.0/include  -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-[...]
# + `which clang++` -O2 -std=c++11 -fPIC -I/home/pamplemousse/.opam/4.09.0/lib/ocaml -I/home/pamplemousse/.opam/4.09.0/lib/bap -Ilib/bap_disasm -I/nix/store/hjbsxvj8az4s02aai49mn742aly98w3k-llvm-7.1.0/include  -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wn[...]
# warning: unknown warning option '-Wno-maybe-uninitialized'; did you mean '-Wno-uninitialized'? [-Wunknown-warning-option]
# warning: unknown warning option '-Wno-class-memaccess'; did you mean '-Wno-class-varargs'? [-Wunknown-warning-option]
# 2 warnings generated.
# /home/pamplemousse/.opam/4.09.0/bin/ocamlfind ocamlmklib -o lib/bap_llvm/bap_llvm_stubs -g '-lLLVMWebAssemblyDisassembler -lLLVMWebAssemblyCodeGen -lLLVMWebAssemblyDesc -lLLVMWebAssemblyAsmPrinter -lLLVMWebAssemblyAsmParser -lLLVMWebAssemblyInfo -lLLVMXCoreDisassembler -lLLVMXCoreCodeGen -lLLVMXCoreDesc -lLLVMXCoreInfo -lLLVMXCoreAsmPrinter -lLLVMSystemZDisassembler -lLLVMSystemZCodeGen -lLLV[...]
# + /home/pamplemousse/.opam/4.09.0/bin/ocamlfind ocamlmklib -o lib/bap_llvm/bap_llvm_stubs -g '-lLLVMWebAssemblyDisassembler -lLLVMWebAssemblyCodeGen -lLLVMWebAssemblyDesc -lLLVMWebAssemblyAsmPrinter -lLLVMWebAssemblyAsmParser -lLLVMWebAssemblyInfo -lLLVMXCoreDisassembler -lLLVMXCoreCodeGen -lLLVMXCoreDesc -lLLVMXCoreInfo -lLLVMXCoreAsmPrinter -lLLVMSystemZDisassembler -lLLVMSystemZCodeGen -lL[...]
# /nix/store/kcp0y7g1ir4dxq2gqz687i4vb7gqy8s0-binutils-2.31.1/bin/ld: cannot find -lcurses
# collect2: error: ld returned 1 exit status
# Command exited with code 2.
# make: *** [Makefile:8: build] Error 1



<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
┌─ The following actions failed
│ λ build bap-llvm 2.2.0
└─
╶─ No changes have been performed

As the title says, there is no curses package for my distribution: it only has ncurses.

Incidentally, I use NixOS, BAP is packaged for it, and the package configuration patches -lcurses out: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/ocaml-modules/bap/dont-add-curses.patch .

[ivg]

Well, the problem is in the upstream (llvm) and we don't really have a lot of choices here, without this line bap will be broken on nearly all distributions. We may alternatively try to link with -lncurses instead of -lcurses and have problems on systems that do not have ncurses but have only curses, if such exist. We can try to investigate it, but it is a change that is really hard to debug.

Alternatively, we can add the explicit support for your system and do not provide -lcurses for it but are you sure that you don't have the curses compatibility shim, usually, there is such a shim, have you tried to find one and to install it?

Concerning the NixOS patch, it is actually breaking bap (in bytecode and toplevel modes). You can, of course, repeat it yourself, as a workaround, just clone the opam-repository and patch it out. Here are the steps:

1. clone the ocaml/opam-repository

git clone git@github.com:ocaml/opam-repository.git

2. rebase the dont-add-curses.patch file and add it to `packages/bap-llvm/bap-llvm.2.2.0/files
3. add two lines to the packages/bap-llvm/bap-llvm.2.2.0/opam

patches: ["dont-add-curses.patch"]
extra-files: [
  ["dont-add-curses.patch" "md5=<the-md5sum-of-the-patch>"]
]

4. commit the changes
5. add the fixed repository opam repo add ./opam-repository
6. opam install bap

Besides, what OS are you using?

As a side note, we need to update the nixos bap package.

[Pamplemousse]

I am using NixOS.

without this line bap will be broken on nearly all distributions

Concerning the NixOS patch, it is actually breaking bap (in bytecode and toplevel modes)

I did not experience the BAP failure when installing, and using from the NixOS package.
Do you have a minimal example showing that it would be broken? That would help justify the change for the package in nixpkgs.

I am familiar with the process and the platform, so I am gonna have a look to updade the package in nixpkgs.

[ivg]

I did not experience the BAP failure when installing, and using from the NixOS package.

The problem occurs during runtime when the plugins are loaded with the dynamic linker. It depends on the kind of the linker, but usually it fails with the bytecode linker. So you can try to run baptop and see if there are any errors mentioning llvm/setupterm.

Also, the problem is really not in BAP or the linker or OCaml. The problem is in LLVM which links this symbol. And this symbol has various availability on various systems, so it depends on the packaging of LLVM and the system-specific libraries. Moreover, the symbol is brought into the llvm library also more or less accidentally, as far as I remember, it is not really used, but comes from a header, so it also depends on the order of inclusions and compiler optimizations.

EDIT: also your patch doesn't drop curses but replaces it with ncurses, which indeed should heal the issue (but please still check that baptop works).

[ivg]

Should be fixed by #1435