Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ocamldebug and ocamldoc segfault or Failure("input_value: unknown custom block identifier") #7771

Open
vicuna opened this Issue Apr 13, 2018 · 6 comments

Comments

Projects
None yet
1 participant
@vicuna
Copy link
Collaborator

vicuna commented Apr 13, 2018

Original bug ID: 7771
Reporter: peterjc
Status: new
Resolution: open
Priority: normal
Severity: minor
Platform: x64
OS: Linux, macOS
Version: 4.06.1
Category: configure and build/install

Bug description

I have packaged OCaml for conda on the conda-forge channel,
https://anaconda.org/conda-forge/ocaml
https://github.com/conda-forge/ocaml-feedstock/

This system compiles binaries on virtual machines (Linux on CircleCI, macOS on TravisCI) and shares the binaries for easy installation, e.g.

conda install -c conda-forge ocaml

Conda supports Windows, macOS and Linux but currently the OCaml package only covers macOS and Linux.

The recipes seem to work, but these commands fail as logged here:
conda-forge/ocaml-feedstock#3

https://circleci.com/gh/conda-forge/ocaml-feedstock/4?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link
Linux CircleCI,

$ ocamldebug -version
Fatal error: exception Failure("input_value: unknown custom block identifier")
https://travis-ci.org/conda-forge/ocaml-feedstock/builds/363182681?utm_source=github_status&utm_medium=notification
macOS, TravisCI,

$ ocamldebug -version
/Users/travis/miniconda3/conda-bld/ocaml_1523031087918/test_tmp/run_test.sh: line 7: 33648 Segmentation fault: 11 ocamldebug -version
https://travis-ci.org/conda-forge/ocaml-feedstock/builds/363190382?utm_source=github_status&utm_medium=notification
macOS, TravisCI,

$ ocamldoc -version
/Users/travis/miniconda3/conda-bld/ocaml_1523032262217/test_tmp/run_test.sh: line 8: 33653 Segmentation fault: 11 ocamldoc -version

I see this on OCaml 5.06.0, 5.06.1 and also 5.02.0 as well, see:
conda-forge/ocaml-feedstock#7

I strongly suspect that something in the conda build scripts in combination with your build process is breaking, but don't really know where to start.

Steps to reproduce

Run the conda builds from https://github.com/conda-forge/ocaml-feedstock/

e.g. You could fork the repository on GitHub, turn on CircleCI and TravisCI for your fork, and push a white space commit to GitHub to trigger the build.

Additional information

Having an OCaml developer join or take over the conda recipe would be most welcome.

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

vicuna commented Apr 13, 2018

Comment author: @gasche

The first wild guess would be a version mismatch between the compiler distribution that produced the ocamldebug/ocamldoc executables and the one that is available on the machine trying to run them.

Can you confirm that all the other commands tested in your test: recipe ( https://github.com/conda-forge/ocaml-feedstock/blob/master/recipe/meta.yaml ) work as expected?

One thing that is special about ocamldoc/ocamldebug is that they remain distributed as bytecode executables by default, while most other OCaml commands (but I'm not sure all of them) give the short name to their native-compiled versions. This suggests that maybe native programs work correctly on your system, but bytecode programs don't. Could you check for example (ocamlc.byte -version) and (ocamlc.opt -version)? If this hypothesis is correct, only the latter should work.

Bytecode-compiled OCaml programs start with a shebang line that indicate the location of the bytecode interpreter (ocamlrun). The path is decided at build time, which is one of the reasons why OCaml programs are not relocatable in practice. Can you lookup the shebang line on your test systems? Is there an ocamlrun program at the right OCaml version in the requested directory? If there is an ocamlrun program at the right version in a different directory, does the command

/path/to/the/right/ocamlrun ocamldebug -version

work correctly?

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

vicuna commented Apr 13, 2018

Comment author: peterjc

As to your first guess, these failing tests are on the same VM which did the compiling - although not exactly the same environment as they try to simulate installing the finished package.

Yes, all the simple test commands (using -version or -help) work except these two. That these two commands are special (bytecode only) seems to be important.

I was wondering about the native vs bytecode, and if I could force native only for example.

This hint seems like a very important clue: "Bytecode-compiled OCaml programs start with a shebang line that indicate the location of the bytecode interpreter (ocamlrun)"

I know that the conda build system does make efforts to re-write paths referring to libraries etc so that on the end user's machine they will point at the conda provided libraries and not the system libraries. That may need to be triggered here explicitly.

I will explore the hashbang line, and try explicitly invoking them via ocamlrun, and get back to you.

Thank you!

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

vicuna commented Apr 13, 2018

Comment author: peterjc

Debugging this here: conda-forge/ocaml-feedstock#8

I'm pretty sure that the hashbang lines are correct, both at the point of the test on TravisCI/CircleCI, and once installed on an end user's system:

Confirming the hashbangs look fine with an actual installation of the ocaml v5.06.0 package under Conda on Linux:

/mnt/shared/scratch/xxx/apps/conda/bin/ocamlrun
$ which ocamldebug
/mnt/shared/scratch/xxx/apps/conda/bin/ocamldebug
$ which ocamldoc
/mnt/shared/scratch/xxx/apps/conda/bin/ocamldoc
$ head -n 1 which ocamldebug
#!/mnt/shared/scratch/xx/apps/conda/bin/ocamlrun
$ head -n 1 which ocamldoc
#!/mnt/shared/scratch/xxx/apps/conda/bin/ocamlrun
On this Linux system those tools seg-fault:

$ ocamlrun -version
The OCaml runtime, version 4.06.0
$ ocamldoc -version
Segmentation fault (core dumped)
$ ocamldebug -version
Segmentation fault (core dumped)
They also fail this way:

$ ocamlrun which ocamldebug -version
Segmentation fault (core dumped)
$ ocamlrun which ocamldoc -version
Segmentation fault (core dumped)

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

vicuna commented Apr 13, 2018

Comment author: @gasche

Does

ocamldoc.opt -version

work well on the system?

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

vicuna commented Apr 13, 2018

Comment author: peterjc

Yes, "ocamldoc.opt -version" works on Linux / CircleIO, macOS / TravisCI, and on our local Linux system where I installed the conda package:

conda-forge/ocaml-feedstock#8 (comment)

This is consistent with the native binaries (*.opt) working but there being a problem with the bytecode versions.

Might this be the problem, quoting your INSTALL file:

9- After installation, do not strip the ocamldebug and ocamlbrowser
executables. (These are mixed-mode executables, containing both
compiled C code and OCaml bytecode; stripping erases the bytecode!)
Other executables such as ocamlrun can safely be stripped.

Is there any easy check? e.g. On the Linux system where the package is installed:

$ file which ocamlc.opt
/mnt/shared/scratch/xxx/apps/conda/bin/ocamlc.opt: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, BuildID[sha1]=2be5e1b8c3665ef9470115ba2b9ae02cc27da238, not stripped

$ file which ocamlc.byte
/mnt/shared/scratch/xxx/apps/conda/bin/ocamlc.byte: data

$ ocamlc.opt -version
4.06.0
$ ocamlc.byte -version
(no output)

Is the file output then a sign of a problem with the byte binaries?

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

vicuna commented Apr 26, 2018

Comment author: peterjc

Any other suggestions for debugging the byte binaries?

I have checked their hashbang, but is there anything to dump the rest of the file?

The following is from a Linux machine with the current (problematic) conda package installed,

$ strings which ocamlc.byte | head -n 1
#!/mnt/shared/scratch/xxx/apps/conda/bin/ocamlrun

$ strings which ocamlc.byte | grep feedstock | head
J/feedstock_root/build_artefacts/ocaml_1523031921566/work/ocaml-4.06.0/boot
L/feedstock_root/build_artefacts/ocaml_1523031921566/work/ocaml-4.06.0/stdlib@
E/feedstock_root/build_artefacts/ocaml_1523031921566/work/ocaml-4.06.0
L/feedstock_root/build_artefacts/ocaml_1523031921566/work/ocaml-4.06.0/driver@
R/feedstock_root/build_artefacts/ocaml_1523031921566/work/ocaml-4.06.0/compilerlibs
E/feedstock_root/build_artefacts/ocaml_1523031921566/work/ocaml-4.06.0
L/feedstock_root/build_artefacts/ocaml_1523031921566/work/ocaml-4.06.0/driver@
R/feedstock_root/build_artefacts/ocaml_1523031921566/work/ocaml-4.06.0/compilerlibs
E/feedstock_root/build_artefacts/ocaml_1523031921566/work/ocaml-4.06.0
N/feedstock_root/build_artefacts/ocaml_1523031921566/work/ocaml-4.06.0/bytecomp@

Are these traces of the file system used to compile the code likely to be a problem? (The term "feedstock" is part of the vocabulary used by the conda-forge build system)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.