natdynlink is broken on arm

[vicuna]

Original bug ID: 5049
Reporter: @glondu
Assigned to: @alainfrisch
Status: closed (set by @xavierleroy on 2013-08-31T10:46:23Z)
Resolution: not a bug
Priority: normal
Severity: feature
Platform: ARM
Category: ~DO NOT USE (was: OCaml general)
Monitored by: mehdi "Julien Signoles"

Bug description

Hello,

While rebuilding all Debian packages (armel port) with ocamlopt enabled (using trunk r10396), ssreflect failed to build. The full build log is available at [1].

The following command crashes with a segmentation fault:

/usr/bin/coqc -dump-glob theories/ssreflect.glob -q -R src Ssreflect -R theories Ssreflect theories/ssreflect

It uses (nat)dynlink. The bytecode counterpart (by adding "-byte") doesn't segfault. Generating a Coq toplevel with ssreflect statically linked in (with "coqmktop -opt -o ssrcoq src/ssreflect.cmx"), and then using it (by adding "-image ./ssrcoq" to the command above) doesn't segfault either.

Running ocsigen in native code on arm also segfaults:

root@daribow:/tmp# ocsigen.opt -V
-- Dependencies of ocsigen.ext.staticmod: ocsigen.commandline, camlp4, ocsigen.ext.staticmod
-- Needed: /usr/lib/ocaml/ocsigen/parsecommandline.cmxs, /usr/lib/ocaml/ocsigen/staticmod.cmxs
Loading extension /usr/lib/ocaml/ocsigen/parsecommandline.cmxs
Loading extension /usr/lib/ocaml/ocsigen/staticmod.cmxs
Segmentation fault

These errors don't happen on amd64.

Cheers,

--
Stéphane

Additional information

[1] http://ocaml.debian.net/debian/ocaml3120dev23r10396/failures/ssreflect_1.2%2Bdfsg-4%2B3.12.0%2Bdev23%2B10396%2B1_armel.build

[vicuna]

Comment author: @alainfrisch

Natdynlink is indeed supported only a limited number of platforms. We simply plan to declare ARM not to be in this list (and, accordingly, it will not be installed).

[vicuna]

Comment author: @glondu

What criterion determines whether natdynlink is supported on a given architecture or not? So far, all architectures that support native code generation supported by Debian support natdynlink as well, so I assumed Linux and FreeBSD as a whole did. Could you summarize the technical difficulties? (al least for future reference)

FWIW, armel is currently the third most popular architecture in Debian [1] (after i386 and amd64). I (and I guess many other people) consider it more valuable that e.g. sparc.

[1] http://popcon.debian.org/

[vicuna]

Comment author: @alainfrisch

It turns out that natdynlink is (was) compiled on all platforms that support dynamic linking, but it was never supported on all these platforms.

The problem is that there are also some constraints on the code generated by ocamlopt in order to have a working natdynlink.

I don't think anyone on the caml development team is going to spend energy trying to make natdynlink work for arm, but if you can identify the problem and submit a patch, it will most likely be considered for inclusion.

Would it make things simpler for you if a version of natdynlink were always installed, even when not functional (trying to use it would raise a clean runtime exception)? As opposed to, say, not install dynlink.cmxa at all.

[vicuna]

Comment author: @xavierleroy

For the record, here are the conditions to be met by a platform for natdynlink to work:

• Either non-PIC code can be put in shared libraries. Then, the ocamlopt code generator need no changes. This is the case for x86-32 bits under Linux, at least. (Recent versions of MacOS X broke this. Under Windows, this works thanks to FlexDLL magic.)

• Or the ocamlopt code generator was changed specifically to generate PIC code. This is the case for x86-64 bits, where the changes and the performance hit were minimal.

• Or the ocamlopt code generator just happens to generate PIC code. I don't think this is the case of any of the other code gens, but Alpha might come very close.

A quick look at ARM code generated by gcc -fPIC suggests that significant work is needed to make ocamlopt produce PIC code on this platform, and the run-time overhead of PIC code is large. So, don't expect a working natdynlink on ARM any time soon.

FWIW, armel is currently the third most popular architecture in Debian [1] (after i386 and amd64).

... at 1% of the installed base...

I (and I guess many other people) consider it more valuable that e.g. sparc.

I certainly agree with you on that; witness my efforts to switch ARM to EABI. That doesn't mean that making natdynlinked-ssreflect work on ARM is a priority.

[vicuna]

Comment author: @glondu

frish wrote:

Would it make things simpler for you if a version of natdynlink were always installed, even when not functional (trying to use it would raise a clean runtime exception)? As opposed to, say, not install dynlink.cmxa at all.

This should probably be asked to a wider audience...

My personal opinion as a programmer would be not to install dynlink.cmxa... as in the OCaml spirit: prefer compile-time errors (such as inexistent dynlink.cmxa) rather than runtime errors. Either way, software relying on dynlink will have to deal with situations where there is ocamlopt, but no natdynlink (the possibilities I see are providing a static counterpart or falling back to bytecode, the latter being the simplest one).

From a maintainer PoV, installing dynlink.cmxa and failing at run-time could have worked if dynlink was never used during the build process... but this approach seems fragile and not very satisfactory.

xleroy wrote:

the run-time overhead of PIC code is large. So, don't expect a working natdynlink on ARM any time soon.

Does that mean that a patch for this would be rejected, as opposed to Alain's invitation to provide such a patch? I don't say that I would write such a patch myself, but other people might have interest...

... at 1% of the installed base...

I didn't hide it :-)

witness my efforts to switch ARM to EABI

Of course I do, and thank you for that.

[vicuna]

Comment author: Julien Signoles

=====
Glondu wrote:
frish wrote:

Would it make things simpler for you if a version of natdynlink were always installed, even when not functional (trying to use it would raise a clean runtime exception)? As opposed to, say, not install dynlink.cmxa at all.

This should probably be asked to a wider audience...

I agree with Stéphane: it is better to have no dynlink.cmxa that a broken one.

Currently the 3 situations already occur anyway:

• no dynlink.cmxa (architectures that do not support natdynlink)
• a broken dynlink.cmxa generating exceptions at runtime (some versions of Mac OS X at least)
• a working dynlink.cmxa

Thus we have to deal with all of these (that is what Frama-C did for instance).

[vicuna]

Comment author: @glondu

I forgot this one:

• Or the ocamlopt code generator just happens to generate PIC code. I don't think this is the case of any of the other code gens, but Alpha might come very close.

As far as Debian is concerned, ocamlopt is not installed on alpha, but it is on powerpc and sparc and at least ssreflect happens to work there.

[vicuna]

Comment author: @alainfrisch

Either way, software relying on dynlink will have to deal with situations where there is ocamlopt, but no natdynlink (the possibilities I see are providing a static counterpart or falling back to bytecode, the latter being the simplest one).

I believe it is not uncommon to have applications that depends on dynlink only when the user explicitly ask to load some kind of addins, but are still fully functional without dynlink. Having the natdynlink module always available with a way to check whether it works would simplify the code organization and build system for these programs. It is always possible to check the configuration statically if needed (there will be a NATDYNLINK variable in the OCaml config file). There are still cases where dynlink.cmxa is not installed, but this could easily be addressed.

I'm still not 100% sure about the right solution.

[vicuna]

Comment author: Julien Signoles

Anyway if you want to be compatible with old versions of ocaml (< 3.11) as well as many architectures, you have to handle the case where dynlink.cmxa is not installed. Furthermore you have to handle the incompatibility of Dynlink interface.

Thus, from a developer point of view, I believe that the simplest way to handle all the possible behaviors is to write an unique front-end to Dynlink and to statically generate the implementation according to the configuration. That a work done by any reasonable build system.

But it is still better to have no dynlink.cmxa that a broken one since the error is detected earlier (as Stéphane already said). Moreover, for a build system, it is easier to detect a missing file that a buggy one whatever is the ocaml config file/version.

Above all, I believe the most important think is to be consistent: either you provide dynlink.cmxa on all architectures with an easy and documented way to check whether it is working, either you don't provide it on architectures without an usable natdynlink.

[vicuna]

Comment author: @alainfrisch

Ok, you convinced me. dynlink.cmxa (and related files) are no longer installed when native dynlink is not supported.

The list of platforms where natdynlink is supported is given in the configure script. Currently, the list is quite conservative, but it should be extended with feedback from users who can test natdynlink on other platforms:

case "$host" in
--cygwin*) natdynlink=true;;
i[3456]86--linux) natdynlink=true;;
x86_64--linux) natdynlink=true;;
esac

[vicuna]

Comment author: mehdi

Maybe I'm not too late here… But, I think that providing dynlink.cmxa on all architectures is a better approach for the following reasons:

• It allows to have the same code to compile on any architecture. Otherwise, if we do use natdynlink in some application, we have to modify the code before compiling depending on natdynlink-less native architectures.

• "unix.cmxa" is another example where you have a completely broken module on some architectures (e.g. Windows). Nonetheless, it's installed on Windows but fails gracefully with an exception.

It would be really nice to have a natdynlink module even when dynlink is broken and raise an exception when broken (and even add a "val has_dynlink : bool" (like the test "is_native") to exploit constant inlining…).

... at 1% of the installed base...
Keep in mind that on this kind of architecture, popcon is usually disabled (s390 is also very under estimated).

[vicuna]

Comment author: @alainfrisch

I'm afraid you're too late, I've already reverted the changes that implemented a Dynlink.is_supported flag, and I don't feel to re-revert it, unless there is some overwhelming popular demand.

(As a side node, I wouldn't say Unix is completely broken on Windows: a lot of its functions work pretty-well.)

[vicuna]

Comment author: @alainfrisch

Natdynlink is indeed supported only on some specific platforms, which does not include ARM. Third party packages should be prepared to deal with other platforms and degrade gracefully.

[vicuna]

Comment author: meurer

Natdynlink should now work for ARM starting with revision 12124 (trunk).