Whole program dead code elimination

[chambart]

I propose in this PR to add another link mode to the ocamlopt compiler with flambda enabled.

There is a new -lto option to the compiler to mark when a file should export sufficient information for the link. When all the .cmx and .cmxa files of a project are built with this option, it is possible to link with it too. When linking, all the flambda informations are concatenated and go through a dead code elimination pass that removes all unreferenced symbols (but keep effets) and build a new big object file containing the whole program.
Of course, this prevent using dynlink in this program as the modules referenced by the loaded module might have been eliminated. It may be possible to provide a mode where some interfaces are requested, and only those would be available for a dynlinked module but this is not implemented yet.
There are no optimization specific to whole program applied yet.

Outside of the undynlinkability, the other drawbacks are:

• cmxa files are really larger. (stdlib goes from 15kB to 1.9MB). Those files didn't contain much,
  for this mode, they must contain the whole code of the included modules since the compiler
  might not have access to the cmx files while linking.
• cmx files are not that much bigger (less than 10% on the stdlib), probably quite more in -Oclassic

Overall compilation time does not change significantly, but link time of course increases a lot:

without -lto

$ time ./ocamlopt.opt -dtimings -g -nostdlib -I stdlib -I otherlibs/dynlink  -ccopt "-Wl,-E" -o ocamlc.opt   compilerlibs/ocamlcommon.cmxa compilerlibs/ocamlbytecomp.cmxa   driver/main.cmx -cclib "-lm  -ldl -lcurses -lpthread" 

real    0m0.565s
user    0m0.464s
sys 0m0.088s

with -lto

$ time ./ocamlopt.opt -dtimings -g -nostdlib -I stdlib -I otherlibs/dynlink  -ccopt "-Wl,-E" -lto -o ocamlc.opt   compilerlibs/ocamlcommon.cmxa compilerlibs/ocamlbytecomp.cmxa   driver/main.cmx -cclib "-lm  -ldl -lcurses -lpthread"

real    0m8.243s
user    0m7.912s
sys 0m0.316s

all: 5.752s
flambda(concatenate)(link): 0.096s
flambda(remove_unused_program_constructs)(link): 0.512s
flambda(backend)(link): 0.452s
cmm(link): 0.604s
compile_phrases(link): 2.992s
selection(link): 0.228s
comballoc(link): 0.020s
cse(link): 0.180s
deadcode(link): 0.088s
spill(link): 0.312s
split(link): 0.136s
liveness(link): 0.264s
regalloc(link): 1.088s
linearize(link): 0.024s
scheduling(link): 0.004s
emit(link): 0.400s
assemble(link): 0.008s

The effect is not wonderful on the compiler itself: the size of ocamlc.opt decreases by ~10% (there is not much dead code there), but on some extreme examples we can get quite a lot. There are still some cases where this does not eliminate as much as expected.

This patch is based on #602 only the commits after 88c2c8c (Also remove linking hack for bytecode) are relevant. This other PR is needed to allow removing unneeded toplevel modules.

Note the -lto or 'link time optimization' is quite badly named. Please suggest a better option.

[let-def]

Rather than making cmxa largely redundant with cmx files, couldn't we get rid of cmxa files (by e.g, allowing to have link options in cmx files)?
I am not asking for this to be mandatory, that would break compatibility very badly. But maybe just to make this workflow viable, now that cmx files are needed for flambda to operate properly.

[lpw25]

@let-def I agree and have a patch that does this as part of a larger "namespaces" proposal.

[xavierleroy]

Concerning @let-def's suggestion, it's exactly what Caml Light did back in the days. Then, someone objected on the basis of the following scenario: you have a library "foo" containing two modules, "foo_aux" and "foo". With the proposed approach, you first compile foo.ml, obtaining foo.cmx, then build the library foo.cmx, overwriting the previous foo.cmx file...

Also, .cmx files describe .o files while .cmxa files describe .a files. What are you proposing? .cmx files that describe .a files? Get rid of .a files?

[let-def]

Yes, offer a workflow without .a files. Libraries add one more level of names, I would like to be able to do without if possible.
The linker could receive all objects (with existing tools, it's mostly a matter of putting all .cmx files in the META description).

[lpw25]

I had assumed @let-def was suggesting allowing to use a directory filled with .cmx files in place of a .cmxa. You already need such a directory for cross-module inlining so why not use it for linking.

[samoht]

@chambart I'm curious to see the gain that this could have on some MirageOS binaries. Is there a way to turn -lto by default on the whole switch (and if yes, could this be added as a new experimental opam compiler?)

[chambart]

@let-def why not, but this probably does not fit in this PR (that is already large enouth). Currently beside link information, there is also no way to tell using only cmx files that a file should be linked only if used. This matter if there is some initialization code. We could of course also add a flag to the cmx files to tell something like that.

@samoht you can use export OCAMLPARAM=lto=1,_ or use something similar to the the hack in: ocaml/opam-repository#6115 (which by the way was not propagated to the current default 4.03 compiler as I just noticed...)

[chambart]

The last patch adds a minimal optimization round when linking to allow to remove some more references to toplevel modules in situations containing something like:

module A

initialize_symbol a (call ...)
initialize_symbol camlA (a.(0))

module B

let v = A.camlA.(0)

When concatenating A and B, B still maintain a reference to camlA but it could have been redirected to A.a. This usually does not matter since reaching the value in a field of a symbol or another has the same performance cost, hence there is no information propagated in the cmx file to know that A.camlA.(0) is an alias of A.a.(0).

It now matters. This alias is something that inline_and_simplify knows about, so running it after concatenation allows to remove all references to camlA.

In practice that can appear for instance if we use stdin that force the whole pervasive module to stay alive. A quite extreme example, with this pass and sufficiently aggressive inlining option (-O3 -inline-max-unroll 4), the complete code of the program:

Printf.printf "Hello world %i\n" 3

Is in clambda (this is after un-anf, hence constants are not shown)

un-anf (init_code):
(seq
  (caml_register_named_value "camlPervasives__Pccall_arg_1389"
    "camlPervasives__Pmakeblock_1388")
  (caml_ml_open_descriptor_in 0)
  (setfield_ptr(init) 0 "camlPervasives__Pccall_1585"
    (caml_ml_open_descriptor_out 1))
  (caml_ml_open_descriptor_out 2)
  (setfield_ptr(init) 0 "camlPervasives__exit_function_1549"
    (makemutable 0 "camlPervasives__flush_all_582_closure"))
  (caml_register_named_value "camlPervasives__Pccall_arg_1460"
    "camlPervasives__do_at_exit_1213_closure")
  (setfield_ptr(init) 0 "camlTest__Pccall_282"
    (caml_format_int "camlCamlinternalFormat__const_string_10802" 3))
  (caml_ml_output (field 0 "camlPervasives__Pccall_1585")
    "camlTest__Pmakeblock_arg_34" 0 12)
  (setfield_ptr(init) 0 "camlTest__Pccall_arg_280"
    (string.length (field 0 "camlTest__Pccall_282")))
  (caml_ml_output (field 0 "camlPervasives__Pccall_1585")
    (field 0 "camlTest__Pccall_282") 0 (field 0 "camlTest__Pccall_arg_280"))
  (caml_ml_output_char (field 0 "camlPervasives__Pccall_1585") 10)
  (setfield_ptr(init) 0 "camlStd_exit__simplify_fv_22"
    (field 0 (field 0 "camlPervasives__exit_function_1549")))
  (apply (field 0 "camlStd_exit__simplify_fv_22") 0a) 0a)

un-anf (camlPervasives__iter_586):
(if param_588/1262
  (let
    (sequence_591/1264
       (try (caml_ml_flush (field 0 param_588/1262)) with exn_592/1267 0a))
    (apply* camlPervasives__iter_586  (field 1 param_588/1262)))
  0a)


un-anf (camlPervasives__do_at_exit_1213):
(apply (field 0 (field 0 "camlPervasives__exit_function_1549")) 0a)

un-anf (camlPervasives__flush_all_582):
(apply* camlPervasives__iter_586  (caml_ml_out_channels_list 0a))

and the constants are

Pervasives.camlPervasives__Pmakeblock_arg_1387: "index out of bounds"
CamlinternalFormat.camlCamlinternalFormat__const_string_10802: "%i"

Pervasives.camlPervasives__Pccall_arg_1389: "Pervasives.array_bound_error"

Test.camlTest__Pmakeblock_arg_34: "Hello world "

Pervasives.camlPervasives__Pccall_arg_1460: "Pervasives.do_at_exit"

Pervasives.camlPervasives__Pmakeblock_1388: block(0,"caml_exn_Invalid_argument","camlPervasives__Pmakeblock_arg_1387")

Notice that the majority of the code is related to do_at_exit

[samoht]

@chambart any chance that you could fix the 4.03 description and add your new compiler to opam-repository? :-) (it not, I'll try to do this next week)

[alainfrisch]

Did you consider an actual "whole program optimizer" based on flambda? I can imagine such a compiler loading e.g. .cmt files (produced "quickly" by ocamlc, or by ocamlopt in non-flambda mode) and doing all the compilation+linking globally. The closed world assumption opens many more opportunities for optimizations (including tweaks to the representation of values if they cannot be observed from C). This would support quick compilation (including with ocamlopt) for reasonably fast edit/compile cycle but also local testing, while allowing full optimization mode for building critical executables.

[bluddy]

@alainfrisch I think we were all thinking about this and hoping for it. Whole-program compilation is the 'holy grail' in terms of optimization potential. It would be nice to introduce optimizations here (like type-specialization of functions and optimized type representation, which would provide serious performance boosts) and then slowly let some of them leak out to the open-universe case.

[mshinwell]

@alainfrisch We haven't really thought about this much, but we're intending to spend some amount of time later this year trying to significantly improve compilation speed at Jane Street, and one thing we're considering is stopping the compiler earlier for a "type-check only" mode (to give fast feedback) with delayed "background" output of object files after that. I think this would fit in with what you propose.

I will undertake to review this patch.

[mshinwell]

(Also, I'm not sure "-lto" is badly named. It describes what is going on and is nearly the same as the corresponding GCC option for the same thing.)

[DemiMarie]

Some of the optimizations I would like will require changes to the OCaml GC – specifically, the ability to have blocks in the heap that contain a mixture of pointers and non-pointers. This would allow for floats, int32, int64, and nativeint to be unboxed everywhere, except possibly where alignment constraints for int64 proved problematic and as arguments to polymorphic functions that were not type-specialized.

Even more far-out, if type-specialization makes most functions operate on unboxed, specialized data (such as machine ints, doubles, and pointers), OCaml might benefit from an LLVM backend. But that is a ways off.

[chambart]

@alainfrisch without changing anything else to the toolchain, it is probably sufficient for you to build with ocamlopt -Oclassic -opaque -lto for the majority of the builds (while developing), and only link with -lto -O3 only when releasing or benchmarking.

This would benefit from a reasonably fast build for each file and have the same ability as bytecode not to rebuild the whole world when you change a file. The performance won't be marvelous of course.
And when linking for release, it will be horribly slow and eat a lot of ram, but that shouldn't happen as often.

Currently this patch does not run the optimization passes when linking, but if you consider that workflow useful, I can change that. This require some changes as the passes do not expect symbols declared from different compilation units and will complain about that.

[alainfrisch]

@chambart If I understand correctlly, you claim that most of the overhead of -Oclassic compared to the legacy pipeline is due to cross-module optimizations. Is this only your intuition, or has this been empirically confirmed?

[samoht]

@chambart I'm getting

Fatal error: exception File "asmcomp/closure_offsets.ml", line 99, characters 6-12: Assertion failed

when I am trying your PR (but using OCAMLPARAM="lto=1,_" instead of adding a file in lib/ocaml). Is that expected?

[chambart]

@alainfrisch no, I claim that what you gain from not recompiling everything when you change a given file is bigger than the overhead of flambda. This of course requires that you build system is able to see that the cmx file didn't change.

[chambart]

@samoht no. I'll look into this.
I assume you are getting that when linking. This is probably linked to the fact that the middle-end does not like symbols from different compilation units in the same file. This will probably need a rewriting pass to clean this.

[samoht]

Latest command shown in the logs is:

cd tools; /Applications/Xcode.app/Contents/Developer/usr/bin/make opt.opt
# ../boot/ocamlrun ../ocamlopt -nostdlib -I ../stdlib -absname -w +a-4-9-41-42-44-45-48 -strict-sequence
-warn-error A -safe-string -strict-formats -I ../utils -I ../parsing -I ../typing -I ../bytecomp -I ../asmcomp
-I ../middle_end -I ../middle_end/base_types -I ../driver -I ../toplevel -c - ocamldep.ml
# ../boot/ocamlrun ../ocamlopt -nostdlib -I ../stdlib -I ../utils -I ../parsing -I ../typing -I ../bytecomp -I
../asmcomp -I ../middle_end -I ../middle_end/base_types -I ../driver -I ../toplevel -I .. -o ocamldep.opt
timings.cmx misc.cmx config.cmx identifiable.cmx numbers.cmx arg_helper.cmx clflags.cmx terminfo.cmx
warnings.cmx location.cmx longident.cmx docstrings.cmx syntaxerr.cmx ast_helper.cmx parser.cmx lexer.cmx
parse.cmx ccomp.cmx ast_mapper.cmx ast_iterator.cmx builtin_attributes.cmx ast_invariants.cmx pparse.cmx
compenv.cmx depend.cmx ocamldep.cmx

It's on OSX 10.11 if that makes a difference...

[alainfrisch]

Ok understood. (But in fast dev mode, I already compile with -opaque.)

[chambart]

@samoht if you want to test again, I fixed a few problems

[mor1]

@chambart @samoht per discussion with @samoht just now, I gave the 4.04.0+forced_lto switch a try. Built a simple hello world program fine (325kB vs 484kB with OSX "system" which is 4.03.0) but hit a snag when trying to build mirage-www -- it seems that opam install ocamlbuild fails with

# Error: Modules Ocamlbuild_pack was compiled without the `-lto`
#        option. It is needed for linking with the `-lto` option.

FWIW, possibly related, opam install base-ocamlbuild also fails due to opam constraint:

[ERROR] base-ocamlbuild is not available because it requires OCaml >= 3.10 & < 4.03.

If there's an easy fix / something I can try to get the build going / you can point me to what I should try pinning and building locally, I'll try again later today :)

[chambart]

@mor1, I did not seriously try packs, I wouldn't be surprised that I missed something in the handling of packs.
The base-ocamlbuild constraint comes from the removal of ocamlbuild from the core distribution. Before 4.03 it was inside and the 'fake' package base-ocamlbuild expose it. Since 4.03, it needs to be installed as a real package.

The reason something is failing in ocamlbuild is probably due to it being the deepest package using pack in the package dependency tree.

I'll try to add some tests for packs and lto in the testsuite.

[chambart]

@mor1 I fixed a few things for pack, this should be better now.

By the way, I wanted to try mirage-www, the version on opam repository is not installable on trunk, is there a different repository with the up to date versions ?

[mor1]

Thanks-- I'll give it a try (may not be for a few days as travelling).

I haven't tried mirage-www recently, I'll give that a look too.

Is there an easy way to try out your updates? Do I just use the same switch as before?

[chambart]

Yes opam switch reinstall 4.04.0+forced_lto should do the work (no need to opam update before)

[chambart]

@samoht I just rebased and updated for 4.05.

I think otherwise the status is still the same: We need some real world test to validate.

[dbuenzli]

So I tried this in the context of uucp where people have been complaining about executable size (see this issue). On this program:

let () =
  Printf.printf "%b" (Uucp.White.is_white_space (Uchar.of_int 0x0020));
  ()

The results are as follows:

9.3M	test.native # Without -use-lto
412K	test.native # With -use-lto

[dbuenzli]

open Base
let () =
  print_endline "Hello Warld!";
  ()

Here's a hello world base:

3.1M	test.native # Without -use-lto
672K	test.native # With -use-lto

[mshinwell]

I tried this on a large executable which measured 220Mb in size when compiled with 4.03+flambda. Using 4.04+flambda with LTO enabled the executable reduced to 105Mb in size. It took more than ten minutes to link, which seems excessive. The ocamlopt.opt memory consumption which I think was around 4Gb is probably more reasonable as there is a lot of code involved here.

I have not yet investigated whether there are further opportunities for dead code elimination. I was hoping for a larger reduction in size, so there may be something about the code that prevents it. I will try to build it in bytecode so I can see what reduction is obtained by ocamlclean.

Examination of the 220Mb executable has also revealed two problems with ELF string tables. These totalled 80Mb (!). Firstly, especially when built without dynlink support, then we shouldn't be having all of the symbols in the dynamic symbol table as well as the normal one. I think the ELF "hidden" visibility support may fix this; I will investigate and submit a pull request. Secondly we shouldn't be generating such verbose symbol names; some of them may also point at duplicate copies of code. We will look at this in due course.

[samoht]

Any chance to update these patches to 4.06 and/or 4.07?

[damiendoligez]

You mean 4.08, right?

[samoht]

Or 4.08 indeed. But just having an opam switch for 4.06+lto and/or 4.07+lto would already be great :-)

[samoht]

So just to report some numbers for that PR, when compiling the hello world unikernel (and using OCAMLPARAM=use-lto=1,_):

For a Unix application:

$ mirage configure --target=unix && make && upx --best _build/main.exe && du -h _build/main.exe
[...]
584K	_build/main.native

For a xen (self-contained) virtual machine image:

$ mirage configure --target=xen && make && strip hello.xen && du -h hello.xen
[...]
832K	hello.xen

But there is an issue with the solo5 backends, as we are cross-compiling the runtime (using ocaml-freestanding) and we are just installing the new .a and .o files:

$ mirage configure --target=ukvm && make
[...]
ocamlfind ocamlopt -g -dontlink unix -dontlink str -dontlink num -dontlink threads -linkpkg -output-obj -package mirage-types-lwt -package mirage-types -package mirage-solo5 -package mirage-runtime -package mirage-logs -package mirage-clock-freestanding -package mirage-bootvar-solo5 -package lwt -package functoria-runtime -package duration -predicates mirage_solo5 key_gen.cmx unikernel.cmx main.cmx -o main.native.o
mirage: [WARNING] using ld as ld (pkg-config solo5-kernel-ukvm --variable=ld)
/home/samoht/.opam/lto2/lib/pkgconfig/../../lib/ocaml-freestanding/libotherlibs.a(bigarray_stubs.o): In function `caml_ba_hash':
bigarray_stubs.c:(.text+0x61d): undefined reference to `caml_hash_mix_float'
bigarray_stubs.c:(.text+0x665): undefined reference to `caml_hash_mix_double'
bigarray_stubs.c:(.text+0x6cc): undefined reference to `caml_hash_mix_uint32'
bigarray_stubs.c:(.text+0x73c): undefined reference to `caml_hash_mix_uint32'
bigarray_stubs.c:(.text+0x751): undefined reference to `caml_hash_mix_uint32'
bigarray_stubs.c:(.text+0x78b): undefined reference to `caml_hash_mix_int64'
bigarray_stubs.c:(.text+0x7c3): undefined reference to `caml_hash_mix_intnat'
bigarray_stubs.c:(.text+0x7fb): undefined reference to `caml_hash_mix_uint32'
bigarray_stubs.c:(.text+0x836): undefined reference to `caml_hash_mix_uint32'
bigarray_stubs.c:(.text+0x860): undefined reference to `caml_hash_mix_float'
run ['ld' '-nostdlib' '-z' 'max-page-size=0x1000' '-static' '-T'
     '/home/samoht/.opam/lto2/lib/pkgconfig/../../lib/solo5-kernel-ukvm/solo5.lds'
     '/home/samoht/.opam/lto2/lib/pkgconfig/../../lib/solo5-kernel-ukvm/solo5.o'
     '_build/main.native.o'
     '/home/samoht/.opam/lto2/share/pkgconfig/../../lib/mirage-solo5/libmirage-solo5_bindings.a'
     '/home/samoht/.opam/lto2/lib/pkgconfig/../../lib/ocaml-freestanding/libasmrun.a'
     '/home/samoht/.opam/lto2/lib/pkgconfig/../../lib/ocaml-freestanding/libotherlibs.a'
     '/home/samoht/.opam/lto2/lib/pkgconfig/../../lib/ocaml-freestanding/libnolibc.a'
     '/home/samoht/.opam/lto2/lib/pkgconfig/../../lib/ocaml-freestanding/libopenlibm.a'
     '-o' 'hello.ukvm']: exited with 1

So the numebrs are really great, but I am not sure how to fix the last error :-)

[samoht]

And the port of the PR to 4.06.1 is done here: https://github.com/well-typed-lightbulbs/ocaml-esp32/tree/4.06.1+lto (thanks to @TheLortex).

4.06.1+lto is now in opam

[copy]

A game server with containers, lwt and a few more dependencies on 4.06.1+lto:

Native with use-lto: 11M
Native stripped with use-lto: 7.3M
Native without use-lto: 23M
Native stripped without use-lto: 16M
Bytecode executable: 20M
ocamlcleaned bytecode executable: 2.2M

Looks like more optimisations are possible, but the reduction in size is already quite significant. Would love to see this being moved forward.

Compilation time increased from less than a second to 8 seconds, while ocamlclean crunched the bytecode executable for 70 seconds.

On another binary that usually weights 41M I got a stackoverflow.

[DemiMarie]

@copy you can increase the native stack size. Does that prevent the stack overflow?

[copy]

@DemiMarie Indeed that fixed the problem, here's another binary built with async and core:

Native with use-lto: 23M
Native stripped with use-lto: 16M
Native without use-lto: 41M
Native stripped without use-lto: 29M
Bytecode executable: 57M
ocamlcleaned bytecode executable: 4.8M

[sabine]

The notion of whole-program optimization and dead-code elimination seems relevant to compiling to WebAssembly.

Right after I posted that, in order to use unboxed native 32-bit and 64-bit integers on the WebAssembly GC, we might be able to make a linker that does dead-code elimination and code-specialization (to emit precise types for the WebAssembly GC heap), I find this. So, I take this as a data point that suggests that it is possible to do dead-code elimination, and that it is probably feasible to emit WebAssembly modules with additional information to monomorphize (to some extent) and do dead-code elimination at link time.

Edit: looks like monomorphization is actually not possible, but the rest still applies. 😄

[EduardoRFS]

@chambart there was any blocking on this PR? I rebased it to 4.10, got some nice reductions on Mirage with flambda from 3.8mb to 1.3mb.

There is some small performance regressions on mine, mostly because I'm actually running the entire flambda pipeline on the whole program instead of just the cleaning pass.

https://github.com/EduardoRFS/ocaml/tree/4.10+lto