Skip to content

Commit

Permalink
Rigorously handle -o and -c options in presence of multiple arguments.
Browse files Browse the repository at this point in the history
This addresses PR#6475.

In 4.02 the behavior of ocamlc/ocamlopt with regards to these options
was as follows:
  * options and arguments are parsed left-to-right in the exact order
    in which they are passed, with compilation taking into account
    only the options leftwards from it;
  * "foo.c" is compiled to "foo.o" in current directory;
  * when "-c" is not specified:
    * "foo.ml" is compiled to "foo.cmo"/"foo.cmxo"
      in current directory;
    * after all files have been compiled, if any .ml files are passed,
      all provided files are linked as:
      * when "-o" is not specified: "a.out" in current directory;
      * when "-o out" is specified: "out".
  * when "-c" is specified:
    * "foo.ml" is compiled to:
      * when "-o" is not specified: "foo.cmo"/"foo.cmxo"
        in current directory;
      * when "-o out" is specified: "out.cmo"/"out.cmxo";
        and then compilation proceeds as if the last "-o" option
        has disappeared.
    * no final link is performed.

The behavior where the build product of the C sources always ended up
in the current directory was problematic: it required buildsystem
hacks to move the file in its proper place and ultimately was racy,
as multiple files with the same basename in different directories
may well end up overwriting each other with e.g. ocamlbuild.

On top of that, the behavior was quite confusing, since it is not
only stateful and dependent on argument order, but also the mere act
of compilation changed state.

The commit 1d8e590 has attempted to rectify that by looking at
the "-o" option when compiling C files, too. After that commit,
the behavior of ocamlc/ocamlopt was as follows (only the handling
of C files was changed, but the entire chart is provided for
posterity):
  * options and arguments are parsed left-to-right in the exact order
    in which they are passed, with compilation taking into account
    only the options leftwards from it;
  * "foo.c" is compiled to:
    * when "-o" is not specified: "foo.o" in current directory;
    * when "-o out" is specified: "out".
  * when "-c" is not specified:
    * "foo.ml" is compiled to "foo.cmo"/"foo.cmxo"
      in current directory;
    * after all files have been compiled, if any .ml files are passed,
      all provided files are linked as:
      * when "-o" is not specified: "a.out" in current directory;
      * when "-o out" is specified: "out".
  * when "-c" is specified:
    * "foo.ml" is compiled to:
      * when "-o" is not specified: "foo.cmo"/"foo.cmxo"
        in current directory;
      * when "-o out" is specified: "out.cmo"/"out.cmxo";
        and then compilation proceeds as if the last "-o" option
        has disappeared.
    * no final link is performed.

There is a non-obvious bug here. Specifically, what happens if more
than one C source file is passed together with a "-o" option? Also,
what happens if a C source file is passed together with a "-o" option
and then a final link is performed? The answer is that
the intermediate build product gets silently overwritten, with quite
opaque errors as a result.

There is some code (and even buildsystems) in the wild that is relying
on the fact that the -o option does not affect compilation of C source
files, e.g. by running commands such as (from ocamlnet):
  ocamlc -custom -o t tend.c t.ml

It might seem that the solution would be to make the behavior of
the compiler drivers for C files match that for the OCaml files;
specifically, pretend that the "-o" option has disappeared once
the C compiler has written a build product to the specified place.

However, this would still break the existing code, and moreover
does not address the underlying problem: that the option parsing
of the OCaml compiler driver is confusing and prone to creating
latent bugs.

Instead, this commit finishes (after 1d8e590 and 55d2d42) overhauls
the way option parsing in ocamlc/ocamlopt works to behave as follows:
  * options are parsed left-to-right in the order they are specified;
  * after all options are parsed, arguments are parsed left-to-right
    in the order they were specified;
  * when "-o out" and "-c" are specified:
    * when more than one file is passed, an error message
      is displayed.
    * when one file is passed:
      * "foo.c" is compiled to "out";
      * "foo.ml" is compiled to "out.cmo"/"out.cmxo".
  * when "-o out" is not specified or "-c" is not specified:
    * "foo.c" is compiled to "foo.o" in current directory;
    * "foo.ml" is compiled to "foo.cmo"/"foo.cmxo"
      in current directory;
  * when "-c" is not specified:
    * after all files have been compiled, if any .ml files are passed,
      all provided files are linked as:
      * when "-o" is not specified: "a.out" in current directory;
      * when "-o out" is specified: "out".

In short, the combination of "-o", "-c" and a single source file
compiles that one file to the corresponding intermediate
build product. Otherwise, passing "-o" will either set the name of
the final build product only or error out.

This preserves compatibility with old code, makes the handling of
C and OCaml sources consistent, and overall makes the behavior
of the option parser more straightforward. However, this may break
code that relies on the fact that options are parsed in-order, e.g.
  ocamlc -o t a.ml -g b.ml
where debug info would be built only for "b.ml".

Some alternative implementation paths I have considered:
  * Collect the C sources and process them after OCaml sources,
    while paying attention to any "-o" or "-c" that may have
    been set. This doesn't work because compilation of C sources
    is also affected by many flags, e.g. "-I", and so this would
    have the same drawbacks but none of the benefits;
  * Compile C and OCaml sources in-order as usual, but error out
    when an improper combination of flags is encountered in
    the middle of a compilation. This is technically feasible,
    and is the option that maximally preserves compatibility, but
    it is very complex: it doubles the amount of implicitly mutated
    global state, and there's no guarantee I will get all edge
    cases right. Moreover, the option parsing remains confusing,
    and I strongly believe that the current behavior should not
    remain in place.

On top of that it is hard to imagine cases where setting new options
in the middle of compilation would actually be desirable, because
this mechanism is very inexpressive: it can only add new options and
option values, since there is no way to negate or clear most of
the driver's state. Most likely is that any code that does so,
does it in error and remains operational by pure chance.
  • Loading branch information
whitequark authored and gasche committed Jun 14, 2016
1 parent 4dc3efe commit da56cf6
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 0 deletions.
9 changes: 9 additions & 0 deletions driver/main.ml
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,16 @@ let main () =
try
readenv ppf Before_args;
Arg.parse Options.list anonymous usage;
if !output_name <> None && !compile_only &&
List.length !process_thunks > 1 then
fatal "Options -c -o are incompatible with compiling multiple files";
let final_output_name = !output_name in
if !output_name <> None && not !compile_only then
(* We're invoked like: ocamlc -o foo bar.c baz.ml.
Make sure the intermediate products don't clash with the final one. *)
output_name := None;
List.iter (fun f -> f ()) (List.rev !process_thunks);
output_name := final_output_name;
readenv ppf Before_link;
if
List.length (List.filter (fun x -> !x)
Expand Down
9 changes: 9 additions & 0 deletions driver/optmain.ml
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,16 @@ let main () =
try
readenv ppf Before_args;
Arg.parse (Arch.command_line_options @ Options.list) anonymous usage;
if !output_name <> None && !compile_only &&
List.length !process_thunks > 1 then
fatal "Options -c -o are incompatible with compiling multiple files";
let final_output_name = !output_name in
if !output_name <> None && not !compile_only then
(* We're invoked like: ocamlopt -o foo bar.c baz.ml.
Make sure the intermediate products don't clash with the final one. *)
output_name := None;
List.iter (fun f -> f ()) (List.rev !process_thunks);
output_name := final_output_name;
readenv ppf Before_link;
if
List.length (List.filter (fun x -> !x)
Expand Down

0 comments on commit da56cf6

Please sign in to comment.