add autocompile support, controlled via --compile (default) and --no-compile

[timbertson]

Discussed on the mailing list a little already, but this PR may be easier to comment / track changes.

[talex5]

Just to record a few of the things we discussed so we don't forget:

• need to support autocompile in GUI mode
• think about whether --autocompile should be the default (e.g. should existing updates doing background updates start to compile?)
• fix unit-tests
• document changes to the selections XML format (not sure mode is a good name; maybe requires_compilation=true or something?)
• consider using `binary_of source impl_type
• update 0compile to support this

[timbertson]

think about whether --autocompile should be the default (e.g. should existing updates doing background updates start to compile?)

I feel strongly that it should be the default, even in the background. My main intention in adding auto compilation is to be able to distribute source code and have it be usable (including updates).

Making it explicitly opt-in is not much more useful than just running 0compile explicitly, which is sufficiently awkward that nobody does it.

fix unit-tests

I've now fixed the compilation errors, and fixed the trivial selection test case failures. But I'm getting a bunch of test failures like:

Error: 0install:13:selections:2:commands

Fake_system.Would_spawn(_)
Raised at file "tests/fake_system.ml", line 256, characters 19-57

Not really sure what's going on there, as it doesn't seem like the backtrace touches any of my new code (and I'm finding oUnit a little obtuse). I do spawn 0compile when we end up with selections that need compilation, but I wouldn't expect any existing test cases to actually trigger that. Any hints?

[talex5]

I've added formatting for these exceptions on master. If you rebase you'll get e.g. (from test_run.ml):

Error: 0install:23:run:2:abs-main

Test tried to spawn process /home/tal/Projects/zero-install/0install/build/ocaml/0install download --refresh --console -v --command run /home/tal/Projects/zero-install/0install/tests/Command.xml

Perhaps your patch makes it decide to try recompiling it instead of using the older binary?

[timbertson]

Thanks. I rebased, and you're right, that does seem to be the reason for the failures. e.g:

==============================================================================
Error: 0install:23:run:2:abs-main

Test tried to spawn process /home/tim/dev/0install/zeroinstall/build/ocaml/0install download --refresh --console -v --command run /home/tim/dev/0install/zeroinstall/tests/Command.xml
Raised at file "tests/fake_system.ml", line 265, characters 19-57

I'm haven't been able out how my changes would affect that though.

[talex5]

I've been wondering if we should break this work up into smaller pieces:

1. Clean up the current mess in impl_provider.ml
2. Implement the current 0compile autocompile behaviour in the OCaml solver, so that 0compile can use that instead of its current hack.
3. Improve the selections (e.g. to include build dependencies in the selections).
4. Add a command-line interface for compiling apps/interfaces.
5. Make autocompiling apps the default.

I've made a start on 1 and 2 here: https://github.com/0install/0install/commits/autocompile

Currently, if you set may_compile = true in the scope filter then it:

1. Includes source feeds even when looking for binaries.
2. Converts every source impl it finds into a binary.

It doesn't yet implement the 0compile algorithm (e.g. adding commands and binary dependencies). Does this look like a reasonable direction?

3/4/5 are the features from your patches that aren't currently in mine.

[timbertson]

Yeah, this definitely seems like a reasonable approach to me :)

[talex5]

OK, 1 and 2 are now done and merged.
I don't think 3 is a good idea, because it greatly increases the number of feeds you need to download to include the build tools and headers for every potential binary, recursively. So, I now just assume every source implementation can be compiled, which is what 0compile does already.