Support -incremental mode

[allevato]

We may be able to do this with a Bazel persistent worker, which would let us non-hermetically collect the outputs of a library's previous compilations and feed them back in as inputs to future compilations, so that the driver can detect whether they have changed.

However, this is more complicated than simply flipping on a flag—we'll need to design and build a local server that is used as the worker process.

[keith]

Are there any related issues elsewhere about supporting the Swift-style incremental compilation model? It sounds like kotlin and rust might also benefit from support for this in bazel, although I'm not 100% sure they use exactly the same model.

[pauldraper]

bazel-discuss using a stateful compiler with Bazel

rules_scala Should we re-consider zinc's incremental compilation?

I helped author https://github.com/andyscott/rules_scala_annex which implements Zinc (stateful incremental Scala compiler). There, the stateful incrementality is an opt-in for workers.

[kastiglione]

Thanks for the references @pauldraper.

[keith]

Posted the first blocker we hit while working on the persistent worker approach here bazelbuild/bazel#6223

[kastiglione]

In bazelbuild/rules_apple#21 (comment), @​dmishe listed two approaches: a non-sandbox approach (persisting incremental artifacts in the file system), and a persistent worker approach. What are the trade-offs between the two alternative options?

[keith]

Posted another blocker we hit bazelbuild/bazel#6230 in theory we could work around this by using a list for building the linker arguments, but being able to combine Args objects seems reasonable to solve this as well.

[pauldraper]

In bazelbuild/rules_apple#21 (comment), @​dmishe listed two approaches: a non-sandbox approach (persisting incremental artifacts in the file system), and a persistent worker approach. What are the trade-offs between the two alternative options?

Both cases are fairly similar to implement, as I imagine(?) the worker will still persist files on disk. The ergonomics of selecting incremental compilation can differ.

Note you will still want to make sure a non-worker, sandboxed, non-incremental fully deterministic method exists.

---

For reference, this is how https://github.com/andyscott/rules_scala_annex works:

If the compiler worker receives a --persistent_dir=<path> startup flag, then it uses that path to stash and retrieve previous state, keyed by the rule's label. This path is resolved relative to the execution root; currently this rules_scala_annex option is not supported with worker sandboxing.

My own use:

• There is a distributed cache
• Devs use --worker_extra_flag=ScalaCompile=--persistent_dir=.bazel-zinc and read from a distributed cache.
• CI does not use --persistent_dir. It reads and writes to the distributed cache.

Devs get rapid incremental builds. CI gets assuredly deterministic builds (plus, the "previous build" is not always clear anyway). Multiples workers on a dev machine share the same analysis store. And devs can use CI's cache because --worker_extra_flag is not part of the cache key.

Philosophically, I view stateful compilation as a worker-ish thing. The purpose of workers is to reuse expensive state from one action to the next -- typically JIT. Stateful compilation is simply additional state that the worker reuses.

[keith]

@pauldraper thanks for all the context here. I noticed that you're using a hidden directory inside the execution root? Is this required? It looks like in my testing it is, and if it isn't hidden bazel wipes it out between builds, is that your understanding as well?

[pauldraper]

I believe it can be any name, so long as it doesn't conflict with existing paths (packages). There wasn't a strong reason to make it hidden.