Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Incremental compilation #4
Point of contact
The goal of the project is to allow the compiler to reuse results from previous compilations. The overall approach is that we dynamically track what data it has accessed to form a dependency graph. You can find more details on the current approach in RFC 1298 -- however, we are also contemplating an architecture shift that has yet to be written up in RFC form. The best notes, perhaps, can be found in the etherpad from the Paris 2017 design sprint.
The newer approach
The critical change in the new design is to be more reluctant to invalidate. Instead of eagerly invalidating everything that might have changed, we invalidate things when we see that a direct input as changed, and this invalidation propagates through the graph. This can result in more re-use, because we may have a node whose inputs have changed but which, when executed, still recomputes the same value (this could happen if, for example, it does not depend on the particular part of the input which changed, but only on other parts).
This algorithm integrates with the "on-demand" processing mechanism that we are phasing in. To describe the algorithm, we will use a series of three colors on the dependency graph: grey, red, and green. Grey is the initial color, and indicates that the node was loaded from a previous session and may or may have a valid value associated with it. Execution happens by a series of queries, which request values (e.g., "compute the
In both cases, we can now obtain the correct value for this particular session (which may or may not have been recomputed). If all inputs to our task come back as Green, then our saved value is still valid and we can re-use it (and color ourselves Green). If at any point we find that any of our inputs come back as red, then we can know that we have to recompute the value we are looking for (e.g., "compute the
The following issues represent "scenarios that we want to work". In some cases, these issues will contain notes on the things that block them from "working" to our satisfaction yet.
We are currently in the "beta period" for the current code. In particular, we are now able -- within a single crate -- to skip the translation into LLVM IR and the optimization phases by re-using object code. There are currently several major goals for moving past this point. Each is listed below, along with issues that track major milestones in that direction: