[Experimental] Project Skymeld: Merging Analysis and Execution Phases of Skyframe

[joeleba]

Description of the problem / feature request:

In a regular build, Bazel loads and analyzes the target patterns with Skyframe to form the ActionGraph, this is the Loading-and-Analysis phase. It then performs some extra-Skyframe setup before commencing executing the actions, or the Execution Phase.

Our hypothesis: by allow interleaving the loading/analysis and execution phases, we could improve the build performance, especially for multi-target builds.

By removing the barrier between the phases, we allow targets which have finished analyzing to immediately start with execution. There tends to be many dormant threads towards the end of the analysis phase, and we could make use of those resources for the execution phase.

More details to follow.

[joeleba]

A quick progress update:

1. "Merging Analysis and Execution Phases of Skyframe" is a mouthful so we renamed the project to "Skymeld". If you see Skymeld in the code, it's referring to this project.

2. The prototype is now basically able to handle build and test commands with a single --package_path. Things that work:

• Basic build and test commands with a single --package_path.
• Action conflict checking
• Action cache
• Error Reporting (sort of, we still need to redefine the meaning of exceptions where applicable)

Things that still need work/verification (non-exhaustive):

• Dynamic/Remote execution
• Progress reporting
• Profiling
• Resource management

[saraadams]

If I understood correctly at your BazelCon talk (thanks!), 6.0 will include this, guarded by an experimental flag. Can you share the flag(s) to set to enable this? The command-line reference for 6.0 isn't up yet, but I'd like to give it a try.

I found https://bazel.build/reference/command-line-reference#flag--experimental_skymeld_ui - but the documentation suggests this only updates the terminal output. Or does this flag have the undocumented side-effect of also enabling skymeld?

[benjaminp]

--experimental_merged_skyframe_analysis_execution

[joeleba]

Hi everyone,

Exciting news! Skymeld has reached a certain level of maturity that we're now comfortable calling for its dogfooding in Bazel!

How to dogfood

• Run bazel from HEAD, or with a rolling release (this is mostly so that you receive the latest fixes), and
• Add --experimental_merged_skyframe_analysis_execution to your build and test commands.

Measuring impacts

Skymeld is expected to improve the end-to-end wall time of multi-target builds with remote execution. All the wall time wins should come from the analysis phase time.

It's best if you already have your own mechanism to track build performance. Using bazel-bench to benchmark your builds also provides a good estimate. To ensure a controlled environment, we recommend using a dedicated machine for this.

We're also very interested in any performance wins or issues (e.g. OOM) that you encounter.

Bug Reporting

Please file a bug with:

• the tag [skymeld-dogfood] in the bug title, and
• mention @joeleba

Excited to fix all the incoming bugs. Thanks in advance, adventurous dogfooders!

Edit 2023.07.24: remove the reference to --experimental_skymeld_ui (now no-op).

[joeleba]

--experimental_skymeld_ui is now no-op and will be removed. Please remove it from your rc files if you have it enabled.