Add stdout parameter to ctx.actions.run - #30408
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
7ec8b60 to
40dc70d
Compare
f6f3427 to
6410728
Compare
Adds a `stdout` parameter to `ctx.actions.run` that accepts a File and
redirects the action's standard output stream into it. The captured
output becomes a regular output artifact of the action, but it is not
reported as regular action stdout: it is not printed to the terminal and
is not included in the build event stream.
The stdout output is exposed to execution via a new `Spawn#getStdout`,
stays part of `Spawn#getOutputFiles`, and is included in the action key
so that moving a file between `outputs` and `stdout` invalidates the
action:
* Local and sandboxed execution redirect the process's stdout file
descriptor directly into the output file instead of the regular
stdout capture buffer, so nothing leaks to the terminal or the BEP.
* Remote execution excludes the output from the REAPI command's
output_paths (the server captures stdout separately) and
reconstructs it from the action result's stdout digest. Unlike
regular stdout/stderr, it is no longer downloaded eagerly: it honors
build-without-the-bytes like any other output (metadata is injected
and it is fetched on demand). Local-then-uploaded actions store the
captured stdout as the action result's stdout digest to keep cache
entries consistent with remotely executed ones.
* Action rewinding works without special handling because the stdout
output is an ordinary output artifact with a generating action key,
so a lost stdout that feeds another action rewinds the producer.
Since persistent workers communicate with Bazel over stdout, combining
`stdout` with the `supports-workers` or `supports-multiplex-workers`
execution requirements is rejected at analysis time.
Only `stdout` is supported; `stderr`/`stdin` are intentionally left out.
RELNOTES[NEW]: `ctx.actions.run` now accepts a `stdout` parameter that
redirects the action's standard output into a declared output file.
Fixes bazelbuild#5511. See also bazelbuild#8275.
6410728 to
1aa04e5
Compare
|
I would like to see a review from @tetromino or @brandjon first |
tetromino
left a comment
There was a problem hiding this comment.
StarlarkActionFactoryApi.java and StarlarkActionFactory.java LGTM. The spawn/bwob changes are a bit outside my expertise, and someone else ought to review them.
General question: shouldn't we support stdout for run_shell too?
I would say "no" for now since using a shell redirect is much less of an ask if you already use |
Description
When the new
stdoutparameter ofctx.actions.runis set, the stdout of the action is stored in the given file rather than printed in the terminal and reported via the BES. The file is otherwise an ordinary action output and follows BwoB download patterns. Since persistent worker communicate via stdout, actions that opt into them are not allowed to setstdout.Note that adding
stdoutonly was the uncontentious part of #5511 as it avoids any discussion of howstdoutandstderrwould behave if set to the same file (merging is not supported with remote execution etc.). The current PR does not make it harder to supportstderrif needed in the future.Motivation
Tools commonly emit useful output to
stdoutand going throughctx.actions.run_shelljust to capture it is overkill and has a worse hermeticity story (dependency on a shell, which is problematic in cross-platform build cases).Implements the most important part of #5511.
Build API Changes
Yes:
Provide a way to redirect stdout/stderr into a file #5511, where the stdout case has seen unanimous support.
Yes, it's a new parameter.
Checklist
Release Notes
RELNOTES[NEW]:
ctx.actions.runnow supports redirecting stdout to a file via the newstdoutparameter.