-
Notifications
You must be signed in to change notification settings - Fork 214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remote execution can't find executable #532
Comments
I think:
|
I think the reason of the problem must be a cooperation problem between Buck2 and Buildbarn. Also, the cooperation between Bazel and Buildbarn also does not have the same question. I think that Bazel not only sends Linux the command but also additional information to Buildbarn. So Buildbarn can search for the command correctly |
To confirm, does buck2 send an empty $PATH variable? Here are two reference images for Buildbarn to illustrate the problem: Standard path, nothing fancy (arbitrary action) Path is empty: |
Hi @stagnation , Reproduce:
Please kindly help:
|
Also, I dumped out PATH when using ' PS.
|
When the error occurred, I saw the message Buck2 sent to Buildbarn. |
Good! Some more background on Buildbarn, you can indeed set variables for the runner which is often useful. But the PATH resolution is made inside the process, here, rather than through the regular process spawn. So it behaves a little bit different from what you expect. This has the benefit of very strict hermticity, and gives the build client the most flexibility and control of which programs are spawn. The system tools of the runner are not available unless explicitly opted in through the PATH variable in the Command message. |
I checked some more, and could not build the original Buildbarn example with the latest Buildbarn images. That is because the change to the stricter path resolution was made after the initial example: buildbarn/bb-remote-execution@1524fef This details the reasoning and that it complies better to the remote execution API. The genrule can be made to work by adding a small PATH to the action:
original buck2 commit from last winter: 134ba21 |
For remote execution of Buildbarn,I think Buck2 should send a default minimum PATH environment(eg. /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin), or like buildbarn/bb-remote-execution#82. Buildbuddy's RE does not require this PATH, nor does the local executor. |
Yeah, so the behavior is technically implementation defined here. I wouldn't be opposed to adding something on the buck2 side to attempt to set some sane path, but we should only do that for implementations that actually don't set one on their own, I don't think we really want to override |
It is actually configurable on the worker side as well: https://github.com/buildbarn/bb-remote-execution/blob/f9ea0294c9a36683d06aef1840ba39c2eaccfb68/pkg/proto/configuration/bb_worker/bb_worker.proto#L313-L320 |
@EdSchouten I had just discovered that myself. That is almost certainly the way to fix it since it makes more sense for the PATH to be configured on the RE side since it depends on the runner Docker image. Thanks. |
Following the discussion in facebook#532 .
Following the discussion in facebook#532 .
Following the discussion in facebook#532 .
Following the discussion in facebook#532 .
The remote execution with Buildbarn example doesn't seem to work. I'm running Buildbarn in Window's Docker Desktop and Buck2 in Ubuntu inside WSL2.
I get:
Error was returned on the stream by RE: Failed to run command: Cannot find executable "sh" in search paths ""
This comes from here: https://github.com/buildbarn/bb-remote-execution/blob/4b3a11b508cf06a3dfa17f141b3ea505bc039434/pkg/runner/local_runner_unix.go#L78
It seems like the "search paths" should be the PATH that Buck2 sends to the remote execution server but I can't find any place in Buck2 where I can set that. Where can I set it?
For the moment I'm using absolute paths in system_cxx_toolchain. Maybe I'm missing something since no one else is having this problem.
The text was updated successfully, but these errors were encountered: