Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build artifacts are missing with "--remote_download_toplevel" cache option #13625

Closed
Tracked by #12665
loshca opened this issue Jun 30, 2021 · 3 comments
Closed
Tracked by #12665
Assignees
Labels
P2 We'll consider working on this in future. (Assignee optional) team-Remote-Exec Issues and PRs for the Execution (Remote) team type: bug

Comments

@loshca
Copy link
Contributor

loshca commented Jun 30, 2021

Description of the problem / feature request:

Assume we have two targets, one dependency of another:

genrule(
    name = "echo", 
    outs = ["echo.txt"],
    cmd = "echo \"something\" > $@",
)

genrule(
    name = "cat",  
    srcs = [":echo"],
    outs = ["cat.txt"],
    cmd = "cat $(locations :echo) > $@",
)

If we get cat target from cache with --remote_download_toplevel: bazel build cat --disk_cache=D:\bazel_disk_cache --remote_download_toplevel, then we will not be able to get output for echo with bazel build echo --disk_cache=D:\bazel_disk_cache --remote_download_toplevel. Instead we got an up to date message and that's it, bazel-bin/echo.txt is not presented :

INFO: Invocation ID: d8e174cf-83cd-49bf-9f51-3367241fd6e1
INFO: Analyzed target //:echo (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:echo up-to-date:
  bazel-bin/echo.txt
INFO: Elapsed time: 0.153s, Critical Path: 0.01s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action

What operating system are you running Bazel on?

Windows & MacOS

What's the output of bazel info release?

4.0.0

@loshca
Copy link
Contributor Author

loshca commented Jun 30, 2021

Probably we need a little bit more complicated logic with shouldTrustRemoteArtifacts.

@coeuvre
Copy link
Member

coeuvre commented Jul 1, 2021

IIUC, in your example, the second command for echo is ran right after the first one for cat. In this case, the ActionCache checked that no change between these two commands so the action will not be re-executed. Since downloading outputs only happens within action execution, the output of echo will not be downloaded.

This is a known issue and I am working on it. #12665

@coeuvre coeuvre self-assigned this Jul 1, 2021
@coeuvre coeuvre added P2 We'll consider working on this in future. (Assignee optional) team-Remote-Exec Issues and PRs for the Execution (Remote) team type: bug labels Jul 1, 2021
@appuser999
Copy link

@coeuvre Hi, I have encountered the same problem, is there any workaround?

load("//tools/docker:web.bzl", "web_container")
load("@npm//vite:index.bzl", "vite")

package(default_visibility = ["//visibility:public"])

_package_segments = len(package_name().split("/"))

vite(
    name = "build",
    outs = ["dist"],
    ...
)
web_container(
    name = "frontend",
    dist = [":build"],
)

error log:

ERROR: /data/xxx/BUILD.bazel:8:14: output 'xxx/dist' was not created

I tried to add tags = ["no-remote"] to the vite rule, and build with --experimental_allow_tags_propagation to get around the problem, but this target can't use the remote cache anymore. Once I change the tags to no-remote-exec (remote-cache enabled), bazel will not download outputs dist to bazel-out and the problem will be reproduced.

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 We'll consider working on this in future. (Assignee optional) team-Remote-Exec Issues and PRs for the Execution (Remote) team type: bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants