Test logs not copied to bazel-testlogs when test target has incoming edge transition

[prestonvanloon]

Description of the problem:

When running bazel test on a test target with an starlark incoming edge transition, no test outputs are copied to bazel-testlogs directory.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

I've made a minimal reproduction repository here: https://github.com/prestonvanloon/bazel-starlark-edge-transition-bug

What operating system are you running Bazel on?

linux - ubuntu 20.04

What's the output of bazel info release?

release 4.1.0

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

N/A

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

N/A

Have you found anything relevant by searching the web?

No

Any other information, logs, or outputs that you want to share?

With the example repo linked above, here are some notes:

We expect a test target with an incoming edge transition would behave the same as one without an incoming edge transition.

When running these two targets, we expect to see the test outputs under bazel-testlogs.

//:80columns_with_transition
//:80columns

However, we only see testlogs for //:80columns.

bazel test //...

tree bazel-testlogs

bazel-testlogs
└── 80columns
    ├── test.cache_status
    ├── test.log
    ├── test.outputs
    ├── test.outputs_manifest
    ├── test.raw_splitlogs
    └── test.xml

We should have seen the same outputs for the transition test, but nothing was copied.

You can also reproduce the issue by running only the test target bazel test //:80columns_with_transition and seeing that bazel-testlogs does not have anything from that target.

[prestonvanloon]

Something I've noticed is a difference in the test output paths. When changing the data to make the test fail, we can see the following paths.

//:80columns                                                             FAILED in 0.1s
  /home/preston/.cache/bazel/_bazel_preston/849dbde84a22ddb2fb36884b9d5c6317/execroot/__main__/bazel-out/k8-fastbuild/testlogs/80columns/test.log
//:80columns_with_transition                                             FAILED in 0.1s
  /home/preston/.cache/bazel/_bazel_preston/849dbde84a22ddb2fb36884b9d5c6317/execroot/__main__/bazel-out/k8-fastbuild-ST-f38523c95a1a/testlogs/80columns_with_transition/test.log

I suspect that bazel is copying everything from /home/preston/.cache/bazel/_bazel_preston/849dbde84a22ddb2fb36884b9d5c6317/execroot/__main__/bazel-out/k8-fastbuild/, but the target with a transition has k8-fastbuild-ST-f38523c95a1a.

[gregestren]

Thanks for the clear & easy repro, @prestonvanloon .

This is a known design blemish. Your second comment is spot on with the analysis. Transitions change output directory bases to prevent the same action configured in two different ways in the same build to write to the same output (which wouldn't be safe). The logic that sets bazel-testlogs predates this and doesn't consider these differences.

This hasn't been solved yet because it's not a trivial fix. The symlink logic may be able to merge both results into the same base, if the actual outputs never clash. Alternatively, each configuration could appear in its own subdirectory under bazel-testlogs. In that case we'd have to be careful not to break consumers that expect the exact directory structure you see now (Hyrum's Law and all that).

This is probably less pressing than symlinks like bazel-bin, which I think consumers rely on more heavily.

This flag speaks to the issue somewhat, although I think it's logic is too simple for your use case.

I'd love to see this cleaned up as a design / feature request. I'm not aware of anyone with the bandwidth to try tackling it now. I don't think it has to be a huge project. But it does need to be consciously thought out.

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[prestonvanloon]

@bazelbuild/triage I believe this is still an issue. Thanks