Skip to content

skyframe: report top-level aspect failures - #29850

Closed
sluongng wants to merge 1 commit into
bazelbuild:masterfrom
sluongng:sluongng/fix-output-conflict-events
Closed

skyframe: report top-level aspect failures#29850
sluongng wants to merge 1 commit into
bazelbuild:masterfrom
sluongng:sluongng/fix-output-conflict-events

Conversation

@sluongng

@sluongng sluongng commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

OutputArtifactConflictTest flakes under Skymeld + nokeep_going because
evaluation can abort with a TopLevelAspectsKey as the only processed
analysis error.

A local stress run on origin/master reproduced the flake with:

bazel test --nocache_test_results --runs_per_test=50 \
  --test_filter=testConflictErrorAndUnfinishedAspectAnalysis_mergedAnalysisExecution \
  //src/test/java/com/google/devtools/build/lib/buildtool:OutputArtifactConflictTest

The failing run looked like this:

FAIL: ...OutputArtifactConflictTest (shard 3 of 3, run 32 of 50)
testConflictErrorAndUnfinishedAspectAnalysis_mergedAnalysisExecution
[skymeld=true,minimizeMemory=true,keepGoing=false]
expected to contain any of:
  [//x:y, //x/y:y, //x:fail_analysis]
but was                   : []

The existing error processor skipped non-configured-target keys in this
path, so BEP consumers could miss the AnalysisFailureEvent even though the
analysis error was reported to the event handler.

Report failures against the top-level aspect's base configured target so
the failure is emitted on the same path used for configured target keys.
Update the BEP integration test expectation for the additional aborted
event that is now reported.

Local validation:

  • Reproduced the baseline flake on origin/master
    (a300dd5): failed on run 32/50
    with empty analysisFailures.
  • Verified the CopyOnWriteArrayList-only test-listener change still failed
    on run 10/50 with the same empty-list assertion.
  • Verified this patch without the listener-list change passed 50/50 and
    100/100 local stress runs of the same filtered test.

@sluongng
sluongng marked this pull request as ready for review June 16, 2026 17:00
@github-actions github-actions Bot added team-Core Skyframe, bazel query, BEP, options parsing, bazelrc awaiting-review PR is awaiting review from an assigned reviewer labels Jun 16, 2026
@sluongng
sluongng force-pushed the sluongng/fix-output-conflict-events branch 4 times, most recently from 4e4bfd1 to a445d7b Compare June 17, 2026 09:01
@meisterT
meisterT requested a review from joeleba June 18, 2026 15:48

@joeleba joeleba left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix. This entire error processing area is plagued by legacy code and needs some reworking. It's on my TODO list but I just need some time to work on it...

@joeleba joeleba added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels Jun 19, 2026
@deepalak56

deepalak56 commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Please look into the failing checks.

OutputArtifactConflictTest flakes under Skymeld + nokeep_going because
evaluation can abort with a TopLevelAspectsKey as the only processed
analysis error. A local stress run on origin/master reproduced it as:

  FAIL: ...OutputArtifactConflictTest (shard 3 of 3, run 32 of 50)
  testConflictErrorAndUnfinishedAspectAnalysis_mergedAnalysisExecution
  [skymeld=true,minimizeMemory=true,keepGoing=false]
  expected to contain any of:
    [//x:y, //x/y:y, //x:fail_analysis]
  but was                   : []

The existing error processor skipped non-configured-target keys there.
BEP consumers could miss the AnalysisFailureEvent even though the
analysis error was reported to the event handler.

Report failures against the top-level aspect's base configured target so
the failure is emitted on the same path used for configured target keys.
Update the BEP integration test expectation for the additional aborted
event that is now reported.
@sluongng
sluongng force-pushed the sluongng/fix-output-conflict-events branch from a445d7b to 8c88070 Compare June 19, 2026 11:18
@sluongng

Copy link
Copy Markdown
Contributor Author

I don’t think the failure was related so i just rebased

@sluongng

Copy link
Copy Markdown
Contributor Author

@joeleba @deepalak56 would appreciate if somebody could help me import this PR

@joeleba

joeleba commented Jun 22, 2026

Copy link
Copy Markdown
Member

It's in progress. Thanks for your patience.

@github-actions github-actions Bot removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Jun 23, 2026
@fmeum

fmeum commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

@bazel-io fork 9.2.0

@fmeum

fmeum commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

@bazel-io fork 8.8.0

bazel-io added a commit to bazel-io/bazel that referenced this pull request Jul 3, 2026
…bazelbuild#29951)

OutputArtifactConflictTest flakes under Skymeld + nokeep_going because
evaluation can abort with a TopLevelAspectsKey as the only processed
analysis error.

A local stress run on origin/master reproduced the flake with:

```text
bazel test --nocache_test_results --runs_per_test=50 \
  --test_filter=testConflictErrorAndUnfinishedAspectAnalysis_mergedAnalysisExecution \
  //src/test/java/com/google/devtools/build/lib/buildtool:OutputArtifactConflictTest
```

The failing run looked like this:

```text
FAIL: ...OutputArtifactConflictTest (shard 3 of 3, run 32 of 50)
testConflictErrorAndUnfinishedAspectAnalysis_mergedAnalysisExecution
[skymeld=true,minimizeMemory=true,keepGoing=false]
expected to contain any of:
  [//x:y, //x/y:y, //x:fail_analysis]
but was                   : []
```

The existing error processor skipped non-configured-target keys in this
path, so BEP consumers could miss the AnalysisFailureEvent even though
the
analysis error was reported to the event handler.

Report failures against the top-level aspect's base configured target so
the failure is emitted on the same path used for configured target keys.
Update the BEP integration test expectation for the additional aborted
event that is now reported.

Local validation:
- Reproduced the baseline flake on origin/master
  (a300dd5): failed on run 32/50
  with empty `analysisFailures`.
- Verified the CopyOnWriteArrayList-only test-listener change still
failed
  on run 10/50 with the same empty-list assertion.
- Verified this patch without the listener-list change passed 50/50 and
  100/100 local stress runs of the same filtered test.

Closes bazelbuild#29850.

PiperOrigin-RevId: 936417007
Change-Id: If2ea9a4b157fb6d646608a8adfb2c5f4d12de5c4

Commit
bazelbuild@8d8826c

Co-authored-by: Son Luong Ngoc <sluongng@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

team-Core Skyframe, bazel query, BEP, options parsing, bazelrc

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants