Skip to content

ci(bazel): restore change-aware matrix scheduling#424

Merged
kristinapathak merged 1 commit into
mainfrom
ci/change-aware-bazel-matrix
Jul 24, 2026
Merged

ci(bazel): restore change-aware matrix scheduling#424
kristinapathak merged 1 commit into
mainfrom
ci/change-aware-bazel-matrix

Conversation

@balajinvda

@balajinvda balajinvda commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Why

Since #343 merged, every PR rebuilds and tests every service, including docs-only changes like #423. #343 (commit f286e51) added a blanket run_all=true override on top of the change-aware scheduler that had been working since commit 126c56b (feat(ci): change-aware GHA bazel matrix). The override was a safety net while debugging cloud-tasks test output during that import, and is no longer needed.

Each subtree is an isolated Bazel module, so a change under one service builds and tests only that service's //... and cannot break another module. Forcing the full matrix on unrelated changes just burns queue time (and pays a cold rebuild on the bazel-docker lanes, which only warm on main pushes).

What changed

Remove the unconditional run_all=true override in the detect job, restoring change-aware scheduling. Everything else is intentionally unchanged:

  • Reverse-dependency edges: a Java framework change (nv-boot-parent) still fans out to every Java service (cloud-tasks); shared Java/root infra still schedules every Java row; ROOT_GLOBS still select the root row.
  • Full-matrix fallbacks stay: workflow_dispatch, a change to this workflow itself, and unreadable/rewritten/new-branch diffs.
  • A docs-only or helm-only change selects zero rows; the bazel-verification gate passes on any=false.

Customer Release Notes

Not customer visible (CI-only).

Testing

Traced the detect selection logic against representative changesets:

  • grpc-proxy source change -> only the grpc-proxy row.
  • nv-boot-parent change -> nv-boot-parent plus every java-service (cloud-tasks) rebuilt and tested in its docker-host lane.
  • docs-only change (as in docs: update 0.6.1 release candidate manifest #423) -> zero rows; verification gate exits 0 on any=false.

YAML validated with a parser; the unconditional run_all=true assignment is removed while the conditional fallbacks remain.

References

Override introduced in #343 (f286e51). Change-aware scheduling originally landed in 126c56b.

Summary by CodeRabbit

  • Bug Fixes
    • Bazel build and test workflows now run only the affected modules and their dependent Java targets by default.
    • Full workflow coverage remains available for manual runs, workflow changes, and changes that cannot be safely analyzed.
    • Improved workflow guidance clarifies how root-level Java dependency or configuration changes are handled.

Drop the blanket run_all=true override so the detect job schedules only
the subtrees a change actually touches. Each subtree is an isolated Bazel
module, so a change under one service builds+tests only that service's
//..., never another module. A docs-only or helm-only change selects zero
rows (the verification gate passes on any=false). Reverse-dependency edges
are unchanged: a Java framework change still fans out to every Java service,
shared Java/root infra still schedules every Java row, and ROOT_GLOBS still
select the root row. workflow_dispatch, a change to this workflow, and
unreadable/rewritten/new-branch diffs still fall back to the full matrix.

This ends full-matrix rebuilds on unrelated PRs (e.g. a docs-only change
rebuilding every service).

Co-authored-by: Balaji Ganesan <bganesan@nvidia.com>
@balajinvda
balajinvda requested a review from a team as a code owner July 24, 2026 16:39
@balajinvda
balajinvda requested a review from apartha-nv July 24, 2026 16:39
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The Bazel workflow now uses existing change-aware detection to select affected subtree modules and Java reverse-dependency rows, while retaining full-matrix fallbacks for dispatches, workflow changes, and unsafe or unknown diffs.

Changes

Bazel workflow scheduling

Layer / File(s) Summary
Enable change-aware matrix detection
.github/workflows/bazel.yml
Removes the unconditional run_all=true override and documents subtree selection, reverse-dependency edges, no-op paths, full-matrix fallbacks, and root-owned Java dependency handling.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: apartha-nv

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title follows Conventional Commits and accurately describes this CI-only Bazel workflow scheduling change.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/change-aware-bazel-matrix

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/bazel.yml:
- Around line 174-184: Replace every pipelined printf-to-grep path match in the
change-aware scheduling logic with a non-pipelined check against the
changed-path data, such as a here-string passed to grep with fixed-string
matching. Update the workflow, framework, and subtree tests while preserving
their existing match semantics and scheduling behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d5529ef2-db6e-41da-88ec-4aa3ec799354

📥 Commits

Reviewing files that changed from the base of the PR and between 2961f91 and 2a391c8.

📒 Files selected for processing (1)
  • .github/workflows/bazel.yml

Comment thread .github/workflows/bazel.yml
@kristinapathak
kristinapathak enabled auto-merge July 24, 2026 16:53
@kristinapathak
kristinapathak added this pull request to the merge queue Jul 24, 2026
Merged via the queue into main with commit 3c63522 Jul 24, 2026
34 checks passed
@kristinapathak
kristinapathak deleted the ci/change-aware-bazel-matrix branch July 24, 2026 16:55
vrv3814 pushed a commit to mesutoezdil/nvcf that referenced this pull request Jul 25, 2026
* ci(bazel): harden detect matching + cap matrix concurrency

Two fixes for the change-aware detect job restored in NVIDIA#424:

1. Replace `printf ... | grep -q` with a here-string (`grep -q PATTERN
   <<<"$changed"`). Under `set -o pipefail`, grep -q closes the pipe on
   its first match and printf can take SIGPIPE (141); pipefail then makes
   the pipeline non-zero, so a real match reads as no-match and the subtree
   is silently dropped from the matrix. The here-string removes the pipe.
   (CodeRabbit finding on NVIDIA#424.)

2. Add max-parallel: 8 to the bazel matrix. A full matrix otherwise starts
   ~20 jobs at once and every runner pulls actions/checkout from
   codeload.github.com in the same instant, tripping GitHub's action
   download rate limit (HTTP 429 in Set up job). Batching keeps the burst
   under the limit; change-aware scheduling keeps most PRs below the cap.

Co-authored-by: Balaji Ganesan <bganesan@nvidia.com>

* ci(bazel): cap docker lane concurrency + rename the merge-gate job

- Add max-parallel to the bazel-docker matrix so the docker-host rows do
  not add to the simultaneous actions/checkout burst (companion to the
  cap on the container bazel job).
- Rename the aggregate gate job from "bazel verification" to "bazel
  required checks" so it is recognizable as the single status check to
  require for merge. Under change-aware scheduling the individual
  bazel (<service>) rows are dynamic and must not be required; this
  always-run rollup is the stable gate.

Co-authored-by: Balaji Ganesan <bganesan@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants