Skip to content

[CI] Move benchmarks to the nightly run, label opt-in on PRs#3982

Open
vmoens wants to merge 2 commits into
gh/vmoens/281/basefrom
gh/vmoens/281/head
Open

[CI] Move benchmarks to the nightly run, label opt-in on PRs#3982
vmoens wants to merge 2 commits into
gh/vmoens/281/basefrom
gh/vmoens/281/head

Conversation

@vmoens

@vmoens vmoens commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Stack from ghstack (oldest at bottom):

Benchmarks cost 2 x ~40-60 min GPU jobs on every push to main and on
every PR push, while their only consumers are trend dashboards.

  • benchmarks.yml: drop the push-to-main trigger. The nightly
    orchestrator's daily workflow_call becomes the sole feed of the
    gh-pages trend: its skip-upload flag now derives from the run mode
    (upload in "full" mode only, so dry-run and smoke-test runs cannot
    touch gh-pages). workflow_dispatch remains for on-demand runs.
  • benchmarks_pr.yml: opt-in via the "benchmarks" PR label, with the
    "labeled" trigger type so applying the label starts the comparison
    immediately.
  • benchmarks_pr_comment.yml: exit cleanly when the benchmark job was
    skipped and no artifacts exist, instead of failing on an empty
    download.

Accepted tradeoffs: the trend granularity drops from per-push to
daily, and the 200% regression alert now implicates a day of commits
rather than a single push.

Co-Authored-By: Claude Fable 5 noreply@anthropic.com

[ghstack-poisoned]
@pytorch-bot

pytorch-bot Bot commented Jul 9, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3982

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 1 Pending

As of commit f1514b9 with merge base a74b478 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI Has to do with CI setup (e.g. wheels & builds, tests...) CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant