ci: add cadence input for test filtering in CI workflows by balasaajay · Pull Request #4561 · NVIDIA/Megatron-LM

balasaajay · 2026-04-30T20:11:28Z

What does this PR do ?

Introduced a new cadence input in the GitHub Actions workflow to filter tests based on the trigger type (pr|nightly|mergegroup).
Updated the action YAML to include the cadence input and modified the test execution logic to respect this new parameter.
Enhanced documentation in SKILL.md to explain the cadence filter and its usage in test cases.
Updated relevant Python scripts to accept and process the cadence argument for test execution.

⚠️ For major changes (either in lines of code or in its impact), please make sure to first share a design doc with the team. If you're unsure what's the best way to do so, contact the @mcore-oncall.

Issue tracking

For PRs from open-source community contributors:

New features: a linked issue is required. Please open a feature request and reference it here before submitting the PR.
Small updates (bug fixes, minor improvements): a linked issue is recommended and will accelerate the PR review process.

Linked issue:

Contribution process

Pre-checks

I have added relevant unit tests
I have added relevant functional tests
I have added proper typing to my code Typing guidelines
I have added relevant documentation
I have run the autoformatter.sh on my PR

Code review

Feel free to message or comment the @mcore-oncall to help accelerate your merge into main. The less complex your PR is, the faster it will be approved and merged!

All PRs start as draft. If you open a non-draft PR, it will be automatically converted to draft.

Step 1: Mark PR as "Ready for Review"

When your PR is ready, click Ready for Review.
An oncall reviewer is auto-assigned and expert reviewers are notified based on your changes.
- Some PRs may jump straight to step 2. This is determined by .github/CODEOWNERS.

⚠️ Only mark as ready once merge-conflicts are resolved and the CI is passing.
Final Review might get declined if these requirements are not fulfilled.

Step 2: Final Review

For PRs that change megatron/core, once all expert reviewers have approved, the Final Review label is applied automatically and final reviewers are assigned.

For PRs outside megatron/core, this step is skipped.

Step 3: Approved

Once all required reviewers have approved, the Approved label is applied automatically.

Merge

Any member of mcore-engineers will be able to merge your PR.

For MRs into `dev` branch

The proposed review process for `dev` branch is under active discussion.

MRs are mergable after one approval by either eharper@nvidia.com or zijiey@nvidia.com.

- Introduced a new `cadence` input in the GitHub Actions workflow to filter tests based on the trigger type (pr|nightly|mergegroup). - Updated the action YAML to include the `cadence` input and modified the test execution logic to respect this new parameter. - Enhanced documentation in SKILL.md to explain the cadence filter and its usage in test cases. - Updated relevant Python scripts to accept and process the `cadence` argument for test execution. - Adjusted existing recipes to include cadence values for better test management.

copy-pr-bot · 2026-04-30T20:11:32Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

balasaajay · 2026-04-30T20:15:20Z

/ok to test bc8b163

balasaajay · 2026-04-30T20:45:13Z

/ok to test ff867d8

balasaajay · 2026-04-30T20:59:21Z

/ok to test 9ff6021

balasaajay · 2026-05-02T00:02:14Z

/ok to test 015fda1

balasaajay · 2026-05-02T06:55:20Z

verified using workflow_dispatch - https://github.com/NVIDIA/Megatron-LM/actions/runs/25234600566 and with PR trigger - https://github.com/NVIDIA/Megatron-LM/actions/runs/25238441652?pr=4561

balasaajay · 2026-05-03T22:44:27Z

/ok to test 05bca68

…t configuration

ko3n1g · 2026-05-04T16:13:03Z

Thanks! For my own understanding: How does this now relate to the existing scope? The scopes are mr,nightly,mr-github,... which overlaps with cadence. What do you recommend to do with scope moving forward?

balasaajay · 2026-05-04T16:28:47Z

Thanks! For my own understanding: How does this now relate to the existing scope? The scopes are mr,nightly,mr-github,... which overlaps with cadence. What do you recommend to do with scope moving forward?

that's right, scope today mixes with cadence. with this PR, cadence becomes a separate axis for when the test runs and scope should shrink to just suite axis. I think, we should incrementally migrate nightly and weekly from scope to cadence. I would like to know wdyt? Thanks!

ko3n1g · 2026-05-04T16:33:03Z

Thanks! For my own understanding: How does this now relate to the existing scope? The scopes are mr,nightly,mr-github,... which overlaps with cadence. What do you recommend to do with scope moving forward?

that's right, scope today mixes with cadence. with this PR, cadence becomes a separate axis for when the test runs and scope should shrink to just suite axis. I think, we should incrementally migrate nightly and weekly from scope to cadence. I would like to know wdyt? Thanks!

okay yeah that makes sense. which scopes do you envision in future? it's like not going to be mr,nightly etc anymore, so perhaps we need a new vocabulary

balasaajay · 2026-05-04T16:49:07Z

once we strip cadence, scope is tracking suite size and github/gitlab. I think we can keep scope as: slim, github-slim, full, github-full, unit-tests to start with?

balasaajay · 2026-05-04T16:50:15Z

/ok to test b7c72f6

ko3n1g · 2026-05-04T18:03:59Z

once we strip cadence, scope is tracking suite size and github/gitlab. I think we can keep scope as: slim, github-slim, full, github-full, unit-tests to start with?

directionally i like it. let's take a moment to think outsize the box about good naming. some projects have p0, p1, p2; others L0, L1, L2.. i wonder if this might make it more intuitive to engineers?

balasaajay · 2026-05-04T18:29:02Z

once we strip cadence, scope is tracking suite size and github/gitlab. I think we can keep scope as: slim, github-slim, full, github-full, unit-tests to start with?

directionally i like it. let's take a moment to think outsize the box about good naming. some projects have p0, p1, p2; others L0, L1, L2.. i wonder if this might make it more intuitive to engineers?

Agreed. L0, L1, L2 are more intuitive. L0 = slim, L1 = full ... so on. so each test declares: scope: [L0], cadence: [pr, mergegroup, nightly]

balasaajay changed the title ~~feat: add cadence input for test filtering in CI workflows~~ ci: add cadence input for test filtering in CI workflows Apr 30, 2026

fix linter issues

ff867d8

copy-pr-bot Bot temporarily deployed to test April 30, 2026 20:46 Inactive

balasaajay added 2 commits April 30, 2026 13:56

add copyright notice to generate_jet_trigger_job.py

157fd43

Merge branch 'main' into abalasa.main-tests-label

9ff6021

copy-pr-bot Bot temporarily deployed to test April 30, 2026 21:00 Inactive

balasaajay temporarily deployed to test April 30, 2026 23:09 — with GitHub Actions Inactive

balasaajay added 3 commits April 30, 2026 16:22

revert changes to unit-tests

a347218

Remove redundant block (already defined earlier in the job).

5c6591c

Enhance CI workflow to skip SSO check for manual dispatch events

015fda1

copy-pr-bot Bot temporarily deployed to test May 2, 2026 00:03 Inactive

balasaajay added the Run functional tests label May 3, 2026

Merge branch 'main' into abalasa.main-tests-label

05bca68

copy-pr-bot Bot temporarily deployed to test May 3, 2026 22:45 Inactive

balasaajay removed the Run functional tests label May 4, 2026

Remove cadence entries for 'pr' and 'nightly' from gpt-1node.yaml tes…

9f50969

…t configuration

Merge branch 'main' into abalasa.main-tests-label

b7c72f6

copy-pr-bot Bot temporarily deployed to test May 4, 2026 16:51 Inactive

balasaajay marked this pull request as ready for review May 4, 2026 18:32

balasaajay requested a review from a team as a code owner May 4, 2026 18:32

svcnvidia-nemo-ci requested a review from a team May 4, 2026 18:32

svcnvidia-nemo-ci added the complexity: low label May 4, 2026

ko3n1g approved these changes May 4, 2026

View reviewed changes

svcnvidia-nemo-ci added the Approved All necessary approvals have been made label May 4, 2026

balasaajay enabled auto-merge May 4, 2026 19:56

balasaajay disabled auto-merge May 4, 2026 19:57

balasaajay merged commit c8fde51 into NVIDIA:main May 4, 2026
68 of 71 checks passed

Conversation

balasaajay commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do ?

Issue tracking

Contribution process

Pre-checks

Code review

Step 1: Mark PR as "Ready for Review"

Step 2: Final Review

Step 3: Approved

Merge

Uh oh!

copy-pr-bot Bot commented Apr 30, 2026

Uh oh!

balasaajay commented Apr 30, 2026

Uh oh!

balasaajay commented Apr 30, 2026

Uh oh!

balasaajay commented Apr 30, 2026

Uh oh!

balasaajay commented May 2, 2026

Uh oh!

balasaajay commented May 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

balasaajay commented May 3, 2026

Uh oh!

ko3n1g commented May 4, 2026

Uh oh!

balasaajay commented May 4, 2026

Uh oh!

ko3n1g commented May 4, 2026

Uh oh!

balasaajay commented May 4, 2026

Uh oh!

balasaajay commented May 4, 2026

Uh oh!

ko3n1g commented May 4, 2026

Uh oh!

balasaajay commented May 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

balasaajay commented Apr 30, 2026 •

edited

Loading

balasaajay commented May 2, 2026 •

edited

Loading