Skip to content

[SPARK-58242][INFRA] Move scheduled CIs for 4.2 to branch-4.2#57408

Closed
gaogaotiantian wants to merge 1 commit into
apache:masterfrom
gaogaotiantian:decouple-branch42-scheduler
Closed

[SPARK-58242][INFRA] Move scheduled CIs for 4.2 to branch-4.2#57408
gaogaotiantian wants to merge 1 commit into
apache:masterfrom
gaogaotiantian:decouple-branch42-scheduler

Conversation

@gaogaotiantian

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Add a unified entry for all scheduled CIs for branch-4.2 (branch42_scheduler.yml). It uses gh workflow to trigger the self-contained build workflows on branch-4.2, and removes the per-build build_branch42_*.yml files from master.

This follows the same approach as SPARK-56990 (#56046) for branch-4.x, SPARK-57327 (#56379) for branch-4.1, and SPARK-57483 (#56530) for branch-4.0. SPARK-58180 (#57319) already laid the ground on branch-4.2 by making the build workflows self-contained and dispatchable, so this PR only needs to change the master scheduled tasks.

The scheduler triggers the following targets on branch-4.2 (also exposed via workflow_dispatch): build_java17, build_java21, build_java25, build_maven, build_maven_java21, build_non_ansi, build_python_3.11, build_python_3.14. Note this differs from the branch-4.1 set: branch-4.2 has a java25 build but no python_pypy3.10 build. The cron schedule preserves the original times and */2 frequency from the build_branch42_*.yml files.

README.md is updated so the branch-4.2 badges point at the self-contained workflows filtered by ?branch=branch-4.2.

Why are the changes needed?

This is part of decoupling our CIs. All branch-4.2 related CIs should only rely on files on branch-4.2, with the exception of this new scheduler file which is needed on master to trigger scheduled tasks (scheduled workflows only fire from the default branch).

Does this PR introduce any user-facing change?

No. CI only.

How was this patch tested?

These workflows can be triggered manually via workflow_dispatch once merged.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 4.8)

This pull request and its description were written by Isaac.

Add a unified entry for all scheduled CIs for branch-4.2
(`branch42_scheduler.yml`). It uses `gh workflow` to trigger the
self-contained build workflows on `branch-4.2`, and removes the
per-build `build_branch42_*.yml` files from `master`.

Co-authored-by: Isaac
@uros-b

uros-b commented Jul 22, 2026

Copy link
Copy Markdown
Member

Thank you @gaogaotiantian and @HyukjinKwon!

@dongjoon-hyun dongjoon-hyun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you, @gaogaotiantian and all.

@dongjoon-hyun

Copy link
Copy Markdown
Member

Merge Summary:

Posted by merge_spark_pr.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants