ci: build the SDK in one workflow, core -> media via needs - #22
Merged
Conversation
Replaces core.yml + media.yml with a single sdk.yml holding a
core-<platform> -> media-<platform> pair per platform, wired with `needs`.
`needs` orders the two halves inside a run; it does not replace the rolling
`continuous` prerelease, which is what shares one core build ACROSS runs. Both
are still there, they just solve different problems: the release still carries
the hashed core, and a leg whose hash is already published still short-circuits
in seconds.
What the split cost that this does not:
- the workflow_run cascade. It fires on *completed*, not *succeeded*, so a
cancelled core still started one; that needed a conclusion guard (35c90c0)
and its own concurrency group to stop a doomed cascade evicting a live run.
- two entry points on master racing in one concurrency group, which left six
cancelled checks on every master commit (f8e4a28).
- resolve-core.sh's `skip` mode, which only existed to defer to the cascade.
Every call here passes is_master=false, so it goes unused.
- building the core TWICE on a pull request. A PR cannot publish a release,
so core.yml built the core and media.yml then rebuilt it inline: on PR #19,
53min in core-linux-x86_64 plus a 304min core+media build in media. The
core job now hands its tarball over as a run-scoped artifact.
Also new: core jobs run on tag pushes (core.yml did not), so a tag reuses the
published core instead of taking media's inline path. PUBLISH_CORE excludes
tags so tagging an older commit cannot clobber the rolling core.
Failure semantics get finer, not coarser: a failed core leg used to fail the
core workflow and skip all six media legs; now it only skips its own platform.
core-windows keeps continue-on-error, and media-windows gets `if: !cancelled()`
so it still runs and falls back to the published core, as it did before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prototype: one workflow instead of the core.yml + media.yml pair, with
core-<platform>->media-<platform>wired byneeds.Why
needsalone isn't enoughneedsorders the two halves inside a run. The rollingcontinuousprerelease is what shares one core build across runs — every later push and every PR downloads a core built hours earlier by a different run. So the release handoff stays exactly as it is; only the cascade goes away.What the split cost that this doesn't
workflow_runcascade. It fires on completed, not succeeded, so a cancelled core still started one. That needed a conclusion guard (35c90c0) plus its own concurrency group so a doomed cascade wouldn't evict a live run.resolve-core.sh'sskipmode, which existed only to defer to the cascade. Every call here passesis_master=false, so it goes unused (the script keeps it for the manualbuild.ymlescape hatch).core-linux-x86_64spent 53min andmedia-linux-x86_64then spent 304min on aSTAGE=fullcore+media build (Extract core into prefixskipped ->mode=inline). The core job now hands its tarball to media as a run-scoped artifact.Use cases
PUBLISH_COREPUBLISH_SDKSDK_RELEASEcontinuous(rolling)sdk*tag${{ github.ref_name }}(not rolling)continuousforce_core_rebuildinput availableTwo deliberate changes:
inlinepath and rebuilt the whole core inside media. Now it reuses the published core, so tag builds get faster and can't diverge from what master published.PUBLISH_COREexcludes tags, so tagging an older commit can't clobber the rolling core with an older one. Everything else still publishes it, as before.Failure semantics — finer, not coarser
continue-on-error-> workflow still succeeds -> media rancontinue-on-errorkept,media-windows: if: !cancelled()-> still runs, falls back to the published coresdk-<ref>group; the second push simply supersedes the firstVerification
Job/step structure was diffed against the originals with a yaml-cpp parser: all six core/media jobs match the old ones step-for-step, in order, with the same
if:conditions. The only additions areHand core to media/Core handed over by ..., and themode != 'skip'guards dropped as unreachable.This PR run exercises the pull_request path end to end. The master and tag paths are expression-level changes on the same steps and scripts; the
planjob prints the resolved values for whatever event it runs under, so the master/tag rows can be confirmed from the first run after merge (or viaworkflow_dispatch) rather than inferred.Since core.yml/media.yml are deleted here, this PR does not double-build: only sdk.yml runs. Reverting is a single
git revert.Open question
media-<plat>needs the wholecore-<plat>matrix, somedia-linux-x86_64waits forcore-linux-aarch64too. That's still finer than today (the cascade waits for all six core legs), and irrelevant when core short-circuits. Splitting into per-arch job pairs, or reusable workflows, would remove it if the wait ever matters.