feat(ci): toolchain volume build pipeline#31
Merged
Conversation
…mage Add next, react, react-dom, typescript, tailwindcss, postcss, autoprefixer, and @types/* packages to the coinbase project's cache_warm configuration. This populates the npm cache during image build so agent npm install hits cache instead of downloading ~200 transitive dependencies cold. Also fix npm cache permissions: the global npm install runs as root, creating root-owned cache entries. Added chown after install so the cache_warm step (running as agent) can write to the cache. Expected impact: reduce npm install from ~80-90s to ~10-20s in the coinbase-ecommerce benchmark scenario.
New CI workflow that builds toolchain layers as filesystem tarballs
and publishes them as OCI artifacts to GHCR. This runs alongside
the existing docker-publish.yml (which continues building images
for backwards compatibility during migration).
Workflow:
1. Build base toolchain from base-system.Dockerfile → export → push tarball
2. Build intermediate layers (foundry, rust, go, scipy) in parallel → tarballs
3. Generate + publish toolchain manifest (stack → layers mapping from config.json)
Artifacts published to ghcr.io/tangle-network/toolchains/{layer}:{sha}
with :latest tags. Hosts pull with oras and extract to /opt/toolchains/.
See: agent-dev-container PR #459 and docs/rfcs/toolchain-volumes.md
for the full architecture and deprecation plan.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add CI workflow that builds toolchain layers as filesystem tarballs and publishes them as OCI artifacts to GHCR. This is the build side of the toolchain volumes architecture that replaces per-stack Docker images.
Context
See agent-dev-container PR #459 and its RFC for the full architecture.
TL;DR: Instead of building 78 Docker images (3-7 GB each), we build ~5 shared toolchain tarballs that get bind-mounted onto a single thin base container. Cold start drops from 30-200s to 3-10s. Host disk drops from 300+ GB to ~23 GB.
What This Adds
New workflow
.github/workflows/toolchain-volumes.yml:base/base-system.Dockerfile, exports filesystem as tarball, pushes toghcr.io/tangle-network/toolchains/base:{sha}config.json), pushes toghcr.io/tangle-network/toolchains/manifest:{sha}Uses ORAS to push tarballs as OCI artifacts (not Docker images).
What This Does NOT Change
docker-publish.ymlcontinues running (builds images for backwards compatibility)config.jsonunchangedThe existing image pipeline and the new volume pipeline run side-by-side until migration is complete, at which point
docker-publish.ymlcan be removed.Triggers
mainwhenbase/,intermediate/,config.json, or the workflow itself changesHost Consumption
Hosts pull with:
The host-agent's
ToolchainSyncerautomates this (in agent-dev-container PR #459).🤖 Generated with Claude Code