feat: Unify artefact formats and integrate the processings service#1
Merged
Conversation
…se builds) Captures the approved design: a third artefact format `branch` that holds a single mutable tarball, federation-excluded by default, so a staging registry can carry main-branch dev builds locally while mirroring production releases.
Records the design that produced the rewritten GitHub Actions section in docs/ci-integration.md (Alpine-built node_modules baked into the tarball, architecture=x64 upload, tag-only flow). Companion to the staging-flow spec landed earlier today.
A third artefact format holding a single mutable tarball — the docker-tag analogue for non-release builds. Each upload to POST /artefacts/branch/<id> replaces the previous tarball atomically; no version sub-docs and no retention bookkeeping. Manifest is extracted on upload so the doc carries packageName/version/licence for display, but _id is operator-chosen and intentionally distinct from the source package name (convention: <package-name>-<branch>). Federation filter: read-API-key callers (the mirroring path) get format !== 'branch' added to their access filter, so branch artefacts never leave the registry that owns them. A staging registry mirrors production releases via federation AND carries its own local branch builds, with no cross-contamination. Consumer support in lib-node-registry: new ensureBranchArtefact() that fetches the artefact doc to read dataUpdatedAt, downloads from /:id/branch/tarball, and caches under <cacheDir>/<id>/current/ with a .current-branch.json meta. Mutable tarball → cache invalidates when the source artefact's dataUpdatedAt bumps. docs/ci-integration.md gains a 'Publishing a branch build to staging' section walking through the staging API key, the GitHub staging environment, and a publish-main.yml workflow.
Clarifies the staging-flow section now that the feature has landed: - Distinguishes the operator-chosen branch artefact _id from the source package name (and explicitly couples that id to `allowedName` on the upload key — they must agree). - Names the federation behavior: production and staging hold the two flavours side-by-side without cross-contamination. - Adds a consumer-side note (pluginId shape, picker chip, lib-node-registry >= 0.3.0 requirement). - Renames the local build dir from `staging` to `build` in both the tag and branch workflow examples to avoid clashing with the "staging registry" terminology used elsewhere in the section.
…ormat Collapses today's npm (multi-version, semver resolver, retention) and branch (single mutable tarball) formats into one. Each release major and each dev branch becomes its own artefact identified by an operator-chosen ref id (convention <packageName>@<ref>). The version selector goes away at the registry boundary; the id is the flag. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
14 tasks: schema/types, mongo cleanup, service trim, federation filter, upload + download routes, legacy route deletion, allowedPackageName, remote-registries sync, UI, lib-node 0.4.0, docs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Also fix stale import of parseSemver/resolveVersionQuery/pruneOldVersions removed in tasks 1-5, and update GET /:id to not query the dropped versions collection. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…pers Task 8 was supposed to handle this but Task 4's removal of parseSemver, resolveVersionQuery and pruneOldVersions made the legacy routes non-functional. Pull the route deletion forward to unblock Task 6. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…back Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… route Also fixes a stale mongo.versions reference in the DELETE handler that was left over from Task 3 (versions collection dropped); without this fix the PATCH & DELETE test group would throw a 500. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The fixture upload calls were swept to /npm/:id but lib-node/index.ts still calls the deleted /versions/:v endpoint. Re-enabled in Task 13. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…on't collide Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… api-key field name Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Artefact model
tarballs: { [arch]: ... }map, with
noarchas the default slot for portable builds.versionscollection, theversiontype, the semverresolver, retention helpers, and the obsolete backfill upgrade scripts.
API
POST /artefacts/npm/:id— per-arch upload, updates one tarball slot.GET /artefacts/:id/tarball?architecture=— arch resolution withnoarchfallback./versionsand/branchroutes.allowedPackageName/allowedNamePrefix/allowedCategoryscopes.processings service can push artefacts and legacy metadata during the
v6.0 first-boot migration.
s3RootDirconfig option.UI
simplification.
lib-node
ensureArtefact, dropped the version param, released 0.4.0.extractDirso multi-arch caches don'tcollide.