Skip to content

RFC: pnpm as the CI engine, and the pnpm CI server - #25

Draft
zkochan wants to merge 9 commits into
mainfrom
pnpm-ci
Draft

RFC: pnpm as the CI engine, and the pnpm CI server#25
zkochan wants to merge 9 commits into
mainfrom
pnpm-ci

Conversation

@zkochan

@zkochan zkochan commented Aug 28, 2026

Copy link
Copy Markdown
Member

Two companion RFCs proposing that pnpm compose its existing pieces — the task orchestration of #23, the workspace task cache of #22, and RFC 0007's artifact transport — into a CI engine, and that pnpr host its control plane.

0000-pnpm-ci.md — pnpm as the CI engine. A single command that makes pnpm the engine a CI run delegates to: an unconditionally frozen install, merge-base affected selection as a pre-pass over the cache (the pipeline always covers everything; the cache decides what executes), the task graph under no-bail semantics with the orchestration RFC's exit contract, read/write cache policy falling out of RFC 0007's possession-based write grant, and a machine-readable event stream plus run summary as the stable product surface. The host — GitHub Actions, GitLab, a cron job — keeps events, machines, and secrets; the pipeline's meaning moves into the repository.

0000-pnpm-ci-server.md — the pnpm CI server. The Buildkite shape with even the control plane on the organization's own server: pnpr grows four severable tiers, ordered by risk — signed run records and a UI over logs the artifact store already holds; task leasing to self-provisioning pnpm ci agent fleets (planning is itself a leased task); forge, cron, and registry-native triggers; and secrets pnpr relays but cannot read. One invariant governs every tier: the server is never the root of trust — its compromise stays bounded by verification clients perform, extending the pattern RFC 0007 established. It ships as a feature-gated surface of the pnpr binary, not a new server.

Both documents are grounded in a working proof of concept (draft PR pnpm/pnpm#14233): pnpm pipeline implements the engine's contract Rust-only with a local-tier task cache, and pnpr carries tier 1 of the server — append-only run-record endpoints and a viewer page. Findings from building and dogfooding it are folded into the texts rather than left as claims, including: the pnpm ci name is doubly taken (it is the clean-install command, and ci is a taken setting name); task-cache keys must be computed over a dependency-closed graph or they become selection-dependent; the task cache's restore record needs content hashes, not a path set (feedback that belongs to #22); red runs must be recorded before the failure exit is raised; and run records should stay opaque to the server, which validates identity and bounds only.

Draft, for discussion — the engine RFC is written to be ratifiable on its own, with the server RFC consuming its contract.


Written by an agent (Claude Code, claude-fable-5).

zkochan and others added 6 commits August 27, 2026 12:45
Proposes a pnpm ci command composing the task orchestration RFC
(#23), the workspace task cache RFC (#22), and the
shared side-effects cache transport (RFC 0007) into a CI entry point:
frozen environment enforcement, merge-base affected selection as a
pre-pass over the cache, no-bail pipeline runs, read/write cache policy
for untrusted refs, and a machine-readable event stream and run summary.
Triggers, runners, secrets, and UI stay with the CI host; distributed
execution is deferred with forward-compatibility constraints recorded.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The host tier (triggers, secrets, UI, coordination) moves from
'pnpm cannot do and should not want' to a companion RFC, with a fourth
forward-compatibility constraint recorded: the server never becomes the
root of trust — pnpr's compromise must stay bounded by client-side
verification, per RFC 0007's pattern. Machine provisioning remains
excluded from the whole sequence.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Companion to the pnpm ci RFC: pnpr as the CI control plane in the
Buildkite shape — control plane on the org's own server, execution on
the org's own agents. Four severable tiers ordered by risk: signed run
records and a UI over the artifact-store logs; task leasing to
self-provisioning pnpm ci agents with planning-as-a-task; forge,
cron, and registry-native triggers with server-side ref-trust
classification; and secrets pnpr relays but cannot read (sealed to
agent identities, or references into external stores). Governed
throughout by the recorded invariant that the server is never the
root of trust.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CI ships as crates in the pnpr workspace compiled into the pnpr binary,
off by default; the differing traffic/exposure profile is answered by
running a second instance of the same binary in a CI role sharing
storage (already supported via RFC 0007's cross-process locking), not
by a separate product.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three corrections from the Rust proof of concept (pnpm pipeline on the
pnpm-ci-poc branch): the name question is settled against pnpm ci (it
is the clean-install command in both stacks, and ci is a taken setting
name — the config section collides too); the selection pre-pass must be
dependency-closed with only affected projects' tasks requested, or task
cache keys become selection-dependent and upstream outputs can be
absent; and the task cache RFC's last-outputs record must carry content
hashes rather than a file set, or a restore clobbers user edits.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The run-record surface shipped as a PoC on the pnpm-ci-poc branch and
validated the process topology (a fourth feature-gated surface of the
pnpr binary; a pipeline-only tier serves alone). Adopted as guidance:
records opaque to the server (identity and bounds validated, shape
belonging to the client), append-only as refusal at the endpoint, red
runs recorded before the failure exit, authenticated reads with an
unauthenticated static viewer page. Recorded as divergences: token auth
in place of the signed-append identity, and a filesystem-only store.
Unresolved questions updated: workspace identity has a working slug
shape but still needs org-auth claiming; per-surface client
configuration (pnprServer implies install offloading) and viewer
authentication are new entries.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 28, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

zkochan and others added 3 commits August 28, 2026 02:39
The engine consumes four project-level answers from the version
control (base, changed projects, input identity, revision +
materialization), so the boundary is drawn there: git as the default
and floor (merge base + diff attribution + index blob hashes), Bit as
the intended second instance, where each answer is state the tool
maintains natively — component-granular change tracking, snap hashes
as input identity, enforced component boundaries, closure-scoped
materialization, and globally claimed identities. Constraints: the
provider is read-only plus materialize, and is shelled out to, never
linked. The server RFC inherits the generalization: provider-mediated
source delivery for agents, provider-dependent lease granularity, the
workspace-identity question dissolving under claimed component names,
and a component-export sibling of registry-native triggers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The proof-of-concept agent (pnpm pipeline --watch) showed the
risk-ordered tiering buried the first genuinely CI-shaped increment
behind coordination machinery a single team never needs. Tiers are now
ordered by independent usefulness: records and UI; the pull-based
agent (one daemon, one repository, one branch — poll, materialize,
spawn, report, with the previously built revision as the selection
base); leasing and fleets as the agent's scaling form; triggers
narrowed to what only pnpr can provide (registry events, with
schedules kept and forge webhooks demoted to an optional ingress
adapter — watching sources is the agent's job); secrets last,
unchanged. The division of labor in one sentence: pnpr is memory and
registry events; the agent is eyes and hands. The topology paragraph
now states the execution boundary explicitly — every role is control
plane, and a build always runs in an agent process that holds nothing,
even sharing a box with pnpr. The engine RFC records the agent's
cache-key lesson: the runtime fingerprint resolves from the workspace,
never the invoking process.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The cleanest external proof of the thesis — zero-config CI because the
build tool's description is the pipeline, automatic caching because the
tool's cache decides — and the limit case of the selection argument,
since input-addressed sandboxed derivations make affected-detection a
non-concept. Recorded with the honest difference: it costs total Nix
buy-in and leads with hosted runners, both bets this sequence takes the
other way.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant