Skip to content

fix(ci): publish every workspace package, and stop a missing npm record stranding a release - #1278

Merged
lex00 merged 8 commits into
mainfrom
fix/publish-pipeline-auth
Jul 31, 2026
Merged

fix(ci): publish every workspace package, and stop a missing npm record stranding a release#1278
lex00 merged 8 commits into
mainfrom
fix/publish-pipeline-auth

Conversation

@lex00

@lex00 lex00 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

The failure

Every publish run since chant-v0.33.0 has failed — 8 consecutive red runs.

@intentius/chant-k8s-client (#1177) and @intentius/chant-lexicon-fountain (#1253) each landed a publish step in publish.yml, but neither package ever got an npm trusted-publisher record. npm publish fell through to ENEEDAUTH while the other twelve published over OIDC in the same job, on the same token.

The registry confirms it. Every chant-lexicon-fly version carries a SLSA provenance attestation; chant-k8s-client 0.31.0/0.32.0 and chant-lexicon-fountain 0.33.0 carry none. Those two only ever went out by hand.

Both are still stranded — k8s-client at 0.32.0 and fountain at 0.33.0, against 0.33.1 everywhere else. On chant-v0.33.0 the failure also skipped all twelve lexicons queued behind it and half-published the release.

The fix

scripts/publish-packages.sh replaces the fourteen copy-pasted steps:

  • Enumerates every non-private workspace package, so a new package is published the moment it exists instead of waiting on someone to remember the step. This is the failure mode that bit twice.
  • Publishes core, then the k8s API client (the k8s lexicon declares it optional, feat(k8s): typed API client — retire the kubectl shell-outs #1074), then the rest.
  • Skips a package already at its version, so re-runs stay idempotent and a partial release is recovered by dispatching again.
  • Never stops on the first failure. Every package gets its attempt; the run exits non-zero at the end with a summary table.
  • On an auth failure only — never a build error — retries with the NPM_TOKEN secret already present and unused in the repo, via a throwaway userconfig so the token never reaches the process table.

The token is a fallback, not the path. A package with a trusted-publisher record still publishes over OIDC and never touches it.

chant onboard no longer adds a per-lexicon publish step, since enumeration makes it unnecessary. Its two docs tables are updated to match.

Also: emulator-freshness

Red every week since 2026-07-20. gh issue create --label chore failed with could not add label: 'chore' not found, and gh declines to create the issue at all when a label is missing — so the advisory issue was never filed. The workflow now creates the label idempotently first.

Testing

All four publish paths validated against a stubbed npm:

path result
happy path publishes only the 2 behind, skips 12, exit 0
ENEEDAUTH + token recovers, exit 0, prints the trusted-publisher hint
ENEEDAUTH, no token fails clearly, still visits all 14, exit 1
non-auth failure no token retry attempted, exit 1

tsc, eslint, and the 14 tests coupled to these files (onboard.test.ts, release-wiring.test.ts, upgrade-wiring.test.ts) pass.

Note on lexicon-upgrade

Also investigated — red weekly since 2026-07-13 with Op "fly-upgrade" not found. Already fixed by fa808db (#1226) on 07-29, after the last scheduled run on 07-27. No change needed here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JTHr67zuAjGHJAX6gF5tYA

lex00 and others added 2 commits July 31, 2026 00:44
…rd stranding a release

Every publish run since chant-v0.33.0 has failed. @intentius/chant-k8s-client
(#1177) and @intentius/chant-lexicon-fountain (#1253) each landed a publish
step in publish.yml but neither package ever got an npm trusted-publisher
record, so `npm publish` fell through to ENEEDAUTH while the other twelve
published over OIDC in the same job on the same token. The proof is in the
registry: every chant-lexicon-fly version carries a SLSA attestation, and
those two carry none — they only ever went out by hand. Both are still
stranded, k8s-client at 0.32.0 and fountain at 0.33.0 against 0.33.1
everywhere else, and on chant-v0.33.0 the failure skipped all twelve lexicons
behind it and half-published the release.

Replace the fourteen copy-pasted steps with scripts/publish-packages.sh. It
enumerates every non-private workspace package, so a newly added package is
published the moment it exists rather than waiting on someone to remember the
step. Order is core, then the k8s API client (the k8s lexicon declares it
optional, #1074), then the rest. A package already at its version is skipped,
which keeps re-runs idempotent and makes a partial release recoverable by
dispatching the workflow again. Failures no longer stop the loop; every
package gets its attempt and the run exits non-zero at the end with a summary
table.

On an auth failure — and only an auth failure, never a build error — it
retries with the NPM_TOKEN secret that was already in the repo and unused,
via a throwaway userconfig so the token never reaches the process table. That
is a fallback, not the path: a package with a trusted-publisher record still
publishes over OIDC and never touches the token.

`chant onboard` no longer adds a per-lexicon publish step, since enumeration
makes it unnecessary, and the two docs tables describing it are updated.

Also fix emulator-freshness, red every week since 2026-07-20. `gh issue
create --label chore` failed with "could not add label: 'chore' not found",
and gh declines to create the issue at all when a label is missing, so the
advisory issue was never filed. Create the label idempotently first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHr67zuAjGHJAX6gF5tYA
…ration

Dispatching publish on this branch (run 30610577415) proved the script's
control flow and turned up two things the stubbed tests could not.

The trusted-publisher records for chant-k8s-client and chant-lexicon-fountain
are genuinely absent — the OIDC attempt still returns ENEEDAUTH. The token
fallback then authenticated but failed EOTP: the NPM_TOKEN secret is a
classic publish or granular token, not an automation token, so npm demands a
one-time password that no one is present to type. That is indistinguishable
from a permissions problem in the raw log, so say it outright and name both
ways out.

The provenance retry also misfired. It scanned the whole publish output, and
`npm notice` narrates provenance on every run — so an EOTP failure looked
like a provenance rejection and got retried for nothing. Match npm's `npm
error` lines instead, which is what the check meant all along.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHr67zuAjGHJAX6gF5tYA
npm's oidc helper is written never to throw: when the token exchange fails it
logs at verbose level and returns undefined, so `npm publish` falls through to
ordinary auth and reports a bare ENEEDAUTH. That one error covers two very
different situations — no trusted-publisher record at all, versus a record
that exists but does not match this workflow's OIDC claims. The previous
message asserted the first, which is a guess.

Replay the exchange against
/-/npm/v1/oidc/token/exchange/package/<name> and print the registry's own
status and message. Neither the GitHub id-token nor the publish token a
successful exchange returns is ever printed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHr67zuAjGHJAX6gF5tYA
@lex00
lex00 force-pushed the fix/publish-pipeline-auth branch from 5bdf164 to a00cb10 Compare July 31, 2026 07:23
A controlled probe run from inside publish.yml settled which packages
actually have trusted-publisher records:

  @intentius/chant-lexicon-fly       HTTP 201 accepted
  @intentius/chant                   HTTP 201 accepted
  @intentius/chant-k8s-client        HTTP 404 package not found
  @intentius/chant-lexicon-fountain  HTTP 404 package not found

Same workflow, same ref, same run — so the split is per package, and the two
that fail have no record rather than a mismatched one. Probing from any other
workflow file 404s for all four, since the record is tied to a workflow
filename; that is why the diagnostic lives here and not in a scratch workflow.

The probe also caught a bug in the diagnostic added a commit ago: it treated
only HTTP 200 as success, so a working record would have been reported as
refused. Accept any 2xx.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHr67zuAjGHJAX6gF5tYA
@lex00
lex00 force-pushed the fix/publish-pipeline-auth branch from a00cb10 to e88822f Compare July 31, 2026 07:24
lex00 added a commit that referenced this pull request Jul 31, 2026
…ent applies

Two gaps that only showed up once something asked the graph a question it had
not been asked before.

A resource nothing declares and nothing references is invisible to every
observation path, so an answer about "my security groups" can be complete for
the declared estate and still not be the answer the question wanted — and the
result gives no sign of it, because a partial set looks exactly like a whole
one. `search` now says that `--ambient` applies to the kind just queried.
Lexicons declare which kinds those are (`ambientKinds`), so it costs no scan to
mention, and the note carries no count and names no resource: it says the flag
is relevant, not what the flag would find. VPCs join security groups on the AWS
list — the account's default VPC is the archetype of a resource nothing
declares that questions are nonetheless about.

Separately, `describe-stack-resources` returns identity and status and nothing
about the resource, so the managed observation had been filling `attributes`
with the *stack's* outputs, copied onto every member. Every node in a stack came
out carrying the same expVpcId/expWebIp keys and none carried its own VpcId.
`describeOwnProperties` reads each kind once in bulk and joins back by physical
id; stack outputs stay, but a resource's own property wins its name.

That fix alone changed nothing visible, because `sourceOverlayGraphs` was
dropping observed attrs on the floor — it copied physical identity onto the
declared canvas and left behind everything observed *about* the resource. So
`search --show VpcId` printed six blank columns, which reads as an estate with
no VPCs rather than a read that never happened. Observed values now land, and
win a name collision: the declared side holds an unresolved reference to another
entity, the account holds the id.

Refs #1278, #1279
lex00 added a commit that referenced this pull request Jul 31, 2026
…hat was declared

"Which of my security groups are unused" is, in AWS's own terms, a question
about network interfaces: a group is in use exactly when an ENI attaches it.
Nobody writes an ENI — EC2 creates one per instance — so it was in none of the
observation paths. Declared: no. Referenced: no, an instance does not name its
interface. Ambient: no, because the ambient scan is bounded by the kinds the
project declares.

That bound is right and it excluded precisely the resource that answers the
question. Asked it, an agent queried `kind:EC2::NetworkInterface`, got nothing,
and rebuilt the attachment map from twenty-nine raw provider calls — over half
of all the account reads in that run.

So the bound widens by implication rather than being abandoned: declaring an
instance is declaring its network interface. Still nothing about the account at
large. The catalog learns what an ENI points at — its groups, its instance, its
subnet — so the edges exist and the question is a query again:

    chant search "kind:EC2::SecurityGroup !<-kind:EC2::NetworkInterface" --at latest

which returns the four unattached groups for the reason they are unattached,
rather than via `!<-kind:EC2::Instance`, which agreed with the right answer on
this estate by coincidence and would not on one with load balancers or RDS.

Refs #1278
lex00 and others added 2 commits July 31, 2026 12:02
…m record

Audited every workspace package's trusted-publisher record from inside
publish.yml, where the workflow filename matches and the answer is therefore
meaningful. Twelve are configured; @intentius/chant-k8s-client and
@intentius/chant-lexicon-fountain are not. Those are exactly the two that
cannot publish.

The reason they were missed is here: the onboarding page walks through every
file `chant dev onboard` patches and every manual step after it, and never
mentions that a new package needs its own trusted-publisher record on npm.
The record is per package, is not inherited from the org, and nothing in the
repo can create it — so a new lexicon ships unpublishable and the failure
surfaces as a bare ENEEDAUTH much later, at release time.

Two further things on that page were simply wrong:

  * It said releases are triggered by pushing a `v*` tag. The workflow matches
    `chant-v*` and `lexicon-*-v*`; a bare `v<version>` tag triggers nothing.
    Point at `just release` / `just release-lexicon`, which push the real tags.
  * It said the workflow passes `--tolerate-republish`. No such flag is used
    anywhere, and npm has no such option. The actual behaviour is the
    skip-if-already-at-this-version check in scripts/publish-packages.sh.
    TESTING.md repeated the same claim in its risk table; corrected there too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHr67zuAjGHJAX6gF5tYA
The trusted-publisher step went missing because the knowledge was spread
across a page about what `chant dev onboard` patches, a release section that
was itself wrong, and nothing at all. An author working with an agent had no
single thing to hand it.

Add Lexicon Onboarding Skill: one self-contained ordered checklist from
scaffold through first publish, written to stand alone so an agent pointed at
just that URL can finish the job. It leads with the step nothing in the repo
can automate — creating the npm trusted-publisher record — because that is
the one whose absence merges cleanly and only surfaces at release time.

The overview and CI pages now lead agents into it rather than expecting them
to assemble the sequence from the section.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHr67zuAjGHJAX6gF5tYA
lex00 added a commit that referenced this pull request Jul 31, 2026
Every account arrives with resources nobody wrote — a default VPC and its
subnets, a default security group per VPC, a main route table, AWS-managed KMS
keys and IAM policies. chant recorded them looking exactly like everything else,
so nothing downstream could tell "this is yours and unattached" from "this came
with the account".

Asked which security groups were unused, agents split three ways on the same
correct set of four: one excluded the three VPC defaults as not real cleanup
candidates and answered one, another counted something extra and answered five.
Every answer that was accepted did the same thing — listed all four and said
which were defaults. They could only do that by recognising them, and chant gave
them nothing to recognise them by.

Almost none of this is chant's judgement. AWS marks these itself, on payloads
chant already reads, and the fields were simply being dropped: `IsDefault` on
VPCs and network ACLs, `DefaultForAz` on subnets, `Main` on a route table's
associations, `KeyManager` on KMS keys, the `aws` account in a managed policy's
ARN. The one derivation is the security group, and it holds because `default` is
a reserved group name that AWS refuses on create.

`providerDefault: true` says the provider created this and stops there. It does
not say unused, safe to ignore, or exempt — deciding that here would put a
conclusion in an observation, and it would be wrong in this very case, since the
accepted answers count defaults as unused. Absent means "not a default, or chant
cannot tell", which is different from false.

Refs #1278, #1280
lex00 and others added 2 commits July 31, 2026 12:39
Twice now a package has shipped without an npm trusted-publisher record and
nobody found out until a release went red, because the only symptom is a bare
ENEEDAUTH at publish time. Verifying it by hand means hand-editing publish.yml
each time, since npm matches the record against the workflow FILENAME and a
probe from anywhere else reports every package as missing.

Add an audit job with no `needs:`, so dispatching the workflow puts a coverage
table in the run summary within about forty seconds, well before the test gate
finishes. Informational only — the publish job remains what fails a release.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHr67zuAjGHJAX6gF5tYA
`chant dev onboard` adds `@intentius/chant-lexicon-<name>` to the root
package.json dependencies, but fly, forgejo and fountain were never added.
Nine lexicons were listed, three were not, and nothing checked — the same
shape as the lexicon-upgrade miswiring (#1218/#1226) and the publish steps
that stranded two packages.

Nothing broke, because resolution never depended on that list:
`workspaces: ["lexicons/*"]` symlinks every lexicon into node_modules whether
it is listed or not, which is exactly why three omissions survived unnoticed.
The entry is an explicit declaration of what core may import, not the
mechanism that makes the import work.

Complete the list and add lexicon-wiring.test.ts so it either covers every
lexicon or fails. Verified the test catches the real drift by removing
fountain again and watching it go red.

The CI page claimed the dependency was what made workspace resolution work,
and that a lexicon missing from it "fails when loaded by the CLI". Neither is
true. Corrected to describe what the entry is actually for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHr67zuAjGHJAX6gF5tYA
@lex00
lex00 merged commit 8f835b1 into main Jul 31, 2026
6 checks passed
@lex00
lex00 deleted the fix/publish-pipeline-auth branch July 31, 2026 21:18
lex00 added a commit that referenced this pull request Jul 31, 2026
…rified commit (#1281)

Closes the two in-repo halves of #1255. The third (a trusted-publisher record
for @intentius/chant-k8s-client) is done on npmjs.com — the package published
0.33.1 over OIDC with provenance in #1278.

## Peer ranges (#1255.2)

`just release` rewrites the `@intentius/*` peer ranges alongside `.version`,
added in #411 because ranges frozen at an old version break clean installs.
`release-lexicon` only ever set `.version`, so a single-lexicon patch that
needed a newer core silently shipped a stale range.

The two recipes cannot share a literal. A whole-repo release moves everything
together, so `^$next` is correct there; a single-lexicon patch leaves core
where it is, so its ranges must track the CURRENT core and github-lexicon
versions. Pinning them to `$next` would demand a core that does not exist.
The test asserts that mechanism rather than a shared value.

`release-lexicon` also now runs `npm install --package-lock-only` and commits
the result, for the same reason `just release` has since #1094 — a bump that
leaves the lockfile recording the old version is the same defect.

## Preflight (#1255.3)

Both recipes pushed the bump commit straight to main, bypassing branch
protection, so the commit a release tag pointed at was one CI never ran — and
nothing checked that the code being released was green either.

scripts/release-preflight.sh, shared by both recipes rather than copied into
each, refuses to tag unless the tree is clean, HEAD matches its pushed branch,
and the `chant` run for that commit concluded success. `release` additionally
requires main, since it pushes main explicitly. `CHANT_RELEASE_SKIP_PREFLIGHT=1`
is the emergency opt-out.

Exercised every path: dirty tree, wrong branch, unpushed HEAD, CI still
running (caught main mid-run and printed the run URL), and the opt-out.

The guards live in release-wiring.test.ts, where #1255 said they belong, and
each was verified to fail by reverting the fix it covers.


Claude-Session: https://claude.ai/code/session_01JTHr67zuAjGHJAX6gF5tYA

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
lex00 added a commit that referenced this pull request Aug 3, 2026
A subnet holding nothing is the same shape of question as an unused security
group, and #1278 left it unreachable. `describeResources` and
`observeDependencies` both resolve outward from what is declared, so a subnet is
recorded only when something in it is recorded — and a subnet with nothing in it
is exactly what "which of my subnets are empty" is asking about.

Found by measuring a snapshot rather than reading the code. An estate with seven
subnets recorded five, and the two it dropped were the two with nothing in them.
The default VPC's subnets are the archetype the ENUMERABLE comment already
names: nothing declares them, and the only one ever recorded was the one that
happened to hold an instance.

VPC was already enumerable and behaved correctly, which is why this looked like
a snapshot-wide blind spot and was in fact one missing table entry.

The test fails without the entry — checked by removing it.


Claude-Session: https://claude.ai/code/session_01KHf72NPSxiR8wMqGiRkCmf

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant