Skip to content

Artifact conformance: one shared pass, bound in every workflow that writes artifacts - #426

Open
m2ux wants to merge 18 commits into
workflow/403-creation-guidesfrom
workflow/403-artifact-conformance
Open

Artifact conformance: one shared pass, bound in every workflow that writes artifacts#426
m2ux wants to merge 18 commits into
workflow/403-creation-guidesfrom
workflow/403-artifact-conformance

Conversation

@m2ux

@m2ux m2ux commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Summary

The pass that measures a planning folder's artifacts against their guides existed as three near-identical copies, under work-package, workflow-design and workflow-authoring, and ran in those three workflows only. Thirteen workflows wrote artifacts with no conformance check at all, so whatever the writing register and the line budgets say, nothing detected drift. This work gives the pass one home under meta, parameterises the two maps it reads, and binds it in every workflow that has artifacts to check.

What changes

One home, no workflow's maps inside it. meta/techniques/verify-artifact-conforms.md takes the caller's canonical-home map and its planning-artifact-to-guide map as inputs, so the shared operation carries no workflow's policy of its own. artifact_dir defaults to the session planning folder, and a caller whose artifacts land elsewhere — prism and its report directory, codebase-wiki and its wiki path — binds that instead. The pass now checks the artifact writing register alongside the guide rules and the canonical-home map, and reports a filename that resolves to no guide as a finding against the folder.

Eleven more bind sites. prism, prism-audit, prism-evaluate, requirements-refinement, codebase-wiki, cicd-pipeline-security-audit, substrate-node-security-audit, midnight-system-review, work-packages and remediate-vuln each bind the pass at the boundary after their artifacts are written and before the gate or delivery that reads them. The three original callers retarget to the shared op and bind their own maps.

Two workflows do not bind it, for stated reasons

ponytail declares four artifacts and no directory to write them to — no planning folder variable, no output path, and no write step. A conformance step there would point at an unbound path, so the gap is recorded here rather than papered over with a binding that cannot run.

meta persists only the planning README, which verify-readme-conforms already checks against the universal template.

Scope of change

One new shared technique, three deleted copies, eleven activity bind sites, three contract citations retargeted, and the README and technique-index entries that named the deleted files.

Acceptance

  • The conformance pass lives in one shared home and carries no workflow's canonical-home map or guide map.
  • Every workflow that persists artifacts into a directory binds it — thirteen in total, up from three.
  • The pass checks the writing register, the guide each filename maps to, and the caller's canonical-home map.
  • All 22 guards pass. The shared operation's return contract is classified in the binding-fidelity triage under the existing shared-op-return-contract rationale, the same way every other meta library op is.

Landing order

The paired server PR carries the binding-fidelity triage entry and the two new guards, so it lands first; this branch is red against the current main guard set without it.

Non-goals

  • Authoring the guides the pass will look for. Those land in the creation-guides item.
  • Giving ponytail an artifact directory. That is its own change, and naming it here is the point.

Part of #403 (W5, corpus half). Stacked on #423.

…acts are written

The pass existed as three near-identical copies under work-package, workflow-design
and workflow-authoring, and ran in those three workflows only. It now lives once, at
meta/techniques/verify-artifact-conforms.md, and carries no workflow's maps of its
own: the caller binds its canonical-home map and its planning-artifact-to-guide map
as inputs, and artifact_dir defaults to the session planning folder so a caller
whose artifacts land elsewhere binds that instead.

Ten more workflows now bind it — prism, prism-audit, prism-evaluate,
requirements-refinement, codebase-wiki, cicd-pipeline-security-audit,
substrate-node-security-audit, midnight-system-review, work-packages and
remediate-vuln — each at the boundary after its artifacts are written and before
the gate or delivery that reads them. The pass checks the guide each filename maps
to, the caller's canonical-home map, and the artifact writing register.

ponytail declares artifacts with no artifact directory to write them to, so it has
nothing for the pass to check; that gap is recorded rather than papered over with a
step pointing at an unbound path. meta writes only the planning README, which
verify-readme-conforms already covers.

The workflow-design guide-map heading reads "to" rather than an arrow, so every
workflow's map answers to the same anchor.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
m2ux and others added 3 commits August 3, 2026 15:07
Three callers authored a guide map and did not pass it, so the pass fell back to a
filename search — the resolution order the map exists to win. All eleven maps are now
bound at their call sites.

The shared operation's contract loses three couplings it should never have had. Its
violations list enumerated one workflow's rule slugs, which contradicted the rule
next to it saying the operation carries no workflow's policy; the slug is now
whatever the guide, the map, or the register that owns the discipline calls it. The
default on artifact_dir read a designator no contract declared, so the meta root
contract declares planning_folder_path, which is where every technique reading
session artifacts already took it from. And a Protocol note restated that default
while attached to the wrong bullet, so it is gone.

The rule that replaced the deleted design-session copy said only what the Inputs
already said. It now carries the prohibition that copy existed for: measure a
workflow's artifacts against the maps the caller bound, never against a familiar map
it did not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Seven artifacts had a real guide the map did not name: the elicitation,
implementation-analysis, assumptions, test-plan, code-review, test-suite-review and
session-trace guides. Resolution reached them only by a filename search over resource
bodies, which reached resources that merely mention the file just as readily. The map
names them, so each resolves to the guide that owns its shape.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@m2ux

m2ux commented Aug 3, 2026

Copy link
Copy Markdown
Owner Author

An audit pass against the workflow canon found and fixed a delivery break plus twelve findings, pushed as follow-up commits on this branch and its base.

The break. Four corpus files the new creation-guide guard depends on — the cicd resources README and the prism, prism-evaluate and work-package guide maps — were authored while developing the guard and never committed. The guards were passing against a working tree rather than against what was pushed. Merged as-is, check-artifact-guides would have failed on the merged corpus. They are now committed on the creation-guides branch.

High findings

  • Seven prism map rows attributed an artifact's shape to a lens its producing pass does not dispatch: the reflect meta pass runs claim, not l12g; the verified initial and corrected passes both run l12, not knowledge-audit and oracle; the first adaptive stage runs deep-scan, not sdl-abstraction; and subsystem regions run whichever prism the assignment gives them. Each row now names the lens the pass actually dispatches, one row per artifact so a wrong lens is visible against the step that writes it. The two synthesis artifacts whose steps dispatch no lens are triaged as owing a guide rather than pointed at one that does not shape them.
  • The guard resolved a guide from any resource whose body mentioned the filename, so 14 of 139 artifacts passed on a false match — a guide's own traces-to line certified evidence-log.md, and the close-out guide certified assumptions-log.md and token-usage.md. Resolution now reads only the span where a resource declares what it is the guide for. The honest count is 21 artifacts owing a guide, up from 16, and the corpus side answered the difference by naming the seven work-package guides that already existed.

Also fixed

  • Three callers authored a guide map and did not pass it; all eleven maps are now bound.
  • The shared operation's violations list enumerated one workflow's rule slugs while the rule beside it said the operation carries no workflow's policy.
  • artifact_dir's default read a designator no contract declared; the meta root contract now declares planning_folder_path.
  • The rule that replaced the deleted design-session copy said only what the Inputs said, and now carries the prohibition that copy existed for.
  • A Protocol note restated a contract fact and hung off the wrong bullet.
  • The wiki index and log and the midnight findings register gained budgets a conformance pass can measure.

Recorded, not fixed: the line-budget class stays inert wherever a guide declares no budget; ponytail still has no artifact directory to bind the pass against; and the walk-snapshot corpus stamp re-baselines after merge.

Guards: 22/22. Suite green apart from that stamp.

m2ux and others added 10 commits August 3, 2026 16:19
The pass corrects in place, and binding it corpus-wide put it over folders holding
outputs whose bytes are a promise: the prism definitive findings that consumer
workflows parse instead of the raw pass artifacts, the audit reports whose sections a
split step renumbers and cross-references, the delivered security reports. Condensing
prose there breaks the numbering the producing step just fixed, and collapsing a table
removes a field a triggering workflow reads.

Such an artifact is now measured and reported with its fix status false, and left
alone; the shape gets corrected in the guide of the step that produces it. That is
also what the epic's own non-goal says about publishable outputs.

The conformance verdict says what it means alongside that. It read as true when the
violations array was empty after fixes, while the array holds every violation with its
fix status — so it could never be true once anything was found. It is true when every
entry was fixed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two of its rules reached for a consumer to explain themselves: the cap was stated once
here "and read by the summary", and the extra columns were the ones "the posted summary
section drops". A creation guide that names who reads its artifact couples a reusable
form to one caller's surface, so both now state the same constraint from the register's
own side — the cap is never recomputed elsewhere, and author class and blocker class are
columns of this register because the cap derives from them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twelve registers are read back by a later step and never by a person, and are markdown
on disk. Neither declaration is true of them: agent contradicts the format, since the
protocol makes JSON part of what agent means, and human contradicts who reads them.
They now carry no audience at all, which is what absent is for — it already means human
by default, so the file reads the same, and no declaration claims a reader the form
contradicts.

The registers are the prism run manifest, the change-block index, the provenance and
evidence logs, the assumptions log, both findings registers, the structural inventory,
the drafting plan, the file review note, the prior-feedback triage, and the debt ledger.
The follow-ups and deferred-items registers have no declaration to leave absent, since
no technique declares either as an artifact.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…g in

The summary report, detailed findings and trade-off analysis each had their whole
layout recipe in the producing technique's protocol: the reference line and the
renumbering it forces, the five inherited fields and the ID heading form, the three
ordered parts of the catalogue. Each is now a guide with a template, rules and a
budget, and the protocol cites it.

These are the reports consumer workflows read, so the rules keep the contract explicit
rather than implied: severities and IDs are inherited verbatim, fields are copied not
re-derived, a lifted section renumbers what follows, and a law the adversarial pass
rejected stays absent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each register in the pending set was reviewed one at a time. Eight are approved for
conversion and stay undeclared until their format changes. Five are kept as prose, and
four of those are declared artifacts, so they carry a settled human reader again: the
change-block index and assumptions log, and both findings registers, whose own guides
document a person reading them alongside the later step.

The follow-ups and deferred-items registers are kept as prose too, and no technique
declares either as an artifact, so there is nothing to declare on them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… own

The run manifest carried its whole skeleton as a fenced block in protocol prose, and the
portfolio and reflect syntheses described their sections inline. Each now has a guide
with a template, rules and a budget, and its producing step cites it.

The manifest's rules keep the part consumers depend on explicit: every unit is a row,
the flat artifact list is complete, and status reflects the filesystem rather than the
intent, so an incomplete run is never reported as success. The portfolio synthesis
records divergence as a result rather than a disagreement to resolve. The reflect
synthesis keeps its four sections, all of them present, and stays about the scan rather
than the target.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The change surface, investigation plan and evidence log each had their layout listed in
the producing step: which fields the inventory records, what the plan's area table and
coverage summary hold, how the evidence accounting and discharge records are arranged.
Each is now a guide with a template, rules and a budget, cited from the step.

The rules keep the parts a later step depends on: refs are recorded rather than implied
so a reader can tell whether the inventory still describes the branch, every failure
class is discharged or marked not-applicable with a reason, a gated probe stays planned
and flagged, inconclusive is its own verdict and never relabelled refuted, and every
evidence item carries the ID adjudication cites it by.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tifacts

ponytail declared four artifacts and had no directory to write them into: no folder
variable, no output path, and no write destination in any step. It now declares
artifact_dir, defaulting to the target itself so a pass with no separate folder keeps
its artifacts beside the code they describe, and each producing step writes there.

The brief, change note, audit findings and debt ledger each get a guide. Two of them
carry a line grammar the ledger and the findings list are totalled by, so the grammar
moves into the guide where the format rules live rather than staying in protocol prose:
one line per finding ranked biggest-cut-first with a net scoreboard, and one row per
marker with its ceiling and upgrade trigger, a missing trigger flagged rather than
omitted.

With a directory to check, the artifact-conformance pass binds here too, which was the
one workflow it could not reach.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tirely

The token-usage ledger, provenance log and ADR each carried their whole layout in the
producing step; the change-block index already had a home in the manual-diff-review
resource and only needed mapping. All four now resolve through a guide, so every one of
the 139 persisted artifacts in the corpus maps to a template with rules and a budget.

The rules keep what a reader depends on: a token total is a total only when coverage is
complete and a floor otherwise, cost is always labelled an estimate, no usage means no
artifact rather than a fabricated figure, the provenance header is fixed so rows stay
comparable, and an ADR carries at least one rejected alternative with the reason it lost.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ds everywhere

The artifact-conformance pass checks a line budget only where a guide declares one, so
the check sat inert across most of the corpus. All 104 guides a map row cites now carry
one, and 24 of them gain the Rules section they had no home for it in.

Each ceiling is derived from the guide's own template rather than picked: a fixed-shape
artifact gets a whole-file budget proportionate to its skeleton, and an artifact that
grows with its subject gets a per-item one — per finding for the review and code-review
reports, per row for the session trace and the follow-ups register, per block for the
change-block index, per requirement for the specification, per area for a comprehension
document. Each budget says what over-budget content usually is, so the fix is a link
rather than a shorter paraphrase.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@m2ux

m2ux commented Aug 4, 2026

Copy link
Copy Markdown
Owner Author

Scope grew after review — every persisted artifact now has a guide.

This PR originally covered the ten artifacts whose shape was prescribed nowhere at all, leaving a further set whose shape sat inline in protocol prose to a classified triage. That triage is now closed: guides were authored for all of them, and scripts/artifact-guide-baseline.json is deleted.

Seventeen distinct artifacts gained a guide in total. Beyond the original ten:

Workflow Artifacts
prism-audit AUDIT-REPORT.md, DETAILED-FINDINGS.md, DESIGN-TRADE-OFFS.md
prism RUN-MANIFEST.md, portfolio-synthesis.md, reflect-synthesis.md
midnight-system-review change-surface.md, investigation-plan.md, evidence-log.md
ponytail lean-brief.md, lean-change.md, audit-findings.md, debt-ledger.md
work-package token-usage.md, provenance-log.md, the ADR record; change-block-index.md mapped to the guide that already owned its shape

The published reports needed no new design work — their consumer contracts were already written into the producing techniques' Rules, so the guides are a lift plus a cite. Where the shape genuinely belongs to something else, the map says so rather than inventing a guide: a per-lens prism artifact takes its shape from the lens the unit runs, and the map names the lens each pass actually dispatches.

Two further changes ride along:

  • Line budgets everywhere. The conformance pass checks a budget only where a guide declares one, so the check was inert across most of the corpus. All 104 guides a map row cites now carry one, derived from the guide's own template — whole-file where the shape is fixed, per item where the artifact grows with its subject.
  • ponytail gained an artifact directory. It declared four artifacts with nowhere to write them; it now declares artifact_dir and binds the conformance pass like every other workflow.

m2ux and others added 4 commits August 4, 2026 07:32
The conformance pass took every markdown file in its artifact directory as in scope and
corrects in place. That is right for a planning folder, which holds nothing else, and
wrong for any directory the run shares: ponytail's artifact_dir defaulted to the target
itself, so a default pass would have measured every README and doc in the repository
under review against guides they were never written to, and rewritten them against them.

Scope is now the artifacts the run persisted, with a rule saying so, and ponytail writes
its four artifacts into a dedicated folder under the target where they are identifiable
as the pass's own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Scoping the pass to what the run wrote dropped the markdown restriction it used to carry,
so the JSON artifacts a security-audit run persists came into scope of a check built for
prose. A template and a line budget say nothing about structured data, and correcting one
against them corrupts it, so scope is the run's human-audience artifacts and an agent
artifact's conformance is its declared schema.

The wiki page guide gains the budget the sweep missed. Its per-type skeletons are named
Concept and Entity rather than Template, so nothing detected it as a guide with a shape
to bound.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The pass corrects in place, so a check that ran before it measured content the pass then
changed. Five bind sites had that order. The wiki lint counted findings and the user
adjudicated them at a checkpoint after the pass had rewritten the pages those findings
described — and lint deliberately never picks a side, which the pass does. The midnight
accounting gate validated that per-area probe and finding counts reconcile, and a pass
that collapses an all-pass table or drops a section with no findings removes exactly those
rows. The prism-audit consistency check, the prism-evaluate report check, and the
work-package README check each measured a state the pass could still change.

Conformance now runs first at all five, so every check measures the bytes that ship and
every finding a user adjudicates describes the artifact as it stands.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two of the fourteen sites named the step something else: workflow-authoring called it
verify-planning-artifacts and workflow-design reused the technique's own name. All
fourteen now read verify-artifact-conformance, so a sweep over the bind sites finds them
all and a step id says the same thing wherever it appears.

The only consumer of a step id beyond history is delivery idempotency, keyed on activity,
step and agent. A saved session mid-activity therefore re-delivers this step once rather
than matching its recorded event, which an in-place conformance pass absorbs — a
conformant artifact gets no change. Saved sessions keep the step id they recorded, which
is what the step was called when they ran.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant