Skip to content

S8 (#189) — a correction procedure for the delivery that exists - #206

Merged
Polichinel merged 1 commit into
developmentfrom
feat/s8-correction-procedure
Aug 2, 2026
Merged

S8 (#189) — a correction procedure for the delivery that exists#206
Polichinel merged 1 commit into
developmentfrom
feat/s8-correction-procedure

Conversation

@Polichinel

Copy link
Copy Markdown
Collaborator

Closes #189. Closes #15. Closes register C-22 — with one step explicitly awaiting an operator decision. Part of epic #181.

Why this was overdue

C-22's trigger fired on 2026-07-27. Run-0 put 64,742 cells × 36 months into the UN FAO's store, live and being served. The written procedure — #15, from June — began "clear stale disk cache" and "re-run with corrected shapefiles". Both were deleted with the runtime mapper.

There was data in a partner's store and no usable way to correct it.

What the document establishes

Identification. run_id and lookup_version, both present by construction. This is why S8 waited on S5: until C-60, lookup_version could silently be the string "unknown", and a procedure whose first step rests on a field that can become a placeholder is not a procedure.

Confirmation, offline. 26 fidelity tests against committed artifacts. Re-running the pipeline to investigate changes the thing being investigated.

And where the fault is not ours. If those pass and a value is still wrong, it is upstream — views-datafactory#387. C-43's scope split, restated where an operator meets it at 22:00 rather than buried in a register entry.

Supersession, not retraction

The contract has no retraction primitive, and the document says so instead of inventing one. Manifest-last commit ordering means a run is replaced by publishing a new complete run.

It also states the thing a reader would otherwise discover the hard way:

views-faoapi resolves the newest manifest matching a broad filter. A corrected run is picked up because it is newer, not because it is correct.

Two consequences spelled out: publishing is the switch-over, and do not publish a test or partial correction to the production bucket — the consumer cannot tell it from the real one. That is C-73, cited rather than re-solved, with #133 named as the fix that would let a consumer select on intent.

Plus the one that would turn a correction into an outage: do not delete the superseded run's files while its manifest still stands — a run that resolves and then fails to load is worse than a wrong value.

The step that is written but not decided

Steps 1–3 and 5 are executable today. Step 4 is not, and it is the one that reaches the partner. Stated verbatim in the document, with an instruction to stop and ask rather than improvise:

1. Who contacts the UN FAO when a delivery is found wrong, through what channel, and how quickly?
No named person, no address, no timing expectation — so it would be improvised by whoever noticed, under time pressure, which is the worst moment to invent a process.

2. Does FAO expect us to retract or to supersede?
Supersession is what the contract does. Retraction has no wire mechanism, and adding one would need an ADR-013 amendment plus agreement from views-faoapi. It is a question for them, not a decision for us.

C-22 closes because the procedure exists and says exactly where it stops. What remains is a decision, not engineering; leaving the entry open would read as unfinished work rather than as a question awaiting an answer.

Carried forward

docs/CLONING.md now warns a clone to answer its partner's correction questions before first delivery. This repo shipped run-0 with that step undecided and it still is — views-crafdapi and views-productionapi should not inherit it.

README.md points at the procedure from the operational surface.

Pinned

Three checks in tests/test_doc_accuracy.py: the document exists and names the identification fields; it describes ADR-013 mechanisms and none of the deleted ones (it is also inside _BANNED's scan); and it still flags its undecided step — so if step 4 is ever answered, the marker has to be replaced with the decision rather than quietly dropped.

Verification

ruff check .                                     # clean
pytest -q                                        # 305 passed, 3 skipped, 41 xfailed, 5 known pyarrow
pytest -q tests/test_doc_accuracy.py             # 13 passed
git diff --stat origin/main -- tests/fixtures/   # empty

Register: 74 concerns, 15 open, 59 resolved.

🤖 Generated with Claude Code

C-22's trigger fired on 2026-07-27. Run-0 put 64,742 cells x 36 months into the
UN FAO's store, live, and the written procedure (#15, June) described disk caches
and shapefiles — both deleted with the runtime mapper. There was data in a
partner's store and no usable way to correct it.

Affected deliveries are identified by run_id and lookup_version, both present by
construction. That is why this story waited on S5: until C-60, lookup_version
could silently be the string "unknown", and a procedure whose identification step
rests on a field that can become a placeholder is not a procedure.

The wire mechanism is supersession, not retraction, and the document says so
rather than inventing one — manifest-last commit ordering means a run is replaced
by publishing a new complete run. It also states the consequence a reader would
otherwise find the hard way: views-faoapi selects the newest manifest over a broad
filter, so a correction is picked up because it is NEWER, not because it is
CORRECT, and a partial correction published to production is indistinguishable
from the real one. That is C-73, cited rather than re-solved.

Confirmation is offline against committed artifacts, because re-running the
pipeline to investigate changes the thing being investigated. And if the fidelity
suite passes while a value is still wrong, the fault is upstream — C-43's scope
split, restated where an operator will meet it at 22:00.

One step is written but NOT decided, and it is the step that reaches the partner:
who contacts the UN FAO and whether they expect retraction or supersession. Both
belong to the operator. The document states them verbatim and tells the reader to
stop and ask rather than improvise — an inconsistent first message to a partner is
harder to correct than a slow one.

C-22 closes because the procedure exists and says exactly where it stops. What
remains is a decision, not engineering, and leaving the entry open would read as
unfinished work rather than as a question awaiting an answer.

docs/CLONING.md carries it forward: answer your partner's correction questions
BEFORE first delivery. This repo did not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant