feat(crafd): CRAF'd producer package — clone of unfao/ (per CLONING.md) - #211
Merged
Conversation
Stand up the views-postprocessing producer for the second consumer, CRAF'd (views-crafdapi). Per CLONING.md this is "replace three, keep everything else": - crafd/product.py — TARGETS = the same three GED fatality series (CRAF'd is FAO extended, same forecasts/geography; additional targets are a later Amendment A1 edit), CONSUMER_DOCUMENT_NAME = "un_crafd" (ADR-034 §6), S_MIN = 2, UPLOAD_ENABLED = False (§11.4 interlock — staged locally, zero store calls, until the views-crafdapi selection guard is deployed). - crafd/appwrite_env.py — CRAFD_ENV (APPWRITE_CRAFD_* coordinates, filled by the operator at views-crafdapi S9 / views-appwrite #38); connection + prod-forecasts env shared with FAO (same internal shelf source). Contract pinned at 1.3.0 / 47172af (same edition as unfao; the CRAFD names are reserved there). - crafd/managers/crafd.py — CRAFDPostProcessorManager: the pipeline-core seam, reads the shared production_forecasts shelf and delivers the same wire to crafd_bucket. Mechanical retarget of the FAO manager; no partner-neutral machinery touched. - test_doc_accuracy: the pipeline-core-importer guard now expects BOTH partner managers (unfao + crafd) — a new partner manager is expected, not blast-radius creep; anything OUTSIDE the managers still fails. VALIDATION: ruff-clean and AST-parses locally; the FULL suite is validated by CI, because this machine (Python 3.13) cannot build the repo's pinned pyarrow 16.1.0 (no 3.13 wheel) — CI runs the correct 3.11/3.12 + 16.1.0 toolchain. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Polichinel
force-pushed
the
feat/crafd-producer
branch
from
August 3, 2026 02:11
13d5327 to
9799e87
Compare
`crafd/` landed exempt from eight guards, because each named `unfao` as a string. Five review passes; each found defects in the previous pass's fixes. The defect crafd shipped with - `crafd/appwrite_env.py` pinned registry v1.3.0/47172af — an edition at which its own four APPWRITE_CRAFD_* coordinates had no values, while the file's docstring cited the PR that filled them. Re-pinned to v1.4.1/90fc105. What was unguarded, and is now - partner list declared ONCE (`tests/conftest.py:PARTNER_PACKAGES`), asserted against the filesystem, and consumed by every guard that needs it - þing-01 dotenv borrow: covered 2 files of 30. A live `load_dotenv(find_dotenv())` in `unfao/appwrite_env.py` — the entry validator whose docstring says the borrow is dead — ran on import with the suite green. Now a package-wide AST walk, which sees imports and calls and cannot see the prose that legitimately describes it - ADR-002 both legs: `contract/` could import `crafd`, and `delivery/ -> contract/` had no test at all. Both are now AST import walks plus a subprocess import-purity check. Regexes escaped three times (relative, bare-package, function-body) before being replaced - line budget: the last hardcoded list; a 906-line third manager passed it. Derived, and applied to the whole `managers/` directory - consumer document name: nothing asserted crafd's. Pinned, plus a gated cross-seam check reading views-crafdapi's own `APIPathManager` literal and its filter field - `_MACHINERY` had no completeness assertion, so `contract/enrichment.py` was exempt from both purity guards Production code - `_ContractStorePort.upload`'s comment claimed the pipeline-core store "only LOGS" metadata failures. It returns `success=False, code="PARTIAL_SUCCESS"`. The comment was self-defeating: if it only logged, the guard it explains could never fire Records disposed of in the same change (ADR-014 §5) - ADR-012 ontology, CLONING.md, role_and_seams, the CIC, unfao/managers/README, README tree, `contract/__init__` — all said "the" partner or "only one" importer - CLONING.md over-claimed þing-02 S24(5) as a flat prohibition; the verdict binds the repos cut from views-faoapi, not an in-repo partner package of the producer. It also dropped #146's named trigger and asserted a tracker state #146 did not have (commented) - register: C-77, C-78, C-79 added; C-33, C-40, C-47, C-57 amended; 76/17/59 -> 79/19/60 Also - retired `test_version_bumped_past_latest_tag`: it asserted a version is NOT tagged, so it went red whenever views-datafactory was in a clean released state. Its purpose is covered by `test_land_gaul_commit_is_in_a_release_tag` - `sibling_repo` briefly raised on a bad override and turned a typo into "0 tests run". Moved to its own assertion - C-33's measurement was wrong six times, twice by edits that shifted lines it had just counted. No entry states manager line numbers now — they name symbols, and the docs state the `diff` command instead of its result 361 passed, 40 xfailed, 0 failed (conda env, pipeline-core 3.0.0 editable, pyarrow 16.1.0); ruff clean. Every guard mutation-proven; ~25 mutations, all reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
Ready to merge (MERGEABLE, CI CLEAN) — and it's now on the critical path for standing up views-crafdapi (released v0.1.0). Flagging the sequencing so the operator can route it:
Read-side contract already verified against this package: crafdapi filters the store on model name Not merging it myself — this is your repo; over to you (or the owning session) to merge. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stands up the views-postprocessing producer for the second consumer, CRAF'd (views-crafdapi) — the thing that will deliver CRAF'd forecasts into
crafd_bucket. PerCLONING.mdit's "replace three, keep everything else."What
crafd/product.py—TARGETS= the same three GED fatality series (CRAF'd is FAO extended — same forecasts/geography for now; extra targets are a later Amendment A1 edit),CONSUMER_DOCUMENT_NAME = "un_crafd"(ADR-034 §6),S_MIN = 2,UPLOAD_ENABLED = False(§11.4 interlock — staged locally, zero store calls, until the views-crafdapi selection guard is deployed).crafd/appwrite_env.py—CRAFD_ENV(theAPPWRITE_CRAFD_*coordinates the operator filled at views-crafdapi S9 / views-appwrite CIC for ReconciliationModule + migration docs #38); connection + prod-forecasts env shared with FAO (same internal shelf source). Contract pinned v1.4.1 /90fc105.crafd/managers/crafd.py—CRAFDPostProcessorManager: the pipeline-core seam, reads the sharedproduction_forecastsshelf and delivers the same wire tocrafd_bucket. Mechanical retarget of the FAO manager; no partner-neutral machinery touched.Second commit (
6d587d1) — what five review passes foundThe package above was correct in substance. What it exposed is that eight guards were scoped to
unfaoby name, so the new partner landed exempt from all of them.The one real defect in the package.
crafd/appwrite_env.pypinned registry v1.3.0 /47172af— an edition at which its own fourAPPWRITE_CRAFD_*coordinates were declared with no values (they were filled in v1.4.0), while the file's own docstring cited the PR that filled them as its justification.unfaowas already at v1.4.1 on this branch. Re-pinned. (The original description here said "same edition asunfao"; that was not true when written.)What was unguarded, and now is not:
contract/could importcrafd— proven, suite stayed greendelivery/ -> contract/had no test at all, while ADR-012 claimed the arrow was "enforced by test"load_dotenv(find_dotenv())inunfao/appwrite_env.pyran on import, suite greenmanagers/directoryCONSUMER_DOCUMENT_NAMEAPIPathManagerliteral and the field it filters on_MACHINERYhad no completeness assertion —contract/enrichment.pywas exempt from both purity guardsThe partner list is now declared once in
tests/conftest.pyand checked against the filesystem, so a third partner cannot be silently exempt.A production-code correction.
_ContractStorePort.upload's comment said the pipeline-core store "only LOGS" metadata failures. It doesn't — it returnsOperationResult(success=False, code="PARTIAL_SUCCESS"). The comment was self-defeating: if the store only logged,successwould beTrueand the guard the comment explains could never fire.Records disposed of in the same change (ADR-014 §5). ADR-012's ontology,
CLONING.md,role_and_seams.md, the CIC,unfao/managers/README.md, the README tree andcontract/__init__.pyall said "the" partner or "the only importer".CLONING.mdadditionally over-claimed þing-02 S24(5) as a flat prohibition — the verdict binds the repos cut from views-faoapi, not an in-repo partner package of the producer — and dropped #146's named trigger. Register: C-77, C-78, C-79 added; C-33, C-40, C-47, C-57 amended; 76/17/59 → 79/19/60.Also retired
test_version_bumped_past_latest_tag: it asserted a version is not tagged, so it went red whenever views-datafactory was in a clean released state. Its stated purpose is covered bytest_land_gaul_commit_is_in_a_release_tag, which passes.Validation
Corrected from the original description: the suite does run, and it is green. The earlier note said it could not be run on this machine (Python 3.13 vs pinned pyarrow 16.1.0). It runs in the
views_pipelineconda env — Python 3.11.14, views-pipeline-core 3.0.0 editable, pyarrow 16.1.0 — where the fivetest_wire_shard/test_wire_sidecarfailures reported earlier do not occur; they were the toolchain mismatch, as suspected.Every new or widened guard is mutation-proven — roughly 25 mutations, each reverted, including the exact defects this PR fixes (the stale v1.3.0 pin; a live
load_dotenvin the crafd manager; all four import spellings at module level and inside function bodies). Sibling checkouts left untouched.Known and registered, not fixed here
model_path.model_name, not fromCONSUMER_DOCUMENT_NAME. Affects both partners and is not introduced here; fixing it changes what lands in the live FAO bucket, which is not a partner-addition PR's business. Noteviews-models/postprocessors/has noun_crafddirectory yet, so CRAF'd's historical name has never been resolved._ContractStorePort.upload's result check has zero tests and fails open.