Assert full 259,200-cell grid in read_gaul_cells — the 50k floor is fooled by a ~200k partial harvest and writes a silently incomplete 'global' bundle

[Polichinel]

Background

The datafactory path in scripts/build_entity_metadata.py reads a GAUL harvest, crosswalks it to VIEWS country_id, and writes the committed priogrid.parquet bundle. Its only completeness guard is a post-crosswalk row floor in main():

# scripts/build_entity_metadata.py:351
if len(priogrid) < 50_000:
    raise ValueError(
        f"Crosswalk produced only {len(priogrid):,} cells — a global "
        "GAUL harvest yields ~66k. ..."
    )

read_gaul_cells (scripts/build_entity_metadata.py:149-157) is the input seam that loads the harvest. Its docstring claims a contract it never enforces: "... Full 259,200-cell grid; ocean/unassigned cells carry a null/empty code." It builds the frame by aligning iso3_code.parquet and gaul0_name.parquet on gid and returns whatever rows exist — no assertion on row count.

Measured numbers. On the real global harvest the crosswalk match fraction is 25.5% (66,205 matched / 259,200 cells). The floor of 50,000 therefore breaks even at ≈195,760 input cells (50,000 / 0.2554). A truncated/partial harvest of ~200,000 cells yields ≈51,084 matched cells — it clears the 50k floor and writes a bundle that looks global but is missing tens of thousands of land cells. Those missing cells then NaN-label at render time: exactly the pre-C-22 map-emptying failure class that ADR-018 / register C-22 exist to prevent.

No test asserts the input row count (tests/test_metadata_assets.py covers reduction, crosswalk buckets, and duplicate-gid rejection, but nothing pins len(gaul_cells)).

Why it matters

• Nygard — fail fast at the cause. The 50k floor is a downstream symptom check on the crosswalk output, distant from and lossy about the actual fault (a short/corrupt harvest input). Because the fault passes through a 4:1 attenuating crosswalk before it's checked, the guard's effective sensitivity is ~1 cell in 4 — it cannot distinguish "global harvest" from "77% of a harvest." The check belongs where the invariant is first knowable: the input.
• ADR-008 — fail loud is this repo's law. A dev-time bundle regeneration that silently emits a partial "global" artifact is precisely the outcome ADR-008 forbids. Silent-partial is worse than crash-loud here because the artifact is committed and shipped to the offline render path.
• Feathers — seams and characterization. read_gaul_cells is the input seam. It currently documents an invariant (259,200 rows) that no characterization test protects, so any future harvest-format drift or partial write degrades coverage invisibly. Pin the seam with an assertion and a test.

Proposed fix

1. Add an input assertion in read_gaul_cells (scripts/build_entity_metadata.py:149-157), immediately after assembling out:

   if len(out) != 259_200:
       raise ValueError(
           f"GAUL harvest has {len(out):,} cells, expected the full "
           f"259,200-cell PRIO-GRID (360 rows x 720 cols). "
           f"Partial/corrupt harvest or wrong --gaul-dir: {gaul_dir}. "
           f"Refusing to build an incomplete 'global' bundle."
       )

   This makes the documented contract executable and catches the fault at the cause, before the crosswalk attenuates it.

2. 50k output floor: demote from primary guard to a cheap secondary sanity check on the crosswalk (it still catches an all-null / wrong-column iso3 join that the input-count assert would miss). Keep it, but its comment should note it is defence-in-depth, not the completeness guarantee.

Acceptance criteria

• read_gaul_cells raises ValueError with a clear, actionable message when the assembled frame is not exactly 259,200 rows.
• The happy path (exactly 259,200 rows) returns unchanged output — no behavior change for a valid global harvest.
• The < 50_000 output floor remains (or is explicitly retired) with a comment stating its role as secondary/defence-in-depth rather than the completeness guard.
• A test exists that fails on a 200,000-row harvest (the interval that currently passes the 50k floor).

Test plan

File: tests/test_metadata_assets.py (module loaded via the existing _load_script() helper; read_gaul_cells takes a dir, so use tmp_path + pd.DataFrame.to_parquet to stage iso3_code.parquet and gaul0_name.parquet).

• red_team — partial harvest rejected (the gap being closed): stage a harvest with ~200,000 gids (well above the 50k break-even but below 259,200); assert read_gaul_cells raises ValueError matching 259,200. This is the case that currently slips through undetected.
• red_team — short harvest rejected: a handful of rows; assert the same raise (guards the message and the boundary).
• green_team — full grid accepted: stage exactly 259,200 gids; assert len(read_gaul_cells(...)) == 259_200 and columns are {gid, iso3, gaul0_name} — characterizes the seam so future format drift trips a test.

---

Origin: epic #230, story #231; surfaced by the PR #238 max-effort review.

[Polichinel]

Fixed in PR #238 (commits 4e365a5 + a9055cd), merged to development at ac3d332.