Annual IO LULC series (2017-2023) for bulk, necr, lnth, kotl: rerun step 3 with every year

[NewGraphEnvironment]

Context

drift v0.14.0 added dft_rast_break_class() (drift#9): a per-pixel scan of the annual class series that separates a sustained switch from flicker and dates the switch. Run on the whole BULK floodplain with all seven IO LULC years, it found that of the 4,620 ha the 2017 -> 2023 comparison reports as change, 19.7% is a switch sustained two years each side, 36.4% is a clean switch where 2017 or 2023 is itself the odd year out (922 ha "broke in 2018", i.e. 2017 alone differs), and 44.0% flickers; a further 3,187 ha flickers while reading stable on the endpoints. One watershed group is a suspicion. Five is a finding — and a decision about what the published transition layers mean.

Problem

scripts/floodplain_lcc/03_lulc_classify.R fetches three snapshots by construction: years <- sort(unique(c(yrs[1], round(mean(yrs)), yrs[2]))) from cfg$change_interval (endpoints + midpoint, #19). So every area's rasters/<scenario>/classified_<year>.tif, the classified_<sp>_<scen>_<year> gpkg layers and the provenance years field carry 2017 / 2020 / 2023 only, and the catalogue (stac_floodplains_bc) publishes the same three. The temporal leg needs every year.

Proposal

• A per-area config switch in area.yml, default off so no other area changes: lulc_annual: true makes years <- seq(yrs[1], yrs[2]) (2017:2023 for the default interval). Everything downstream already iterates years — the classified_hashes loop, the gpkg layers, the raster writes, provenance years — so the pipeline change is the one line plus the config key. (As shipped it is larger, and deliberately: review of the branch found that an empty lulc_annual silently ran three years, that the drift floor asserted >= 0.6.0 while this work established >= 0.10.0 as a correctness floor, and that the one-way door below had no detector. All three are in the PR.) The transition stays endpoint-to-endpoint (yrs[1], yrs[2]), unchanged.
• The available_years guard the original proposal asked for already exists at 03_lulc_classify.R:90-95 as setdiff(years, lc_available), which covers a seven-element set with no edit.
• Rerun step 3 only for bulk, necr, lnth, kotl: Rscript scripts/run_area.R <area> 3. Steps 1-2 are untouched, so the floodplain geometry and sub-basins are byte-stable and the provenance shows a regeneration, not a comparison that cannot tell "already current" from "never re-run" (Roll the #65 provenance fields out to the remaining 18 areas #73).

PINE is excluded — four areas, not five

data/pine/ has no provenance.json and its rasters/bt_ff04/*.tif are dated 2026-07-12, which predates flooded 0.5.0. That is the bankfull-units vintage CLAUDE.md calls dead, not merely superseded — the regression was fed hectares where Hall et al. specify km². Running step 3 there would classify land cover over a floodplain the repo has already declared wrong, and stamp fresh landcover provenance beside two absent sections: provenance-check.R §7b would then report network[bt3] and floodplain[bt_ff04] MISSING, while run_region.R would begin treating pine as cached-complete because lulc_summary.rds and provenance.json both exist.

PINE and MCGR are the only 2 of 23 area dirs in that state, and #76 tracks both. MCGR is absent from this issue's list for the same reason, so the asymmetry is deliberate rather than an oversight. PINE's annual run belongs in #76, after the fresh vs fresh.streams_vw_bcfp staleness question is settled — it needs steps 1-2-3 and a fresh baseline, not this issue's step-3 rerun with an "identical to the previous run" assertion.

Cost — the previous figure described a path these runs do not take

The "23.6 min for BULK at tile_size = 20000" figure was measured with FP_TILE_SIZE set. No area sets tile_size in a committed config — it appears in none of them — so Rscript scripts/run_area.R <area> 3 takes the untiled bbox path, and CLAUDE.md records tiling as benchmarked and rejected under #8 (FRAN at 20 km is 0.79x, i.e. untiled is faster). Re-measured untiled (2026-09-05): necr 9.2 min, lnth 14.4, kotl 32.4, bulk 34.9 — 49.3 min wall across two machines against 91.9 sequential. Per-area timings and peak RSS are in the PR body and the committed evidence log.

Two further corrections to the cost model:

• The fetch is not the expensive part. terra::as.polygons() runs once per year, so 7x instead of 3x, and Pass 2 builds a cropped/masked grid per year — which for a whole-WSG area with a single sub-basin is the entire grid again. That is the new load. KOTL was predicted the OOM candidate and was not — it peaked highest at 54.3 GB, but BULK at 168 Mcells peaked at only 20.6 GB and NECR at 55 Mcells peaked above LNTH at 62. Peak RSS does not track grid size; the two large runs were on different hosts, so the plausible cause is terra sizing its working set against available RAM. It was not isolated and must not be quoted as a per-area requirement. 64 GB sufficed for the largest area run on it.
• The three shared years are a genuine re-derivation, not a cache hit. drift's stac_cache_key() excludes years, so a warm cache would serve 2017/2020/2023 unchanged. All four baselines were produced under drift 0.8.0, whose untiled cache keys predate the 0.10.0 change, so nothing hits and every year is re-fetched.

Acceptance

Machine-checked per area with the repo's own tool, and grepped for rather than read from an exit code — provenance_ab-compare.R reports inputs_hash differing as a failure, and under this change it must differ:

Rscript scripts/floodplain_lcc/provenance_ab-compare.R <baseline>.json data/<area>/provenance.json <area> before after

entry	inputs	outputs	datetime
landcover[<scen>]	DIFFER (7 years, 7 digests, new item_hash, drift 0.8.0 -> 0.13.0)	same	moved
network[*], floodplain[*]	same	same	SAME (steps 1-2 do not re-run)

Any other line is a defect. Plus, per area:

• Seven classified_<year>.tif in rasters/<scenario>/, seven classified_<scen>_<year> gpkg layers and no eighth, provenance years = 2017..2023
• The 2017 / 2020 / 2023 classified_content_sha256 values unchanged element-wise against the baseline
• outputs.transition_content_sha256 and outputs.transition_patches unchanged
• provenance-check.R <area> and bridge-check.R <area> green
• Timings and peak RSS per area recorded in the PR body
• The four areas loaded to the catalogue (stac#59) and analysed in drift (drift#62)

Two consequences downstream, neither a defect, both flagged on stac#59: floodplain_landcover.gpkg bytes move for every area even where content does not (rewriting one layer into an existing gpkg is not byte-stable, #45), and nge:landcover_key moves because it publishes item_hash, which is built from the requested years.

Related

• Load to the catalogue: Publish all seven classified years for bulk, necr, lnth, kotl stac_floodplains_bc#59
• Analysis once published: Temporal QA across five watershed groups from the published annual series: does the BULK split generalise? drift#62
• PINE (and MCGR): MCGR and PINE blocked from the #73 provenance rollout — upstream fresh gaps #76