From 9971661b36b854fbd26ba161000629c92c06d9e3 Mon Sep 17 00:00:00 2001 From: almac2022 Date: Fri, 1 May 2026 15:26:23 -0700 Subject: [PATCH] Archive PWF for link#88 (subsurfaceflow fold) Deferred archive of the issue#88 PWF (link 0.20.0). The active wire-up PWF for issue#96 / fresh#158 lives in planning/active/ via the v0.22.0 release. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../README.md | 5 ++ .../findings.md | 75 +++++++++++++++++++ .../progress.md | 17 +++++ .../task_plan.md | 37 +++++++++ 4 files changed, 134 insertions(+) create mode 100644 planning/archive/2026-05-issue-88-subsurfaceflow-fold/README.md create mode 100644 planning/archive/2026-05-issue-88-subsurfaceflow-fold/findings.md create mode 100644 planning/archive/2026-05-issue-88-subsurfaceflow-fold/progress.md create mode 100644 planning/archive/2026-05-issue-88-subsurfaceflow-fold/task_plan.md diff --git a/planning/archive/2026-05-issue-88-subsurfaceflow-fold/README.md b/planning/archive/2026-05-issue-88-subsurfaceflow-fold/README.md new file mode 100644 index 0000000..8c0f8ab --- /dev/null +++ b/planning/archive/2026-05-issue-88-subsurfaceflow-fold/README.md @@ -0,0 +1,5 @@ +## Outcome + +Folded `prep_subsurfaceflow` into `prep_natural` so subsurfaceflow positions enter the per-species observation/habitat lift via `.natural_barriers` (gated on `cfg$pipeline$break_order`). Restores bcfp parity for blkey 356286055 BT rearing on HARR (0 → 6.509 km). HARR full-WSG diffs collapsed: rearing_stream −10.4% → −4.19%, rearing −1.84%, spawning −1.6%. 15-WSG `tar_make` 33/33 across both bundles, 53m. Reproducibility verified — second `tar_make` byte-identical (`link_value` digest `5a641892b82604259b0ba168ea093661`, 0/1057 rows differ). Default-bundle bit-identical vs pre-fix (0/581 rows changed). HORS −7.68% BT residual confirmed as a separate mechanism (parent-stream-order/child-order rearing bypass — fresh#158 / link#96). + +Closed by: PR #89 (merged into link v0.20.0 release commit `d8566d2`). diff --git a/planning/archive/2026-05-issue-88-subsurfaceflow-fold/findings.md b/planning/archive/2026-05-issue-88-subsurfaceflow-fold/findings.md new file mode 100644 index 0000000..2be1894 --- /dev/null +++ b/planning/archive/2026-05-issue-88-subsurfaceflow-fold/findings.md @@ -0,0 +1,75 @@ +# Findings — link#88 + +## Diagnosis (2026-04-30) + +### Single-stream trace: HARR blkey 356286055 + +- link `fresh.streams_habitat`: 21 segments, all `rearing=FALSE` for BT, all `accessible=FALSE` for BT +- bcfp `streams_access`: `access_bt = 1` on every segment, `barriers_bt_dnstr = {}` (zero natural BT barriers) +- bcfp `barriers_anthropogenic_dnstr`: 2 entries (DAM at 356282804 DRM 739, ROAD/DEMOGRAPHIC at 356282804 DRM 658) — both flagged in `barriers_remediations` (REMEDIATED) +- Two **subsurfaceflow** points downstream on 356282804 (DRMs 265, 279) in `barriers_subsurfaceflow` +- 55 anadromous obs (CH/CM/CO) upstream of (356282804, 265) — clears bcfp's threshold (1 for BT, 5 for anadromous) +- bcfp lifts both subsurfaceflow points → `barriers_bt` and `barriers_ch_cm_co_pk_sk` empty in this drainage → BT/CH/CO credit upstream + +### Why link doesn't lift + +`lnk_pipeline_prepare()` build order (current): + +1. `prep_load_aux` → falls, definite, control, habitat +2. `prep_gradient` → gradient_barriers_raw (pruned, ltree-enriched) +3. **`prep_natural` → `.natural_barriers` = gradient + falls** ← subsurfaceflow NOT here +4. `prep_overrides` → calls `lnk_barrier_overrides(barriers = natural_barriers)` → per-species skip list +5. **`prep_subsurfaceflow`** (opt-in) → `.barriers_subsurfaceflow` ← runs AFTER overrides + +`lnk_pipeline_classify_build_breaks()` then UNIONs `barriers_subsurfaceflow` directly into `fresh.streams_breaks` with label `blocked`. Since the override skip list never saw it, `frs_habitat_classify(barrier_overrides = ...)` cannot lift it. All species get blocked at the subsurfaceflow position. + +### bcfishpass natural barrier construction (per-species) + +`model/01_access/sql/model_access_bt.sql` — `barriers_bt`: + +- gradient_25 + gradient_30 + falls + **subsurfaceflow** + user_definite +- LIFT: any obs upstream (BT/CH/CM/CO/PK/SK/ST), or any habitat upstream +- user_definite always retained + +`model/01_access/sql/model_access_ch_cm_co_pk_sk.sql` — `barriers_ch_cm_co_pk_sk`: + +- gradient_15/20/25/30 + falls + **subsurfaceflow** + user_definite +- LIFT: ≥5 anadromous obs upstream (post-1990), or any habitat upstream +- `user_barriers_definite_control.barrier_ind = TRUE` blocks the obs lift; habitat lift unaffected +- user_definite always retained + +bcfp's `barriers_anthropogenic` (PSCIS, dams, road crossings) is **NOT** in the per-species barrier set. It's tracked separately for downstream-of-crossing accountability. Anthropogenic barriers don't gate species access in bcfp's habitat sums. + +### Design diagnosis + +`.lnk_pipeline_prep_subsurfaceflow` was added in PR #82 as its own helper. Treated subsurfaceflow as a parallel concept needing a parallel pipeline phase. But subsurfaceflow is just **a third row in the same union** that `prep_natural` already builds for gradient + falls. The wiring miss: subsurfaceflow's positions never reached `natural_barriers`, so the per-species lift skipped it entirely. + +`prep_natural` is the right home — bcfp's source of truth confirms gradient + falls + subsurfaceflow is *the* natural-barrier union per species. + +### Where it surfaces (15-WSG rollup, parity) + +| WSG | sp | metric | link | bcfp | diff_pct | +|------|----|-----------------|------|------|---------:| +| HARR | CH | rearing_stream | 118 | 139 | -14.8 | +| HARR | CO | rearing_stream | 134 | 155 | -13.3 | +| HARR | ST | rearing_stream | 157 | 177 | -11.6 | +| HARR | BT | rearing_stream | 292 | 326 | -10.4 | +| HORS | BT | rearing_stream | 366 | 396 | -7.7 | +| LFRA | BT | rearing_stream | 1020 | 1103 | -7.5 | +| LFRA | BT | rearing | 1670 | 1800 | -7.2 | +| HORS | CH | rearing_stream | 167 | 179 | -6.8 | + +LILL/VICT unaffected — sparse subsurfaceflow + sparse fish observations above what does exist. + +## What is NOT involved + +- #83 (anthropogenic dam design): the dam at DRM 739 and road at DRM 658 carry `barrier`/`potential` labels in `fresh.streams_breaks`. Default `label_block = "blocked"` means they don't gate. Not the cause. +- `barriers_definite`: intentionally separate per bcfp — never lifted via obs/hab. Link mirrors that. +- `barriers_remediations`: bcfp tracks for downstream reporting only; doesn't gate access. + +## Versions at diagnosis + +- link 0.19.0 (commit e4e7a6e) +- fresh 0.25.0 +- bcfishpass 440bc1e (2026-04-28) +- fwapg local Docker (port 5432) diff --git a/planning/archive/2026-05-issue-88-subsurfaceflow-fold/progress.md b/planning/archive/2026-05-issue-88-subsurfaceflow-fold/progress.md new file mode 100644 index 0000000..4413dd3 --- /dev/null +++ b/planning/archive/2026-05-issue-88-subsurfaceflow-fold/progress.md @@ -0,0 +1,17 @@ +# Progress — link#88 + +## Session 2026-04-30 + +- Diagnosis: traced HARR blkey 356286055 BT under-credit to subsurfaceflow positions on downstream tributary 356282804 not reaching `natural_barriers` for the per-species observation/habitat lift. +- Read bcfp SQL (`model_access_bt.sql`, `model_access_ch_cm_co_pk_sk.sql`) — confirmed bcfp's natural-barrier union includes subsurfaceflow with same lift rules. +- Confirmed default-bundle off-switch is preserved verbatim (omit `subsurfaceflow` from `cfg$pipeline$break_order`). +- Filed link#88 with diagnosis + proposed fix. +- Branch `88-fold-subsurfaceflow-natural` from main. +- PWF baseline (commit 4bd9ca0). +- Code change: extended `.lnk_pipeline_prep_natural` signature `(conn, aoi, cfg, loaded, schema)`; absorbed subsurfaceflow body, gated on `cfg$pipeline$break_order`; deleted standalone `.lnk_pipeline_prep_subsurfaceflow`; pruned conditional call from `lnk_pipeline_prepare()`. `devtools::document()` clean. +- Tests: 3 new test cases in `tests/testthat/test-lnk_pipeline_prepare.R` — opted-out, opted-in (per-statement assertion that link#88 fix INSERT fires), control-table honoured. 44/44 pass. +- Code-check: 3 rounds. Rounds 1–2 clean. Round 3 caught a fragile cross-statement regex in the test; replaced with per-statement `any(grepl & grepl)`. Sanity-verified the assertion catches the regression. +- Pre-flight: HARR single-WSG `compare_bcfishpass_wsg(wsg = "HARR", config = lnk_config("bcfishpass"))` 89.5 s. blkey 356286055 BT credits 6.509 km (was 0). HARR BT diffs collapsed: rearing_stream -10.4% → -4.19%, rearing -1.84%, spawning -1.6%. +- 15-WSG `tar_make` (53m 2s, 33/33). Parity dramatic on HARR (CH/CO/ST <0.32%; BT residual -4.19%) and LFRA (CH/CO/ST <0.6%; BT residual -3.75%). HORS unchanged (-7.68% rearing_stream BT) — different mechanism, follow-up needed. Default-bundle bit-identical (0 of 581 rows changed). +- Reproducibility re-run (52m 55s, 33/33). 0 of 1057 link_value rows differ; digest match (`5a641892b82604259b0ba168ea093661`). ✓ +- Code commit `a21a8f8`. PWF + verification logs commit. Ready for PR. diff --git a/planning/archive/2026-05-issue-88-subsurfaceflow-fold/task_plan.md b/planning/archive/2026-05-issue-88-subsurfaceflow-fold/task_plan.md new file mode 100644 index 0000000..1794441 --- /dev/null +++ b/planning/archive/2026-05-issue-88-subsurfaceflow-fold/task_plan.md @@ -0,0 +1,37 @@ +# Task Plan — link#88: Fold subsurfaceflow into natural barriers + +## Phase 1: Setup +- [x] File link#88 with diagnosis + proposed fix +- [x] Branch `88-fold-subsurfaceflow-natural` from main +- [x] PWF baseline (task_plan, findings, progress) + +## Phase 2: Code change +- [x] Extend `.lnk_pipeline_prep_natural(conn, aoi, cfg, loaded, schema)` to absorb subsurfaceflow body, gated on `"subsurfaceflow" %in% cfg$pipeline$break_order` +- [x] Append subsurfaceflow rows to `.natural_barriers` (label `blocked`) +- [x] Delete `.lnk_pipeline_prep_subsurfaceflow` helper +- [x] Remove conditional call from `lnk_pipeline_prepare()` +- [x] `devtools::document()` — refresh roxygen for prep_natural + +## Phase 3: Tests +- [x] `tests/testthat/test-lnk_pipeline_prepare.R`: subsurfaceflow opted in → INSERT into natural_barriers fires (per-statement assertion) +- [x] Same file: subsurfaceflow not opted in → no subsurfaceflow code path runs +- [x] Same file: subsurfaceflow honours barriers_definite_control +- [x] `devtools::test(filter = "lnk_pipeline_prepare")` clean (44/44 PASS) + +## Phase 4: Code-check +- [x] `/code-check` on staged diff — 3 rounds. Round 3 caught fragile cross-statement regex; tightened to per-statement `any(grepl & grepl)`. Final clean. + +## Phase 5: Verification +- [x] HARR single-WSG pre-flight `tar_make` — `data-raw/logs/20260430_11_preflight_harr_link88.txt` +- [x] blkey 356286055 BT rearing credits **6.509 km** (was 0) +- [x] Full 15-WSG `tar_make` — `data-raw/logs/20260430_12_tar_make_15wsg_link88.txt` (53m 2.2s, 33/33 targets) +- [x] HARR CH/CO/ST rearing_stream closed to ±0.32% (BT residual -4.2%, separate mechanism noted) +- [x] LFRA CH/CO/ST closed to ±0.6% (BT residual -3.75%) +- [x] HORS unchanged (-7.68% BT) — different mechanism, follow-up issue +- [x] Default-bundle rollup bit-identical (0 of 581 link_value rows changed) +- [x] Reproducibility: second `tar_make` byte-identical (`link_value` digest `5a641892b82604259b0ba168ea093661` matches across runs; 0 of 1057 rows differ) + +## Phase 6: Ship +- [x] Atomic commits with PWF checkbox flips +- [x] PR with `Fixes #88` and `Relates to NewGraphEnvironment/sred-2025-2026#24` — PR #89 merged 2026-05-01 as part of link 0.20.0 +- [x] Archive PWF after merge