Problem
user_barriers_definite.csv entries (227 reviewer-identified positions in the bcfishpass config bundle) can currently be re-opened by upstream observations in link's pipeline. bcfishpass does not allow this.
Current behaviour in link
.lnk_pipeline_prep_natural() builds <schema>.natural_barriers as the UNION of three sources:
gradient_barriers_raw + falls + barriers_definite
natural_barriers is then passed to lnk_barrier_overrides() as the barriers argument. The override function iterates over every row and, for each species, emits an override row when count(observations upstream) >= threshold (or habitat confirmation upstream). The override is later consumed by fresh as a skip list — any barrier listed is ignored during access gating.
So a user_barriers_definite row at (blue_line_key, downstream_route_measure) with, say, 5+ historical observations upstream will have fresh skip it during gating — habitat upstream becomes accessible. That's the same override mechanism that #44 just locked down for barriers_definite_control TRUE rows, but via a different mechanism entirely.
bcfishpass's treatment
model_access_*.sql (per-model access SQL) build the barrier CTE from gradient + falls + subsurfaceflow only — barriers_user_definite is NOT joined during the observation- or habitat-filter step. Instead it's appended post-filter to the final per-model barrier table:
-- from model_access_ch_cm_co_pk_sk.sql (tail)
select * from barriers_filtered -- gradient/falls/subsurface after filters
union all
select ... from bcfishpass.barriers_user_definite
where watershed_group_code = :'wsg'
on conflict do nothing;
Same pattern in model_access_bt.sql, model_access_st.sql, etc. User-definite barriers are absolute in bcfishpass — they always stay in the per-model barrier table, regardless of observations or habitat upstream.
Scale
227 rows province-wide, distribution by barrier_type:
| Type |
Count |
| MISC |
223 |
| EXCLUSION |
2 |
| GRADIENT_25 |
1 |
| SUBSURFACEFLOW |
1 |
MISC ("reviewer-identified, doesn't fit model categories") dominates. EXCLUSION entries (Stump Lake Creek Dam, Erickson Creek) are the most obviously wrong-to-override case.
Whether any current link WSG actually trips this in practice is an empirical question — same investigation pattern used in #44 on ADMS/BULK/BABL/ELKR. For #44 the TRUE control rows were all rescued by threshold or habitat, so the filter defect was latent on our validation WSGs. It's possible (likely, given the volume) that this defect is active on at least one validated WSG and was offset by some other pre-#44 defect. Worth checking.
Proposed fix
Match bcfishpass's architecture: exclude barriers_definite from the set passed to lnk_barrier_overrides(), and append it to each per-model barrier table (barriers_bt, barriers_ch_cm_co_pk_sk, barriers_st, barriers_wct) post-filter.
Two reasonable shapes:
Shape A — drop from natural_barriers, append in .lnk_pipeline_prep_minimal
.lnk_pipeline_prep_natural(): build natural_barriers from gradient + falls only. Remove the barriers_definite UNION.
.lnk_pipeline_prep_minimal(): after frs_barriers_minimal() produces each per-model table, append barriers_definite rows (WSG-filtered) via INSERT ... SELECT ... FROM barriers_definite.
Clean but touches two helpers.
Shape B — keep natural_barriers shape, exclude from override input
natural_barriers stays as-is (used for other break-sourcing purposes — verify).
.lnk_pipeline_prep_overrides() passes barriers = "<schema>.natural_barriers_no_def" (new intermediate) to lnk_barrier_overrides().
- Append
barriers_definite to per-model tables in .lnk_pipeline_prep_minimal() same as Shape A.
Conservative — no change to natural_barriers callers — but adds an intermediate table.
Shape A is cleaner; preferable unless a natural_barriers caller depends on the definite union.
Verification
- Pre-fix: query
working_<wsg>.barrier_overrides for any row whose (blue_line_key, downstream_route_measure) also appears in working_<wsg>.barriers_definite for any of the 5 WSGs in the targets pipeline. If non-empty → active defect on current validated WSGs → post-fix rollup will shift.
- Post-fix:
tar_make() reproducibility bit-identical across two runs. Rollup shifts only if the pre-fix query found matches; direction must be toward bcfishpass (link_km DECREASES on affected rows because those user-definite barriers now correctly block upstream habitat).
If the pre-fix query is empty across all 5 WSGs, add a sixth WSG that actively exercises this defect (same approach as DEAD for #44).
Vignette correction (rides on this PR)
vignettes/reproducing-bcfishpass.Rmd currently says user-definite barriers are "eligible for per-species override via lnk_barrier_overrides when enough upstream observations clear the threshold" — accurate to today's link but wrong vs bcfishpass and wrong biologically. Correct to: "always-blocking, always a break position, never eligible for observation-based override — matches bcfishpass's post-filter append." Do the edit in the same PR that fixes the code.
Cross-references
Versions at filing
Problem
user_barriers_definite.csventries (227 reviewer-identified positions in the bcfishpass config bundle) can currently be re-opened by upstream observations in link's pipeline. bcfishpass does not allow this.Current behaviour in link
.lnk_pipeline_prep_natural()builds<schema>.natural_barriersas the UNION of three sources:natural_barriersis then passed tolnk_barrier_overrides()as thebarriersargument. The override function iterates over every row and, for each species, emits an override row whencount(observations upstream) >= threshold(or habitat confirmation upstream). The override is later consumed by fresh as a skip list — any barrier listed is ignored during access gating.So a
user_barriers_definiterow at(blue_line_key, downstream_route_measure)with, say, 5+ historical observations upstream will havefreshskip it during gating — habitat upstream becomes accessible. That's the same override mechanism that #44 just locked down forbarriers_definite_controlTRUE rows, but via a different mechanism entirely.bcfishpass's treatment
model_access_*.sql(per-model access SQL) build the barrier CTE from gradient + falls + subsurfaceflow only —barriers_user_definiteis NOT joined during the observation- or habitat-filter step. Instead it's appended post-filter to the final per-model barrier table:Same pattern in
model_access_bt.sql,model_access_st.sql, etc. User-definite barriers are absolute in bcfishpass — they always stay in the per-model barrier table, regardless of observations or habitat upstream.Scale
227 rows province-wide, distribution by
barrier_type:MISC ("reviewer-identified, doesn't fit model categories") dominates. EXCLUSION entries (Stump Lake Creek Dam, Erickson Creek) are the most obviously wrong-to-override case.
Whether any current link WSG actually trips this in practice is an empirical question — same investigation pattern used in #44 on ADMS/BULK/BABL/ELKR. For #44 the TRUE control rows were all rescued by threshold or habitat, so the filter defect was latent on our validation WSGs. It's possible (likely, given the volume) that this defect is active on at least one validated WSG and was offset by some other pre-#44 defect. Worth checking.
Proposed fix
Match bcfishpass's architecture: exclude
barriers_definitefrom the set passed tolnk_barrier_overrides(), and append it to each per-model barrier table (barriers_bt,barriers_ch_cm_co_pk_sk,barriers_st,barriers_wct) post-filter.Two reasonable shapes:
Shape A — drop from
natural_barriers, append in.lnk_pipeline_prep_minimal.lnk_pipeline_prep_natural(): buildnatural_barriersfrom gradient + falls only. Remove thebarriers_definiteUNION..lnk_pipeline_prep_minimal(): afterfrs_barriers_minimal()produces each per-model table, appendbarriers_definiterows (WSG-filtered) viaINSERT ... SELECT ... FROM barriers_definite.Clean but touches two helpers.
Shape B — keep
natural_barriersshape, exclude from override inputnatural_barriersstays as-is (used for other break-sourcing purposes — verify)..lnk_pipeline_prep_overrides()passesbarriers = "<schema>.natural_barriers_no_def"(new intermediate) tolnk_barrier_overrides().barriers_definiteto per-model tables in.lnk_pipeline_prep_minimal()same as Shape A.Conservative — no change to
natural_barrierscallers — but adds an intermediate table.Shape A is cleaner; preferable unless a
natural_barrierscaller depends on the definite union.Verification
working_<wsg>.barrier_overridesfor any row whose(blue_line_key, downstream_route_measure)also appears inworking_<wsg>.barriers_definitefor any of the 5 WSGs in the targets pipeline. If non-empty → active defect on current validated WSGs → post-fix rollup will shift.tar_make()reproducibility bit-identical across two runs. Rollup shifts only if the pre-fix query found matches; direction must be toward bcfishpass (link_km DECREASES on affected rows because those user-definite barriers now correctly block upstream habitat).If the pre-fix query is empty across all 5 WSGs, add a sixth WSG that actively exercises this defect (same approach as DEAD for #44).
Vignette correction (rides on this PR)
vignettes/reproducing-bcfishpass.Rmdcurrently says user-definite barriers are "eligible for per-species override via lnk_barrier_overrides when enough upstream observations clear the threshold" — accurate to today's link but wrong vs bcfishpass and wrong biologically. Correct to: "always-blocking, always a break position, never eligible for observation-based override — matches bcfishpass's post-filter append." Do the edit in the same PR that fixes the code.Cross-references
barriers_definite_controlflavour. This fixes theuser_barriers_definiteflavour.research/bcfishpass_comparison.mdunder "Remaining gaps" — if this defect is active on BULK/BABL/ELKR, resolving it may shift some of the +1-4% spawning excess attributed there to segmentation boundaries.Versions at filing