Skip to content

CABD source-pull: download-cached path (v2 follow-up to #102) #104

@NewGraphEnvironment

Description

@NewGraphEnvironment

Problem

link#102 ships the CABD source-pull as a tunnel-DB join (read from cabd.waterfalls / cabd.dams over the same SSH tunnel that comparison rollups use). That path:

  • requires every link host (M4, M1, cypher, future provincial-split nodes) to have tunnel access to db_newgraph
  • ties refresh cadence to whatever cadence db_newgraph's CABD snapshot is updated on
  • hard-fails when the tunnel is down (no offline option)

bcfp's loader uses a local cabd schema in their fwapg; their refresh cadence is whatever the upstream CABD release publishes.

A complete v2 path is a download-cached source pull: pull CABD's public release artifact (likely a Geopackage or CSV from CABD's release page), cache locally per host, refresh on demand. Decouples link from db_newgraph for the CABD data path.

Why not block #102 on this

#102 needs to land first to surface the bite + verify on the CAMB pre-flight. Tunnel-only is fine for the immediate parity work and the early provincial runs (M4 + M1 + cypher all have tunnel access per rtj#82). v2 download path is a robustness-and-portability follow-up, not a correctness gate.

Scope of this issue

  • Identify CABD's canonical public-release artifact (release page, file format, license terms)
  • Decide the per-host cache location + refresh trigger (manual / on-demand / scheduled?)
  • Implement a lnk_cabd_download(version) or similar entry point that materializes the same cabd.waterfalls / cabd.dams shape locally as the tunnel path produces
  • Wire lnk_falls_load / lnk_dams_load to fall through to the cache when tunnel is unavailable
  • Version-pin the CABD release in bundle config provenance so two hosts running the same bundle get bit-identical CABD data

Out of scope

Reference

  • link#102 — CABD waterfalls v1 (tunnel)
  • link#103 — CABD dams v1 (tunnel)
  • bcfp model/01_access/sql/load_falls.sql, load_dams.sql — same source-pull logic, against bcfp's local cabd schema

Relates to NewGraphEnvironment/sred-2025-2026#24

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions