dbt-costgate v0.9.0

Added

• models[].basis in the JSON report — full_refresh, incremental_form,
  direct, or null when it could not be established. It says which query shape
  was dry-run, and so whether that model's figure is a rebuild or a single run.
  is_incremental cannot answer this: it is true for both. Additive — every
  existing field is unchanged.

• dbt-costgate init — writes a starter .dbt-costgate.yml:

  dbt-costgate init

  Config discovery has always worked, but nothing told you to create the file,
  and the documented example had every setting filled in — copy it and you had a
  configured project rather than a starting point. The written file documents
  every setting with its default, its type and an example value, and leaves all
  of them commented out, so it changes nothing until you uncomment one. It
  refuses to overwrite an existing config, including one under either of the
  other discovered names, and takes --project-dir when your dbt project is not
  the directory you are standing in.

• A published container image, so CI that isn't GitHub Actions can run the
  same check without a Python environment of its own:

  docker pull ghcr.io/drichards124/dbt-costgate:v0.9.0
  docker run --rm -v "$PWD:/workspace" ghcr.io/drichards124/dbt-costgate:v0.9.0 check

  Every release publishes both :vX.Y.Z and :latest. Pin the version tag —
  :latest moves under you, which is the one thing you do not want from the job
  that decides whether a pull request merges. The repository also ships the
  Dockerfile itself, so you can build it and push it to your own registry:

  docker build -t dbt-costgate .
  docker run --rm -v "$PWD:/workspace" dbt-costgate check

  The image runs as a non-root user, mounts your project at /workspace, and
  contains dbt-costgate only — not dbt, so compile in the image you already use
  for that and hand this one the target/. The
  usage guide
  has a GitLab pipeline that authenticates keylessly, with no service-account key
  anywhere. A published image on ghcr.io is wired but not switched on yet;
  build it yourself or push it to your own registry until it lands.

• A pre-commit hook, id dbt-costgate. If your team runs
  pre-commit, adding this repository to your
  .pre-commit-config.yaml catches an expensive change on your own machine
  rather than in review. The config block to copy is in the
  usage guide.

  It runs at the pre-push stage, so install it with the stage named:

  pre-commit install --hook-type pre-push

  A plain pre-commit install will not fire it. Pre-push rather than pre-commit
  because the check needs a compiled target and a BigQuery round-trip per changed
  model — too slow to sit in front of every commit. It needs the same two things
  the CLI does (dbt compile first, BigQuery via ADC), and every check flag is
  available through args:. Requires pre-commit 3.2.0 or newer.

Changed

• The incremental caveat is now one footnote instead of one line per model.
  A change touching five incrementals printed the same sentence five times,
  which pushed the caveats that are about a specific model — a dynamic filter,
  a missing baseline — under a wall of repeats. Reports now tag the rows and
  explain the tag once:

    fct_orders_daily  (full-refresh): 819.20 GiB → 2.91 TiB   +264%
    fct_events_hourly (full-refresh): 40.10 GiB → 44.02 GiB   +10%
  
    ⚠ full-refresh — for the rows tagged above, the figure is the full-refresh
      scan, not an incremental run.
  

  The per-row full-refresh tag is unchanged, so you can still see exactly
  which models it covers, and no other warning is collapsed. If you parse the
  terminal or markdown output, note the string incremental — figure is the full-refresh scan no longer appears in either. The JSON payload is
  unchanged — models[].warnings still carries that warning per model, since
  a machine reader has no repetition problem to solve.

• Priced reports now disclose the free tier they do not deduct, in the footer
  beside the rate they used:

  Priced from the first byte scanned: BigQuery's 1 TiB/month on-demand free
  tier is per billing account, so it is disclosed here and never deducted.
  

  Not new behaviour — costs have always been priced from the first byte — but it
  was stated only in the docs, which is the wrong place for it: someone comparing
  a report against a bill has the report in front of them. The allowance is drawn
  down by every other query the billing account runs that month, which a dry-run
  cannot see, so deducting it would mean guessing. A figure therefore reads high
  by at most one TiB's worth, and a gate that over-reports is safer than one that
  lets a regression through.

  Not configurable, by design: a setting could only mean "assume the tier is
  still unspent", a claim about the whole billing account this tool cannot check.
  The line does not appear when you have set pricing.usd_per_tib: 0.00 — that
  report quotes no money for it to adjust, and the tier is an on-demand allowance
  that does not apply under capacity/Editions pricing at all. Nothing about the
  gate, the breaches or the exit code changes; if you parse reports, note this
  adds a line to the terminal footer and a <br/> segment to the markdown one.
  The JSON payload is unchanged.

  The docs previously called the free tier "not modeled by default", which
  implied a setting that has never existed. That wording is gone.

• The docs now say plainly that dbt-costgate prices compute, not storage.
  BigQuery meters the two separately, and a dry-run reports the bytes a query
  would scan — a compute figure that carries no storage information. Nothing
  about what the tool measures has changed; it is now stated as the scope it has
  always been, under
  what it will not do,
  with the case worth knowing called out: a view becoming a table, or an
  incremental becoming a full table, moves real money on a meter nothing here
  watches.

Fixed

• A baseline with no compiled SQL is no longer reported as a basis mismatch.
  When the baseline manifest had no compiled code for a model, it was still
  assigned a query shape — and if your branch happened to compile to the other
  one, the report said:

  ⚠ mixed basis — baseline is full_refresh, current is incremental_form;
    recompile the baseline the same way
  

  naming a shape for SQL that was never compiled, directly beside the warning
  saying the baseline had no compiled SQL at all. Recompiling could not fix it,
  because the mismatch was not real.

  Such a model is still reported and still not gated — with no baseline
  bytes, the whole current scan reads as an increase, and a threshold firing on
  that is firing on a missing measurement rather than a regression. That
  outcome is now decided where the missing baseline is detected. Previously it
  fell out of the bogus mismatch, which meant it only applied when your branch
  compiled to the other shape: the same missing baseline gated or did not gate
  depending on something unrelated to it. If you have such a model, expect the
  spurious mixed basis line to disappear and the gating to stay off in cases
  where it was previously inconsistent.

  Unestimated rows also no longer carry a full-refresh / incremental tag,
  and models[].basis is null for them in JSON — there is no figure for a
  basis to describe.

• An incremental model compiled against its existing table is no longer
  labelled full-refresh. An incremental has two prices — the cost to rebuild
  the table, and the cost of one run against the table as it already stands —
  and which one a dry-run measures is decided by how the model was compiled.
  Every incremental row was tagged full-refresh and told "figure is the
  full-refresh scan" regardless, so a figure that was one incremental run read
  as rebuild cost. On a large fact table those differ by orders of magnitude,
  and the mislabelled one reads low.

  This was not a rare case: a prod-run manifest captures incrementals in their
  incremental form, which the usage guide already documents.

  Rows are now labelled from the basis actually measured — full-refresh or the
  new incremental tag — with a footnote under the table for each one present:

    fct_orders_daily  (incremental): 92.16 GiB → 112.64 GiB   +22%   USD +0.13/run
  
    ⚠ incremental — for the rows tagged above, the figure is one run against the
      table as already built, so it does not gate rebuild cost.
  

  If you have a max_usd_total / max_tib_total ceiling on an incremental
  believing it capped rebuild cost, check the tag: on an incremental row it
  does not, and never did — the label was what said otherwise. The same applies
  to run_frequency, which should count rebuilds for a full-refresh row and
  runs for an incremental one.

  Two things change in output. The incremental tag is new, so anything matching
  on the literal full-refresh will no longer see these rows. And in JSON,
  models[].warnings carries a different sentence for an incremental-form model.
  models[].is_incremental is unchanged — it is true for both shapes, which
  is why it could never have answered this.