v0.2.6

v0.2.6

Jobs can run dbt. A gold build that had to be driven from an operator's shell
is now a task the scheduler owns, which is the difference between a pipeline
demonstrated and a pipeline run.

docker pull ghcr.io/calvinchengx/databricks-emulator:0.2.6

dbt_task runs against the warehouse it names (#51, #52, #53)

The project travels from the workspace store to the statement agent inline and
dbt is invoked through dbtRunner. dbt is a warehouse client, so running it
as a job changes who invokes it, not what it connects to — the same models,
the same endpoint, a different caller.

This was the single gap holding a consumer's gold step outside its own
orchestrator: the models built, but from a host script standing in for the
scheduler, so the cell could not claim to be green through Jobs.

Witnessed end to end in #53, which found and fixed two bugs on the way — the
reason to witness rather than to unit-test. The agent image is pinned by digest
(#52) because it is the thing that carries dbt; a floating tag would decide
silently whether this feature exists.

Consumers get the capability, not the wiring. A gold step that runs from
the host keeps running from the host until it is rewritten as a task. Three
requirements shape that rewrite, and they are here because they are easy to
miss:

• dbt run and dbt test as separate invocations — a caller depends on
  their exit codes separately.
• target/run_results.json surfaced back, since a caller reads that artefact
  rather than stdout and should assert args.which == "test" before believing
  it.
• a failing dbt test exits non-zero while still having produced its
  artefacts, because a failed run must still record what it measured.

condition_task is decided in the shim (#50, #54)

A condition needs no engine, so it is evaluated here and works with no Spark
attached. depends_on.outcome selects the arm, which is the only thing that
can: a condition SUCCEEDS either way, so branching on task state cannot work.

Witnessed on both operator families over identical operands, with both arms
writing — so a single-family implementation and a broken outcome gate each fail
differently rather than both looking like "the wrong branch ran".

python_wheel_task, run_job_task and for_each_task are now refused by
name instead of reaching a generic error. An unimplemented task type should
say which one it is.

Two tasks in one wave keep their own parameters (#56)

A witness, and one worth reading the caveat on: it is red against the pinned
agent 4.2.0 and green against one built from the fabric-emulator fix. It is
therefore not in CI until SPARK_CLIENT_DIGEST moves. The test is here so
the gap is visible rather than remembered.

Security and toolchain (#55, #57, #58, #60)

govulncheck reported standard-library vulnerabilities against Go 1.25.11;
they clear at 1.26.6, and the build image moves with it. Secret and
vulnerability scanning are now part of CI, which reads the Go version from
go.mod rather than repeating it (#60 finishes that for the condition_task
job).

sqlparse is overridden past dbt-core's upper bound: four advisories name
0.6.0 as first patched and dbt-core pins <0.6.0, so a constraint cannot
resolve and an override is the only honest way through. Both host-side dbt
gates pass on the bump.

Also: three verified drifts fixed in the roadmap and family-integration docs
(#49).

Upgrading

Nothing to do. dbt_task is additive, and the task types now refused by name
were already failing — they just say which one they are.