Repository navigation
v0.6.0
Changelog
Added
-
A late row reaches SQLMesh. An
incremental_by_partitionentity atscd: type1may
declarearrival_lag:(30h,2d,1w). Its SQLMesh model pinsinterval_unit 'day'
and setslookbackto the lag in whole days, rounded up, so a row that lands after its
interval ran is loaded on a later run. A cron more frequent than daily on such a model is
refused by SQLMesh unless the caller also setsallow_partials. Declared on any other
entity, a type 2 entity included, it is refused. dbt emits nothing for it. A project that
declares no lag compiles as before, apart from the fingerprint header. -
A declared key is checked, not only trusted. Every entity with a
key:and no
dedupe:gets a generated blocking audit,<entity>_key_unique: no two rows share the
key, among the current versions on anscd: type2entity and among all rows otherwise.
dedupe:enforces the key by construction and removes the audit; auniquerule or an
incremental merge does not. On SQLMesh anincremental_by_partitionentity checks only
the run's own range of keys against the whole model; every other entity, and every
entity on dbt, audits the whole model.Upgrade note. A source that repeats keys now stops the build: declare
dedupe:on
the entity or fix the source. History written before the audit existed is not re-checked
by a scheduled run:sqlmesh runaudits only the range it builds, and adding an audit is
a metadata-only change thatsqlmesh planchecks over the plan's own start..end range. To
cover it, run the audit's query once over the whole model after upgrading, or deploy the
upgrade with aplanwhose range spans the history.
Changed
-
The MetricFlow manifest carries owners. A mart's
owner:reaches its semantic model
and a metric'sowner:its metric, both asconfig.meta.owner; with no owner,config
stays null. An entity's owner is not carried: an entity reaches MetricFlow only as entity
elements inside a mart's semantic model, which carries the mart's owner. An exposure
still emits nosaved_query— a saved query is a request someone wrote, an exposure a
consumer — and the manifest'ssaved_queriesstays empty.Upgrade note. A project that declares an owner on a mart, or on a metric the
manifest emits, compiles a differentsemantic_manifest.json, so its manifest hash
moves; a metric MetricFlow cannot serve is omitted, owner and all. -
bloomery_ir_versionis 26 (it was 25 in 0.5.0).EntityIRcarries the declared
arrival lag,Nonewhere none is declared, so every project's fingerprint moves, and with
it the header of every artifact that carries one. An IR written by 0.5.0 is refused by
plan()andir_from_jsonwith the version mismatch message; recompile the project.
Fixed
-
A replayed parent brings its children back. When a change lets a quarantined parent's
rows back in,bloomery planalso names inreplay_scopeevery child that quarantined
on it before the change, parents first, with a cycle ofreferentialrules broken at its first
member by name; a child that rewrote or flagged its orphans
(on_missing: unknown_memberorflag) is named inbackfill_scope. Before, the child
stayed orphaned and a blockingcoverage:check on the parent failed every run. Rows
bronze no longer holds keep their rewrite; the quality docs name the remedy. -
A fact from an upstream project grades as imported. The evidence guard now reads the
composed project, so a strict (requires_evidence: locked) mart over an imported base,
a strict mart listing an imported metric, and a strict exposure naming an imported mart
are refused, naming the upstream alias and fingerprint. The fix is
requires_evidence: assumedon the consumer, or carrying the requirement on the
upstream's mart.Upgrade note. A strict consumer resting on an imported fact compiled before and is
refused now. -
A dbt replay runs only against the project it was compiled with.
dbt_project.yml
carriesvars: {bloomery_fingerprint: ...}, and each replay macro refuses before its
first statement when that var differs from the fingerprint it was emitted under, or when
the entity or its reject table lacks a column the statements write. The refusal names
the missing columns and the rebuild that adds them:dbt run --full-refreshfor an
incremental model, which a plain run never widens. A fingerprint mismatch is fixed by
compiling again with bloomery into a clean directory, not by a rebuild. -
A schema change fails loudly on dbt. Every incremental dbt model, an entity's and its
reject table's, carrieson_schema_change='fail', so a field added, dropped or renamed
stops the build instead of leaving the table silently without the column.bloomery plannames the fix beside each such field change:dbt build --full-refresh -sover the
relations that fail, which are the entity when it is an incremental type 1 model and its
<entity>__rejecttable whenever it quarantines. Afulltable rebuilds whole and a
type 2 snapshot widens itself, so neither is named. The refresh recomputes history as
SQLMesh's rebuild does; a refreshed reject table loses its resolved rows, as the dbt
how-to says.Upgrade note. On dbt, a field change on an incremental entity, or on any entity that
quarantines, now needs the full refresh the plan names. -
A partition transform builds on Trino. The SQLMesh model spells
days(...),
months(...),years(...)andhours(...)in Trino's singular form (day,month,
year,hour), which Trino's Iceberg connector requires; it refused the plural. A
Hive catalog takes no transform at all: partition by a date column there. Measured on
Trino 483: a daily interval over a coarse partition overwrites nothing outside its own
range, so no interval unit is derived. -
A mapped vector is refused at the field. A mapping key or field, recipe or macro aimed
at avector(...)column is refused at type-check with its path. It reached emit and
failed there with no address; a vector comes from a step's declared output. -
The fuzz lane's first findings are closed. A refused spec no longer keeps its whole
validation graph alive, which grew a long-running caller's memory with every refusal;
WHILE*2andIF a, which sqlglot parses as a statement or gives up on as aCommand,
are refused as SQL text; and authored SQL text nests at most 32 levels, refused at load
for a spec field or a quality rule's expression, and at compile for a step body, a
registered macro body or the expression a chain of macros composes, instead of
overflowing the stack at a later re-parse.Upgrade note. SQL text nested deeper than 32 levels, or that only parses as a
statement, is refused now.