v0.2.18
A large release combining a feature roadmap (15 features across 5 phases)
with a sweep of correctness fixes uncovered by a real-world test-run
review (CTE scoping in lineage/check, diff non-determinism on aggregated
decimals, progress bar floods on non-TTY, and several API edges).
Added
havn shell: psql-style multi-line REPL with readline history,
\dt/\d/\dn/\dfslash commands,\timing,\copy, server-aware
routing throughhavn serve.havn explain <model>with--analyze,--json,--raw. Surfaces
DuckDB's plan tree using the existingengine/explain.pyprimitives.
API counterpart atGET /api/models/{name}/explain[?analyze=true].havn diff --exit-nonzero-on-change: exit code 2 when models have
row/schema changes, for CI gating. Composes with--format json.havn initseeds.sqlfluff: relaxed default config (excludes
RF03/AM05/ST06/LT05) so new projects don't drown in violations on
idiomatic SQL.- Model directives:
@grain <cols>: synthesises a uniqueness assertion post-build.@owner <label>: propagates onto every assertion result for alert
routing.@assert ..., severity=warn|error: assertions can warn-only
(continue) or halt downstream models on failure.@source_freshness <table>, max_age=24h, on=<col>, severity=:
pre-build contract; stale sources skip the model and (for error
severity) cascade-skip downstream.@watermark <col>: one-line incremental sugar that synthesises the
WHEREclause; equivalent to writing the fullincremental_filter
by hand.
- Downstream models are skipped with status
skipped_upstream_blockedwhen an upstream errors or fails an
error-severity assertion. havn freshness --sourceswith--source-min-rows N: surfaces
upstream row counts and max-on-column timestamps from each model's
@source_freshnesscontracts. Resolves "fresh model on top of
zero-row source". API counterpart accepts?include_sources=trueand
?source_min_rows=N.- Stdlib PII macros (
havn.stdlib.pii):mask_email,mask_phone,
mask_fnr,mask_credit_card,mask_ip,hash_consistent.
Auto-registered for every project, even those without amacros/
directory. User macros with the same name shadow stdlib (warning
logged).havn macroslists stdlib entries with origin tag. policies.denyinproject.yml: column-level deny-list
("column X may not appear in schema gold"). Caught at compile time
byhavn checkAND enforced at build time byhavn transform
(denied models markedpolicy_deniedbefore any tier executes).havn watch --route <glob>: filter watched paths and rebuild
only the matching model, not the whole DAG.- Editor "Run on save" toggle: persists in localStorage; saves
chain intorunSingleModelfor transform.sqlor
runCurrentScriptforingest/export .py. - TablesPanel structured docs: per-column descriptions on hover
and inline rail; model-level grain / owner / description block above
the column list.
Fixed
- CTE scoping in
havn lineage: multi-source models with CTEs
mis-attributed columns because_extract_sourcesdefaulted
unqualified columns todepends_on[0]and CTE references leaked
into the table alias map. Now builds a separatecte_alias_map,
threads it through, and resolves unqualified columns from the
per-SELECT FROM/JOIN scope (with information_schema as tiebreaker). - CTE false positives in
havn check: false-positived on CTE
columns (e.g.flows.inflow_nokgot looked up against
silver.fact_transactions) and onb.*star expansions (sqlglot
represents them asColumn(name="*", table=b)). Now builds a CTE
name set + per-CTE column set, validates qualified CTE-column refs
against the CTE outputs, and short-circuitsname=="*"tokens. havn lineageCLI now opens the warehouse soSELECT *and
unqualified columns can resolve viainformation_schema.havn diffnon-determinism: reported +N/-N on identical content
becauseEXCEPTis type-sensitive (temp-rebuilt columns drift on
DECIMAL/DOUBLE precision fromSUMs). Switched to MD5 hash of the
per-column VARCHAR projection with a presence-prefix NULL sentinel
(V:/N) that can't shape-collide with real data.- DuckDB progress bar flood on non-TTY stdout:
enable_progress_bar
writes carriage-return updates that turn into thousands of newlines
when stdout isn't a TTY. Now gated onsys.stdout.isatty()/TERM
with aHAVN_PROGRESSenv override. POST /api/transformwith no body: returned 422; body is now
optional viaBody(default_factory=...).- Unknown
/api/*GETs returned the SPAindex.htmlwith status
200; the catch-all now 404s anything under/api/. havn querytruncation at the server-side 50k row cap was
silent; now surfaces a yellow warning to stderr (so CSV/JSON
piped output stays clean) when the response'struncatedflag is
set.havn lintdefaults dropped RF03
(unqualified-reference-in-single-table) from the correctness rule
list; it fires on idiomatic SQL and produced ~37 violations on a
12-model project. Also pinned
unqualified_single_table_references=allowin the pyproject
sqlfluff config.run_assertionsearly-returned on empty list, so a model with
only@grain(no@assert) never had its grain check run.
Removed the early return; grain now always evaluates.havn checkcaughtpolicy.denybuthavn transformbuilt the
model anyway. Hoisted deny evaluation into_evaluate_deny_rules()
called fromrun_transform; both sequential and parallel runners
now pre-mark denied models aspolicy_deniedbefore any tier
executes.- Parallel runner blocked every later tier on ANY previous-tier
failure. Made blocking dependency-aware via_is_blocked()
walking the actualmodel.depends_ongraph; siblings of
failed/denied models now build correctly. check_freshnesswithinclude_sourcescrashed on
timestamp-with-timezone columns when pytz wasn't installed. Cast
MAX(<col>)to VARCHAR in SQL so the value never crosses the
DuckDB to Python boundary as a Python timestamp._parse_durationnow warns and falls back to 24h on malformed
input (previously crashed withValueErroronmax_age=invalid).parse_assertion_specsstrips unrecognizedseverity=
qualifiers (e.g.severity=critical) instead of leaving them in the
expression where they crash_evaluate_assertionas bad SQL.- Shell statement detector rewritten as a single forward pass:
SELECT 1; -- trailing commentnow correctly recognised as
complete. Comment-swallowed semicolons and unterminated string
literals handled correctly.
Internal
- New
_havn.source_freshnesstable;_havn.assertion_results
migrated withseverity+ownercolumns. SQLModelgainsgrain,owner,source_freshness,watermark,
assertion_specs.AssertionResultgainsseverity,owner.generate_structured_docssurfaces grain/owner/source_freshness per
model so the SPA can render them.register_macros()always loadshavn.stdlib.*(even with no user
macros/dir); user macros override on name collision.- ducklake-extension tests skip with a
requires_ducklakemarker
(probes once per session) when the extension can't be installed
fromextensions.duckdb.org, instead of failing with HTTP 403
noise.