Skip to content

ods 0.1.1

Choose a tag to compare

@github-actions github-actions released this 01 Nov 21:47
· 81 commits to main since this release
70be81c

The dashboard is complete: every screen in its design is built (#309), so this is the
first 0.1 release (ADR-0019 §1a). 0.1.0 was prepared but never released; 0.1.1 is that
release, with security updates to its dependencies and a new look for the docs site.
Beside the dashboard, health checks become configurable and pluggable, and ODS takes
plugins, built into a custom ods or run as programs of their own. It changes SDK contracts and the health record's format: see Breaking.

  • The rest of the dashboard: Settings and About, the Semantic layer placeholder,
    the Run page's Threads tab with the critical path, saved table versions as freshness
    evidence, and dark mode checked on every screen.
  • Health checks you configure: declared checks, coverage targets, read-only SQL
    probes under a login shown to only read, script checks, and [health] after_run to
    check after every build. Anything a project defines that runs code or SQL runs only
    once trusted with ods health trust.
  • Plugins: health checks and warehouse plugins in a custom ods (ods_cli::Ods),
    or out of process in any language ([plugins.<name>]); ods plugin list, show
    and test. Databricks is now a built-in warehouse plugin, and DuckDB a second one.
  • SDK 0.9. Rebuild out-of-process plugins and custom ods builds against it.

Breaking

  • The error_catalogue contract is 0.5 (#415): ErrorCatalogue::classify_attributed
    names the catalogue that recognised an error, for a catalogue that consults others.
    It has a default, so what to do: nothing beyond rebuilding against this SDK.
  • SDK_VERSION is now 0.9 (#392): the health_check contract is 0.2, as NodeFacts
    gains test_types, described and constraints (all default when absent). What
    to do:
    rebuild out-of-process plugins against SDK 0.9; a check needs no other change.
  • The relation_probe contract is 0.3 (#392, ADR-0030 §5, §4e). 0.3: a probe statement
    can name the relation's parts as string literals (ProbeStatement::by_name,
    {database}, {schema}, {name}); an implementation that can't write a part as a
    literal safely answers unknown and runs nothing (conformance case
    a_name_that_cant_be_a_literal_is_never_probed). 0.2: RelationProbe::probe takes
    ProbeTargets (an id and a name, for any node with a relation, not only sources) in
    place of RequestedSources, ProbeReport::sources is targets, a ProbeRequest can
    carry a timeout, and statements run only against the requested targets' relations.
    The dbt provider now passes the requested ids to dbt show (several calls for a long
    list) instead of running the statements against every source. What to do: a
    provider implementing RelationProbe takes &[ProbeTarget], runs nothing against a
    relation it wasn't asked about, and honours ProbeRequest::timeout.
  • ods_cli::plugins::Origin holds owned names (Cow<'static, str>), so an
    out-of-process plugin can have one, and is no longer Copy. What to do: build one
    with ods_cli::origin!() or Origin::new(name, version), and clone it where you
    copied it.
  • The health record's format is 1.2, as a check's source can be declarative, probe
    or script and a report holds its coverage targets' verdicts. ODS 0.0.2
    doesn't read 1.1 or 1.2 records: run ods health check again after going back to it.

Added

  • [health] after_run = true (#392, ADR-0030 §6): every ods state command that builds
    runs the health checks once it succeeded and is recorded, as ods health check
    would, and records them for the dashboard. The report ends with their counts and
    names what fails at severity error (health in --json). Their verdict never
    changes the build's exit status: gate CI with ods health check. Off by default.

  • An out-of-process warehouse plugin gives everything a built-in one does (#392,
    ADR-0031 §3d): "Open in warehouse" links (link, asked for every relation of the
    project by ods state, ods health check and ods lineage), error explanations
    (classify, asked for a failed run's errors before dbt's catalogue), observed
    lineage (observed_lineage, for ods lineage compare --observed), its SQL dialect,
    and probes over its own connection (probe and privileges, with [health.probes] plugin = "<name>" in place of target; the login it reports is the login it probes
    with). The dashboard and MCP never start a plugin: the commands that run one keep its
    links and explanations beside the state store (<state db>.plugins/, format 1.0),
    and they read them from there. A plugin's link must be a safe https:// URL, or the
    relation has none (NoRelationLink::NotAnswered, new). ods plugin test runs the
    link, catalogue (--errors) and lineage (--export) suites too. ods_sdk::protocol
    gains the requests and answers; the probe and privilege types are serde types, read
    only through their checked constructors.

  • Out-of-process plugins (#392, ADR-0031 §5): [plugins.<name>] with a command (never
    run through a shell) and a timeout declares a program, in any language, that speaks
    the plugin protocol over stdin and stdout. ODS asks it to describe itself, refusing
    it when a contract it implements was built against a version this ods can't accept,
    then calls it: its health checks run in ods health check (tuned with
    [health.plugins.<id>]), and a warehouse it serves gives the ods state commands
    that build their sources' data versions, of the kind its capabilities name
    (relation_versions or source_freshness). A plugin a project declares runs only once
    trusted with ods health trust, or with --allow-scripts (now also on the ods state commands that build) for one run. One that can't run fails ods health check
    (ODS-E0802) and is a warning in ods state; a call that fails makes what it was
    asked about unknown. ods plugin list lists the declared plugins; ods plugin test <name> runs the SDK's conformance suites against one (ODS-E0803 when a case fails).
    ods_sdk::protocol gains DescribeRequest/DescribeResponse and
    VersionsRequest/VersionsResponse; RequestedSource and SourceVersion are serde
    types.

  • Script checks (#392, ADR-0030 §4): [[health.checks]] with kind = "script" runs a
    command you name (never through a shell) in ods health check, writing the selected
    nodes to its stdin and reading its findings from its stdout as JSON, the script
    protocol 1.0 (ods_sdk::protocol). It takes params and a timeout (60 seconds by
    default). A project's scripts run only once trusted with ods health trust, which now
    shows each script's command; trust also covers the files the command names in the
    project, so a commit that changes one needs trust again; a script that fails, times out or answers with anything
    but the protocol makes its nodes unknown, never a pass; a timeout stops every
    process it started. The dashboard shows what the
    last run recorded. The new ods-provider-process crate is the only code that starts
    one.

  • DuckDB is the second built-in warehouse plugin (#415, ADR-0031 §3a), in
    ods-provider-duckdb: DuckDB's error patterns, recorded from real dbt-duckdb runs,
    and the duckdb dialect. It offers no source versions (DuckDB has no table version
    to read), so ods state reads sources.json alone, as before. ods plugin list,
    ods version and ods doctor list it.

  • A probe check's sql may give a query per warehouse kind, with a default (#415,
    ADR-0031 §3b): sql = { databricks = "…", default = "…" }. The query for the
    project's warehouse runs, else one for a warehouse it is built on, else default;
    with none, the probe is unknown, never run with a guess. Each query is checked as
    read-only in its own warehouse's dialect, and trust covers them all. A probe with one
    query is unchanged and stays trusted. Findings' evidence names the warehouse the query
    was given for (sql_for); ods health trust --json gives sql as configured (a
    string, or a table by warehouse), and the dashboard shows the query for the project's
    warehouse.

  • A probe's query in its warehouse's own SQL (e.g. Databricks' version as of) is no
    longer refused before the project is read (ods serve, ods health trust): a query
    for every warehouse must then be one read-only query in some dialect ODS knows, and
    ods health check checks it in the project's own dialect before it runs.

  • ods plugin list and ods plugin show <name> (#415, ADR-0031 §3c): the plugins this
    ods runs with and what each offers, detected by asking it, in human, plain and
    --json forms. show gives each feature's contract and version, and a warehouse's
    parents and what it inherits from them. An unknown name is a usage error
    (ODS-E0801, exit 2) listing the plugins there are. The MCP server's
    ods_list_plugins tool answers the same, and docs/plugins.md lists the built-in
    warehouses from the same detection (a test keeps them in step).

  • The dashboard's About page (/settings/about, #415, ADR-0031 §3c): this ods's
    version and SDK version, and each plugin with what it offers, from the same detection
    as ods plugin list. It names the plugin serving the project's warehouse, or says
    that none does and what ODS then knows less about. A feature the configuration
    can't use says why, never with a setting's value. /api/settings/about returns the
    same view model.

  • The dashboard's Settings page (/settings, #351): the effective configuration
    as ods config explain lists it (secrets as references, credentials in values
    removed as ods doctor removes them), the project, target and state store a run
    resolves, the configured providers and their capabilities, ods doctor's checks
    that only read (no dbt, no connection), and the Server mode placeholders. Nothing on
    it writes; beyond loopback it shows no value, path or check detail.
    /api/settings returns the same view model.

  • The dashboard's Semantic layer page (/catalog/semantic, #352), a read-only
    placeholder. It lists the semantic models and metrics manifest.json declares:

    • each semantic model's model, measures, dimensions and entities;
    • each metric's type, what it is computed from, its dimensions, and the models it
      depends on.

    A project without a semantic layer gets an empty state. Artifacts that can't record a
    semantic layer (the Information Schema) say so instead, and a definition that can't be
    read is named rather than dropped. Nothing is queried or validated.
    /api/catalog/semantic returns the same view model.

  • The Run page's Threads tab (/state/runs/<run>?tab=threads, #355): the run by
    thread, a bar per node coloured by outcome and the idle stretches between them, with
    the critical path (the chain of dependencies that set the run's length) outlined and
    named. A bar opens the replay on the DAG at the node's start, with the node selected.
    A journal rebuilt from final results has no start times, so it gets no bar: the tab
    says so and lists when each node finished. /api/state/runs/<run> has the same view
    as threads.

  • Saved table versions (#388, ADR-0022 §7): ods state build and run (and their
    --dry-run) save the sources' table versions they read in
    <state-db>.versions.json. ods state plan, ods state explain and the dashboard,
    which never connect, now decide with them and say which command read them and when;
    the Freshness evidence screen shows them as the evidence now. A version read before
    a model's last build still says nothing about data since, so its readers build until
    ods state build --dry-run reads the versions again.

  • Plugins in a custom ods (#392 phase 6, #387, ADR-0031). ods_cli::Ods is the
    released CLI as a library: a crate of your own adds health checks
    (Ods::health_check) and warehouse plugins (Ods::warehouse) and runs it.

    • Warehouse plugins: a WarehousePlugin serves one warehouse kind (dbt's
      adapter_type). Over dbt's own connection it gives ods state a source-version
      provider and probe checks a login check; from the warehouse's [providers.<name>]
      settings it builds "Open in warehouse" links; it reads observed lineage exports;
      its error catalogue is asked before dbt's when a node fails; and it names the SQL
      dialect column lineage parses in (#415, ADR-0031 §3a). As dbt's adapter dispatch
      does, a warehouse inherits error patterns and the dialect from the warehouses it is
      built on: its plugin's parents (Databricks: Spark), or
      [warehouses.<kind>] extends in configuration (§3b). A cycle is a configuration
      error. What a
      plugin offers is detected by calling it, never declared (§3c). Databricks is now
      the built-in plugin, and behaves as before. ods version and ods doctor now also
      list its links and observed lineage. With several plugins that read observed
      lineage, --observed uses the project's warehouse's, and a project on any other
      warehouse is told which plugins read exports.
    • Plugin checks: they run with the others in ods health check.
      [health.plugins.<id>] sets their severity (or off), select and exclude; an
      id that names no plugin check is a configuration error.
    • Visibility: ods version (plugins in --json) and ods doctor
      (capabilities.plugins) list every plugin, saying which are built in.
      ods doctor --provider plugin selects the checks about a custom build's warehouse
      plugin.
    • SDK: PrivilegedProbe, one object that probes and reports its login's
      privileges, and Provider and RelationProbe for Arc<T>, so a plugin can wrap
      the connection it is given.

    examples/custom-ods is a complete custom ods, with a health check and a
    source-version provider that pass their conformance suites. docs/plugins.md shows
    how to write one.

  • Probe checks on Databricks check their login in Unity Catalog (#392, ADR-0030 §4e):
    ods health check reads system.information_schema through the probe target's own
    connection and runs a probe only on a managed table, view or materialized view whose
    login (or its groups) owns none of the relation, schema, catalog or metastore, holds
    nothing on them beyond SELECT, BROWSE, USE CATALOG and USE SCHEMA (nor on
    the metastore beyond the default USE MARKETPLACE ASSETS), and isn't a workspace
    admin; anything it can't read, and any external
    table, is unknown. Known limits (e.g. account admins aren't visible) and how they
    will be fixed are listed in ADR-0030 §4e. The Databricks provider has UnityCatalog,
    a relation_probe that also reports relation_privileges.

  • Probe checks run (#392, ADR-0030 §4c): on their own dbt target, [health.probes]
    target (and optional profile), never the build's; without one they don't run.
    dbt probes sources, models, seeds and snapshots, matching the query's columns whatever
    their case. Every probe is refused (unknown) unless its login is shown to only read,
    which no provider reports yet; ods health check --allow-elevated-login runs them
    anyway, at the user's own risk, warning on every run with the login and what was
    found (in --json, a warning diagnostic ODS-W0704), and marking each such
    finding login_check: overridden. A probe reads only the relation the build made: a
    node dbt resolves elsewhere under the probe target is unknown. A probe target equal
    to the build's is a configuration error, and the trust digest covers [health.probes],
    and the dbt program, project_dir, profiles_dir and profile configuration
    sets, so pointing trusted probes at another target or login needs trust again. A node
    the manifest names no relation for (e.g. ephemeral) is never probed; sources can't be
    probed yet, and a probe that selects nothing is reported (ODS-W0705) and fails
    --strict at severity error. ods health check
    takes --dbt, --profiles-dir, --dbt-profile, --vars and --dbt-output.

  • A ProbeTarget can name the relation it must be (expecting); relation_probe
    answers unknown for a node it finds elsewhere, and its conformance suite checks it
    when a harness has such a target (ProbeHarness::elsewhere).

  • The relation_privileges contract and capability (#392, ADR-0030 §4c), with a fake
    (FakeRelationPrivileges) and a conformance suite: a provider reports whether its
    login can only read each relation, or what it holds beyond reading. The health engine
    asks it just before every probe, refuses a probe whose login can do more than read or
    can't be shown to only read (unknown, naming the login and what it found), and judges
    pass on the row a probe returns. A run that allows an elevated login marks each such
    finding login_check: overridden and keeps the login and what was found in the report
    and health record (elevated_login). ods health check wires a probe connection in
    the next step of #392.

  • Probe checks and the trust store (#392, ADR-0030 §4a–§4d): [[health.checks]] with
    kind = "probe", select, sql (one read-only query with {relation}) and pass
    (e.g. n > 0). A probe whose SQL could write, lock or change a session is a
    configuration error before anything connects (exit 4). A probe a project defines runs
    only once ods health trust has trusted that exact definition for that project; trust
    is kept in trust.json in your own configuration directory, a changed query is
    untrusted again, ods health trust --revoke forgets a project, and
    ods health check --allow-scripts trusts for one run only. Until then a probe is
    unknown, never a pass. Probes run under a read-only login in the next step of #392.

  • Declared health checks (#392, ADR-0030 §2–3): [[health.checks]] in ods.toml with an
    id, select/exclude, require (description, tests, test:<type>,
    constraints, tag:<tag>) and a severity (default warn). They run on the
    project's metadata in the dashboard's badges and in ods health check's gate, and
    name what each failing node is missing. A check that can't run as written is a
    configuration error (exit 4).

  • Coverage targets (#392, ADR-0030 §3): [health.coverage.<measure>] with a target
    share and a severity (default warn) for Home's measures (tests,
    descriptions, constraints, source_freshness). Home draws each target on its bar
    and says whether it is met (target on each /api/home coverage row), and
    ods health check lists them (coverage in --json) and, for one missed at
    error, exits 5. Nothing to measure is unknown, never met.

  • The dashboard counts the checks ods health check recorded (#392, ADR-0030 §6): it
    works the built-in checks out itself, and takes every other check's findings from the
    newest health record for its scope, in the badges and counts on Home and in the
    Catalog. The "how" text names those checks and when they ran, /api/home and
    /api/catalog give the time as health_recorded_at, and a node the record doesn't
    cover is unknown for them, never healthy. A new record reloads the page.

  • The docs site has a Meet the gang page (Project): Memo, the purple elephant who is
    ODS's mascot, her daughter Tui, and Tui's Nana and Grandpa. Tui also keeps the site's
    404 page.

Changed

  • The dashboard meets WCAG AA contrast on every screen, light and dark (#353). Three
    light colours are darker than before:

    • the faint grey (separators, —);
    • success green;
    • warning amber.

    The ERD legend's ? is no longer black in dark mode. A browser test measures every
    screen, and each screen has a dark still in the dashboard tours.

  • dbt's error catalogue is at version 8 (#415): dbt-databricks's own messages (unavailable
    compute, timeouts, missing credentials) are now the Databricks plugin's catalogue
    (databricks 1), and DuckDB's (Binder Error, Catalog Error, Conversion Error
    and the rest) the DuckDB plugin's (duckdb 1). Each is asked first on its own
    warehouse's project, and after dbt's on any other (a run before dbt names the
    warehouse, or from before a project moved), so explanations recognise what they did.
    An explanation names the catalogue whose pattern recognised the error (databricks
    1, duckdb 1 or dbt 8), and a warehouse's message reads as its adapter's
    (dbt-duckdb said:).

  • The docs site has one roadmap page, /ROADMAP/ ("Roadmap" in the navigation), in place
    of "Overview" (/roadmap/) and "Full plan" (/ROADMAP/), which had become copies of
    each other. /roadmap/ redirects to /ROADMAP/.

  • The README, the docs' Home and Install pages and the roadmap say v0.0.2 is released;
    the install commands use 0.0.2 as the example version.

  • ods health trust highlights each probe's SQL on a terminal: keywords, strings,
    comments and dbt's Jinja blocks in the standard terminal colours. Plain and JSON
    output are unchanged.

  • The docs site has a new look and layout: top-level tabs (Get started, Guides,
    Reference, Architecture, Contributing, Project) in place of one long sidebar, a slate
    and teal theme with a logo, a new home page, and one dismissible pre-alpha banner.

Fixed

  • ods serve bound to loopback but behind a proxy (--allow-host) no longer shows
    local paths, configuration values or error text: requests may come from elsewhere,
    so every page treats it as beyond loopback (#351).
  • The docs site's CLI reference showed a backslash before each | in table cells
    (ods state compile\|run); it shows ods state compile|run, as on GitHub.

Security

  • Dependencies with RustSec advisories published since 0.0.2 are updated: tokio 1.54.1
    (RUSTSEC-2026-0386), zstd 0.14.3, zstd-safe 8.2.0 and zstd-sys 2.1.2
    (RUSTSEC-2026-0391, RUSTSEC-2026-0398, RUSTSEC-2026-0403, RUSTSEC-2026-0410), and
    pulldown-cmark 0.13.6 (RUSTSEC-2026-0414).

Install

pip install opendatasuite==0.1.1        # or: pipx install / uv tool install opendatasuite
cargo binstall --git https://github.com/buchochelliq-labs/open-data-suite ods-cli

Homebrew, Chocolatey, direct downloads and verification: https://buchochelliq-labs.github.io/open-data-suite/install/

Checksums

60a5442e0983fccdf62752c0f0971d81e3b6789298540d490330a469c1efca7a  ods-v0.1.1-aarch64-apple-darwin.tar.gz
4f5a3518abc369c92f11a9926b042c89d44208a6463d799f575e732c9b971002  ods-v0.1.1-aarch64-unknown-linux-musl.tar.gz
a14265f368bd131cb74c692a121d71fc847ade741dc512b89614362469e7718d  ods-v0.1.1-x86_64-apple-darwin.tar.gz
ef9f880651b3ef7db7ac8ada6e9fdaaed7c221be2688719cc2181d3b72bf0024  ods-v0.1.1-x86_64-pc-windows-msvc.zip
c93feb64b2cc40f76a96fbbf0a05ece1566ee66c8ea750006507dfad05b15184  ods-v0.1.1-x86_64-unknown-linux-musl.tar.gz
e670fc35cdae3f70ef738bc7a828f93212cac57669148eee8e5262807421e80a  ods.rb
6a14c408a4bed7805e0290ac6aec99b8e23a2aa8351d073d803a1c01a47e0f18  opendatasuite-0.1.1-py3-none-macosx_10_12_x86_64.whl
42d5690f906688a67b60b326da1276298305c782885cce84b0218fd49ce1271e  opendatasuite-0.1.1-py3-none-macosx_11_0_arm64.whl
39ce124159fb252a10363a75e9b894d03f30bd4aca0b9578ff466f0ba5e6172b  opendatasuite-0.1.1-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
c7225eb1955d07df6aeb51ef353080b77f33124f686fb2fe06b56247b7496d98  opendatasuite-0.1.1-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
a22a467f5a0b0aeee1e83beb66b01520eccf29d155086025df228c3e7cc4e1d5  opendatasuite-0.1.1-py3-none-musllinux_1_2_aarch64.whl
a390263d26f41056f52838d8474a62b465fd4c9b6c5979b249d19682a3182d7a  opendatasuite-0.1.1-py3-none-musllinux_1_2_x86_64.whl
81357bcd4a525ef7a5eb95be8f292fc9e284c9f486fcc463a09dc7a4eca75c8f  opendatasuite-0.1.1-py3-none-win_amd64.whl