Skip to content

v1.11.0

Choose a tag to compare

@jarichb jarichb released this 02 Oct 20:49
· 52 commits to main since this release
Immutable release. Only release title and notes can be modified.
b8f680c

Added

  • model2data guide [setup|triage|tune] prints one page of Markdown written for a coding
    agent about to use model2data: setup (install, a starter model, the first run), triage
    (what validate, generate and dbt build print and what to do about each, in order) and
    tune (every option, model key and exit code). Without a topic it picks triage when the
    current directory holds a *.model2data.yml or *.dbml file, else setup, and says so on
    its first line. The pages ship in the package, so they match the installed version; the
    command reads only file names in the current directory and writes nothing.
  • Spec 0.4.0: a column can depend on another column's value, with when. completed_at with
    generate: {after: created_at, when: {status: [done]}} holds a timestamp on every done task
    and is null on every other one, where a todo task used to get a completed_at and a
    cancelled subscription none. when maps another column of the table (an enum, boolean,
    number or text column) to the values it must hold; several columns must all match, and a row
    whose named column is null does not. On the matching rows the column is generated as before
    and never null, unless it has a null_rate, which then counts only those rows. It composes
    with after, the time shape hints and distribution. On days after the first, a row an update
    moves into a listed status gets its value that day (its updated_at when the table has one),
    and a row moved out loses it. Each when writes a dbt test, model2data_when (and a
    null_rate beside it model2data_when_max_null_share, over the matching rows), shipped in
    macros/model2data_when_tests.sql, and the defects report checks it like any other. The
    model's checks say what is wrong with one: a column that is not there (and which one was
    meant), a value its enum does not have, a not_null, keyed, unique, foreign-key or defaulted
    column carrying it. In DBML it is a note hint like the others:
    [note: '{"when": {"status": ["done"]}}']. when is spec 0.4.0, which adds it and nothing
    else: a model using it says model2data: 0.4.0 (DBML with a when note converts to 0.4.0),
    when in a 0.2 or 0.3 document is an error saying to write 0.4.0, and 0.2 and 0.3 documents
    read exactly as before. A writer keeps the version a document was read with, and writes 0.4.0
    once it uses when. The schema's $id is now
    https://www.jbanalytica.com/model2data/spec/0.4.0/model.schema.json (0.2.0 and 0.3.0 stay
    served, see model2data/spec/PUBLISHING.md); the spec README, model2data guide and LLMS.md
    describe it.

Fixed

  • A foreign key cycle with a nullable link is generated, not flagged. Tables were ordered by
    every foreign key, so accounts.owner_user_id -> users.id beside users.account_id -> accounts.id left both tables in "an unresolved FK cycle", whatever their nullability, and one
    side pointed at rows that did not exist. A cycle is now broken at a table whose foreign keys
    into it are all nullable: that table is generated first, and those columns are drawn once
    their parent exists, from its rows. Only a cycle of required foreign keys is still reported.
  • Determinism: models without when generate exactly what they did before, dbt project
    included. In a model with when, only its columns change: what when decides is drawn from
    streams of its own, so every other column comes out as it does without the hint, except a date
    or timestamp that follows a when column (after it, or a later stage), which moves on a row
    where it would otherwise be earlier than the value when filled in.
  • Determinism: a model without an FK cycle generates exactly what it did before. A model
    whose cycle had a nullable link now writes different foreign key values in that cycle's tables
    (valid ones, where some used to dangle), and different columns mirrored from them.
  • PostgreSQL serial types are integers. serial, serial2, serial4, serial8,
    smallserial and bigserial were not recognised as integer types, so a bigserial primary
    key was filled with UUIDs and a bigint foreign key onto it failed with invalid literal for int() with base 10. They now count as integers everywhere an int type does (generation,
    min/max/distribution hints, the Int64 dtype, defect kinds, dbt seed column types), and
    a serial key generates exactly what the same model typed integer/bigint/smallint does.
    The declared type is kept as written.
  • Determinism: only a model with a serial-typed column generates differently (it crashed or
    produced UUIDs before); every other model is byte-identical.