Skip to content

v0.9.21

Latest

Choose a tag to compare

@ashwin-agami ashwin-agami released this 29 Oct 09:48
· 105 commits to main since this release
d58ba00

Fixed

  • sm validate and the query runner work when started from inside the plugin's lib/ folder (#539).
    The folder held a partial copy of the model package (semantic_model/ with only __init__.py and
    units.py). With lib/ as the working folder, or first on the import path, Python loaded that
    copy instead of the installed package: python -m semantic_model.cli failed with No module named semantic_model.cli, and the query runner refused every statement with model_unavailable. The
    folder now ships units.py as _semantic_model_units.py, so nothing there can stand in for the
    installed package. Runs from anywhere else are unchanged.

  • sm remove-example names each example in full and says it keeps them (#550). It cut each
    question to about 50 characters, so two that opened the same way couldn't be told apart; its
    rejected labels and the curation log now carry the whole question. The output also adds an
    examples list (area, question, id when the example has one, status) and a note saying
    they were marked rejected (kept for audit), not deleted; the existing keys are unchanged. Its
    help says the same. sm curate --help no longer says --signer/--role are for approve ops only:
    they are recorded in the curation log on every op, and an approve (or an edit of an approved
    join's tables, columns or cardinality) also stamps them on the item as its sign-off.

  • Sensitive-column detection reads whole words, checks sampled values and looks inside JSON
    (#516).
    A new build split column names into words, so phoneme_error_rate, mobile_sessions,
    headphones_sold, adobe_campaign_id and ip_address are no longer flagged (they stay in the
    suspected list), while customer_name, contact_name, first_name, customer_ssn, zip_code,
    pan_number and e_mail now are. Every other name the old rule flagged is still flagged. The
    table's one sample (now taken for any table with text or JSON columns) also flags a column whose
    values are mostly email addresses, lists phone-number-shaped and birth-date-shaped columns as
    suspected (with a caveat on the column, shown by sm sensitive and the explorer's PII tab), and
    flags a JSON column whose keys hold personal data, with a caveat naming the key paths
    (Contains personal data at: email, contact.phone); a key that may itself be data (a person's
    name used as a key) is shown as *. Postgres JSON read through the CLI is recognised too. A column
    holding emails or phone numbers (digit-only ones included), or flagged or suspected as personal,
    gets no value list, whether read from the data or declared by an enum type or CHECK; a list
    with even one key that is a phone number is dropped, even when the sample is empty or unreadable. No sampled
    value is written to the model or the report. A birth date stored as a timestamp at midnight is
    recognised with or without a time zone (Z, +05:30), and digit-only values in an identifier
    column (customerId, trackingCode, id_number) are not read as phone numbers. ipv4_address/IPv4Address stay suspected-only. An existing
    model keeps its flags on refresh; no model field is added.

  • metric_binding_unknown_column resolves schema-qualified source tables (#500). The check
    looked a metric's source_tables up by bare table name, so s.orders matched nothing and a typo'd
    column in its binding was never reported, and sales.orders and archive.orders shared one column
    set, so a column on the other schema's table hid a typo. A qualified source now finds that schema's
    table (or a schema-less definition of the name), a bare one every table of that name, and a table
    whose name embeds its schema (name: public.orders) is found either way. It stays a warning.

Changed

  • A wide database's proposed subject areas follow its joins, and keep the plural (#513). A
    single-schema database of more than 24 tables used to get one area per table-name prefix, with
    any name starting with a shorter one's prefix pulled in (pro_x with products) and names
    singularised by dropping an "s" (supplie, statu). Tables are now grouped where they join each
    other most densely (greedy modularity over the join graph), with no target number of areas, so a
    hub table that half the database joins doesn't pull everything into one area; no group passes the
    area size limit, and ties go to the tables sharing more words in their names and descriptions.
    Tables with no joins go with tables sharing their whole first name word. Area names are the
    tables' own words, and the grouping no longer depends on the order the tables arrive in. Only
    proposals change: a curated model is never regrouped, while a model whose areas are still exactly
    the engine's (a batched first build, or one built by an earlier release and never edited) is
    regrouped by introspect --append. A join whose two tables end up in one area is written as that
    area's join, and a refresh that would write a file the model can't load back is refused, with the
    model left as it was.

  • sm suggest-metrics no longer proposes a share-of-true rate for every boolean column (#530).
    On a large onboarding these <table>_<column>_rate metrics were 698 of the 706 it suggested and
    about half of all metrics in the models, while the business's own definitions named one. They are
    now left out by default; the output carries one flag_rates summary instead (how many boolean
    columns could have one, up to 20 of them by name, and how to ask). --flag-rates <table.column>[,...] proposes the named ones (matched case-insensitively, schema.table.column
    when the table name is in more than one schema; a name that matches no boolean column is reported
    in errors, and one that matches more than one table is refused before anything is written, with
    the matches listed), and --flag-rates all proposes one for every flag column
    as before. --cite "<document>" records where a requested rate is defined: its description reads
    Defined in: <document>.. Requested rates go through the same dry run, --write, --names and
    --profile test run as every other proposal. Rate metrics already in a model are not changed; the
    explorer's Review tab can reject them in bulk. The agami-connect skill requests a rate only when the
    customer's definitions name it.

Fixed

  • sm add refuses a subject area the model doesn't list, and takes --area _cross (#513).
    The loader reads only the areas datasource.yaml lists, so a metric added under any other name
    was reported applied and validated and then never served. The error now lists the model's areas.
    --area _cross writes the datasource's cross-area list (cross_subject_area_metrics.yaml, or
    _entities.yaml), validated like any other add, and those items can be signed off through
    sm curate, where "area": "_cross" reads only the cross-area lists and otherwise acts as an op
    with no area does (a missing name, or one in both places, fails the batch). Re-adding an item that datasource.yaml holds inline replaces it there, so one name is
    never defined twice.

  • A foreign key over several columns is now one join on all of them, and a query joining on only
    some of them is flagged (#510).
    Introspection wrote each column of a composite key as its own
    approved many-to-one join, and either one used alone repeats rows (981 became 2,502 in one
    database). Every dialect's foreign-key query now returns the constraint name and column position,
    and the generic information_schema query (Redshift, Snowflake, DuckDB, Databricks) and BigQuery's
    match the referenced column by position, which they used to pair with every column of the key.
    Where a catalog can't run the generic one (an older Redshift may lack
    position_in_unique_constraint), introspection falls back to the query it used before, keeps its
    single-column keys as before, and leaves out a composite key with a note rather than guess how its
    columns pair; the report says when the fallback was used. A
    key is written as one on: join, its cardinality read from the whole key; single-column joins are
    unchanged. Where a catalog gives no constraint name, the columns are still kept apart but left for
    review. A refresh replaces the split joins introspection approved for such a key with the one
    join and lists that in the preview; if a person signed off any of them, nothing changes for that
    key and the preview says so. On the receipt, a join written on part of a declared composite key
    makes an aggregate on either side multiplied, with a fan_trap naming the missing columns; the
    query still runs. Code that read only from_column/to_column now reads every column of an
    on: join too (adding reference joins no longer raises on one), and the validator warns when an
    on: names a column its table doesn't define. A join that is a key equality plus a fixed value
    (... AND customers_curr.current_flag = 'Y') is not a composite key: its receipt and refresh are
    as before, its key column counts as a referenced column, and hiding that column now hides the
    join. BigQuery and Oracle keys are kept apart per dataset/owner, a BigQuery key into another
    dataset keeps its single-column join, and an on: now also parses in the engine's own grammar
    (`OrderId`). A refresh keeps a single-column FK the catalog declares beside a composite
    one, and only an on: join introspection signed goes back to review when a column is retyped.
    The partial-key check reads each join on its own (an aliased self-join and a USING join too),
    skips SEMI/ANTI joins and a join another declaration names whole, and counts only the equalities
    between the join's two tables. A column an earlier USING or NATURAL join merged counts for
    every table it merged, and one whose table can't be told leaves the aggregate undetermined. A
    key column or table named after a reserved word (order) is quoted in the on:, and a refresh
    matches a split join by its schema even when the schema is written in the table name
    (crm.shipments), and keeps such a join in its own schema's area only.
    A refresh's relabelling
    from the database's keys (#511) labels a composite join from all its columns, and a split join the
    refresh replaces is not also listed as relabelled.

Fixed

  • Column scope checks an unqualified column that sits beside a CTE or subquery (#339). When a
    query read a declared table and a CTE or derived table in the same FROM, an unqualified column was
    skipped, since the CTE might have supplied it. A column the model excludes was then returned:
    WITH c AS (SELECT 1 AS n) SELECT secret_col FROM orders, c ran, while the same statement without
    c was refused. The gate now reads what each CTE, derived table, LATERAL subquery, VALUES list
    and table function names in its projection or alias. A column one of them names passes. Otherwise it
    is refused when that scope reads a declared table, and a scope with no source supplying the name
    defers to the enclosing query as a correlated reference does, so SELECT (SELECT secret_col) FROM orders is refused too. An output with no readable name (SELECT count(*) with no alias) does not
    count, and the refusal says to name it with AS or to qualify the column with the source's alias.
    Alias column lists rename by position, a quoted output name keeps its case where the engine does,
    and a CTE inside a scalar subquery still sees the outer query. A key a USING join compares must
    be declared on the side of that join it comes from, and a name a NATURAL join compares from a
    CTE or subquery must be declared on the tables of that join. An output alias stands for itself
    only as a whole ORDER BY or GROUP BY item, or as a window-function alias in QUALIFY (DuckDB,
    Snowflake, BigQuery, Databricks), where engines read the alias. Anywhere else, including
    HAVING, expressions and FILTER, the engine reads the input column of that name, so SELECT secret_col AS secret_col FROM orders is refused, and so is an alias referenced in HAVING; that
    refusal says to write the expression out, and the SQL rules and server instructions now tell the
    model to. A parenthesised join is read like the FROM around it: the sources inside it name their
    columns, its ON, USING and NATURAL are checked, and its alias ((...) AS w) reaches only
    the columns its tables declare, so w.secret_col and ON secret_col = ... inside one are
    refused where they ran before.

Added

  • A table can move to another subject area, keeping its joins, metrics and entities (#513).
    The curate op {"op":"move","kind":"table","name":"<table>","area":"<from>","to_area":"<to>"}
    ("schema" too when two schemas share the name) moves the table and its area entry, and creates
    the target area when the model doesn't have it. A join that now spans two areas becomes a
    cross-area join, and a cross-area join whose two tables now share an area moves into it. Metrics
    and entities on that table move with it when every table they read is now in the new area; one
    that also reads a table outside it stays, and the result's notes names it; one left in another
    area moves too once all its tables have reached the new area by separate moves. A join that names
    its table by a bare name shared by two schemas is resolved by the area it sits in and gets the
    schema written; a move with a join on the table whose other end matches tables in two schemas is
    refused, with nothing written. A join to a table no area defines stays where it is, and notes
    says so. A join the target area already holds is kept once: a copy a person signed off over one
    the engine approved, and that over an unreviewed one, taking a description only the other copy
    has; notes says which was kept. A move is refused when two copies a person signed off disagree.
    A composite-key on: join moves intact beside another join between the same tables. The batch is
    validated and undone on failure (folders it created included, and no notes reported), logged
    with both areas, and committed when the model is its own git repository. The model explorer has a
    Move to area control on every table that queues the op.

  • A metric can carry caveats (#524). Tables and columns could, metrics could not, so a limit
    of a metric's formula ("gives no value when the start date is missing") went into its description
    or calculation, where it reads as part of the definition. A metric now takes an optional
    caveats list, validated like a column's (no empty entries) and editable through sm curate
    and the model explorer. It is served wherever the metric is: the metric's entry in
    get_datasource_schema, metric search, the admin model page, the explorer's metric card and the
    review queue. The validator reads it too: a metric caveat saying a missing value counts as 0
    silences metric_null_filled the way the calculation saying so already did.

    Reading a model: a metric file that sets caveats is refused by a release older than this
    one, which does not know the key; files with no caveats stay readable, because the key is left
    out of a metric file when the list is empty. Upgrade every server that reads the model (including
    deploy images) before writing metric caveats.

Added

  • A customer's data dictionary is applied as metadata in one command (#522). sm describe-file
    stamped every description source: ai, so a dictionary the customer wrote could only land as their
    own text through a script that built curate ops, which agami-connect forbids. It now takes
    --source metadata and reads a CSV (table, column, description, optional area / schema
    header; quoted text may hold commas and newlines) or JSON (a list of those objects, or an object
    mapping "table.column" to a description) as well as the TSV it already read, by file extension or
    --format. Each row is matched to a model column, exactly first and then ignoring case, and lands
    on the model's spelling. A SCHEMA.TABLE in the table column is read as that schema's table when
    the model has the schema, and listed under schema_prefix_stripped; a schema the model doesn't
    have is reported as such. A row that can't be applied comes back under unmatched with a reason
    (no_table, no_column, ambiguous_table, ambiguous_column, malformed, empty_description),
    where a TSV's bad lines used to be skipped without a word; a CSV row with more fields than the
    header is malformed, and broken quoting fails the whole file (exit 2) rather than merging rows. A
    folder with no semantic model is refused (exit 2). A dictionary row never replaces a description a
    person wrote; those are listed under kept_human. A metadata description is taken as written
    from the customer's dictionary; it is not checked against the data (see #564). Without the flag
    the command stamps ai as before. agami-connect applies a dictionary file this way and reports
    the unmatched rows.

  • A column's list of allowed values is the whole list, or none (#517). Introspection built each
    choice_field from the first 500 rows the engine returned, so on a table of a few thousand rows a
    status with 14 values could list 6, and the list read as complete: a question about a missing value
    was written against the list, and the reconcile probe graded a correct literal as a query defect.
    Values the catalog declares now come first, on a table of any size and with no table scan: a
    Postgres enum type, a MySQL/MariaDB ENUM(...) column, and a CHECK constraint that only lists one
    column's values (col IN ('a','b'), col = 'a' OR col = 'b'), read on Postgres, MySQL 8, SQL
    Server, Oracle, SQLite and DuckDB; a declared list is kept as declared, up to 200 values, integer
    columns included. Only a CHECK the engine enforces on every stored row counts (not Postgres NOT VALID or NO INHERIT, MySQL NOT ENFORCED, a disabled or untrusted SQL Server CHECK, or an
    Oracle one that is disabled or NOVALIDATE), a CHECK whose name another table also uses is skipped,
    and one that casts the column to a non-string type, or casts a listed number in a way that can change
    it (CAST(1.9 AS INTEGER)), declares nothing. Two columns whose names differ only by case (Postgres
    status and "STATUS") get no declared list. A declared list that can't be kept (over 200
    values, two CHECKs that disagree, or a value the column's type can't hold) gives the column no list
    at all. On SQL Server and a SQLite COLLATE NOCASE column a string list also gets a caveat that the
    database compares values case-insensitively; on MySQL/MariaDB, case- and accent-insensitively. Postgres shows a CHECK's text only to the table's owner (the function that shows
    it to any role is refused by the recon gate), so a SELECT-only role reads enums and no CHECKs. When nothing is declared, a small table (the same size rule as before) reads the
    list with one bounded SELECT DISTINCT per column, at most 1,000 in a run; more than 25 values
    gives no list rather than part of one. Keys (*_key, *_sk, *_fk), measures and years get no
    list from the data. Values are listed exactly as stored: 'Open ' is no longer trimmed into 'Open',
    and a column whose values differ only by spacing or case gets a caveat naming them; loading a model
    no longer trims a choice_field key either. A refresh leaves an existing column's list alone, so
    labels a person filled in are kept; new columns get the new lists.

    Reading a model: a release older than this one trims choice_field keys as it loads them, so it
    reads a list holding both 'Open' and 'Open ' as one 'Open'.

Fixed

  • A table is identified by its schema as well as its name (#258, #337).

    • Storage. The served model stored tables keyed by subject area and bare name, so
      sales_data.orders and staging.orders in one area landed on one key and the deploy failed.
      Migration 028_model_table_identity adds the schema to model_table's key. Rows
      already stored copy across unchanged and still load, and the next deploy writes them in full.
      One area defining the same table twice is now the validator error duplicate_table_identity,
      which includes public.orders written next to orders with schema: public.
    • dataset_names. It used to strip the schema, so public.orders could return
      archive.orders. It now returns the table in the schema it names. A name declared under two
      schemas returns both tables, each keyed schema.table. A schema the name isn't declared in is
      answered with the spellings that do resolve.
    • Routing map. On mode="index" it publishes schema.table where a name is ambiguous, and
      passing that name back returns that table.
    • Name carrying its schema. A table defined as name: public.orders can now be selected as
      either orders or public.orders, and the scope gates match it.
    • Joins. A join in joins_to_other_tables whose schema the returned table can't confirm now
      stays whole, so it no longer reads from: orders with the schema dropped.
    • Scope gates.
      • A model's tables live in the connection's own database. A reference qualified with another
        database or project is refused once the credentials say which one the connection is in. When
        the credentials don't name one, the qualifier is refused. On Databricks the connection is
        always in the workspace's default catalog, so a catalog prefix is refused there; write
        schema.table. A Snowflake database in the credentials is compared as Snowflake reads it,
        upper-cased unless quoted, so database: analytics accepts "ANALYTICS".SALES.ORDERS.
      • A four-part name (server.database.schema.table) never matches a declared table.
      • database..table is refused, saying to write the schema: it reads the connecting user's
        default schema, which on SQL Server can be dbo.orders beside a declared sales.orders.
      • On Postgres, Snowflake and Oracle, a quoted name has to be the declared spelling exactly, so
        "Orders" no longer passes for orders, nor "ORDERS" for a declared orders on Snowflake.
        Unquoted names still match without regard to case there. Where case alone tells two declared
        tables apart, column scope reads the columns of the one the statement names.
      • A dot inside a table name is part of the name when the model gives the schema separately, so
        Postgres public."orders.v2" is one table.
      • On MySQL a table or database name has to match the declared case exactly, quoted or not:
        on Linux, its default, sales.Orders and sales.orders are two different tables.
      • The other engines match names without regard to case, as they resolve them.
  • A join on a unique key is labelled one-to-one, so a metric over it no longer carries a false
    fan-trap or chasm-trap finding (#511).
    Introspection labelled a join from the primary keys
    alone, so a join onto a UNIQUE column, such as passports.person_id → users.id, came out
    many-to-one. Unique keys are now read from the catalog on PostgreSQL (unique constraints and
    unique indexes), MySQL, SQLite, SQL Server, Oracle and DuckDB, and kept on the table as
    unique_keys. Snowflake and Redshift declare unique keys without enforcing them, so one is kept
    only when the data bears it out; BigQuery, Databricks and Trino have none. A join is one-to-one
    when its columns are the whole of a key on both sides, and the same rule now labels a join a
    query writes that the model doesn't declare, a join to a grouped CTE, and the references
    enrichment adds. When no key settles a join but its many side turns out unique in the data, the
    label is kept and the join waits for review with a note saying so, since data that is unique
    today is not a promise. A refresh relabels each join introspection approved itself from the
    database's keys and lists it under joins corrected from the database's constraints; a join a
    person signed off is never changed, and a person's edit of a join's cardinality, columns, on or
    tables in the explorer makes the sign-off theirs (with no signer recorded, the engine's sign-off
    is cleared and the join waits for review), so a refresh leaves the edit alone. Only enabled,
    valid keys count: an invalid PostgreSQL index, a disabled SQL Server index and a disabled or
    unvalidated Oracle constraint are not read, a SQLite unique index over an expression is not
    shortened to its plain columns, and a SQLite unique index written with its own COLLATE is not
    trusted. Labelling reads each table's own keys, so two tables of one name in two schemas no longer
    lend each other a key, at introspection or at query time. At query time a join the model doesn't
    declare counts as one-to-one by its keys only when its ON is on its own table, that table is read
    once in the query, and every other join beside it is one-to-one too; otherwise its numbers stay
    undetermined. A refresh that can't read a table's unique keys (a failed catalog read or data
    check) keeps the stored ones and leaves that table's joins alone, with a note
    unique keys not re-read for <table>.

    Reading a model: a table that sets unique_keys is refused by a release older than this
    one, which does not know the key. Upgrade every server that reads the model (including deploy
    images) before re-introspecting with this release. A model with no unique keys stays readable by
    older releases. CI that validates the model with a newer release won't catch this.

Fixed

Fixed

  • sm suggest-metrics prints its proposals before writing, and its bindings run and mean what
    they say (#514).
    It used to write straight into the model, and every proposal kept from it on a
    large onboarding had to be fixed by hand. It is now a dry run by default: it prints each metric's
    binding, calculation, source table and flags, and --write saves them (--names keeps only some,
    or a flagged one a person chose). --profile <name> test-runs every binding first, one bounded
    read per table (opt-in, as a pay-per-scan warehouse bills it): a binding that errors is never
    written, and a constant, all-zero or NULL result is flagged degenerate and left out unless named.
    Proposals on a table under 50 rows are flagged low_confidence (from the test run's own row
    count when there is one). A table defined by SQL is test-run over that SQL; when that read fails
    its proposals are flagged not_test_run and each proposal says whether it was test-run. The SQL itself: a mixed-case
    column is quoted where the engine folds case (PostgreSQL, Redshift, Snowflake, Oracle), and a
    column named after a word the engine reserves (index on SQLite, key on MySQL) is quoted; a pair of
    DATE columns subtracts to days on PostgreSQL and DuckDB, where EXTRACT(EPOCH ...) of the integer
    failed, and uses DATE_DIFF on BigQuery; BigQuery and Oracle timestamp pairs no longer fail; a
    flag rate leaves NULL rows out instead of counting them false, and a T-SQL BIT flag is compared
    rather than used as a predicate. A duration leaves out rows where the end is before the start and
    its calculation says so (compared directly on engines whose day difference rounds a reversed
    same-day pair to 0), pairs a planned start only with a planned end (and actual with actual),
    uses an audit column such as sys_created_on only when nothing else fits, and offers an ambiguous
    pairing flagged rather than picking one (its reason names up to three of the others). A storage type with no SQL dialect now fails clearly
    instead of being written as PostgreSQL.

  • A curate op on a join, entity or metric that doesn't name an area is resolved, or fails the batch (#538).
    An op on a join between two subject areas was skipped unless it named one side's area, so the batch
    reported success with that change missing. A metric or entity defined across areas could not be
    curated at all, so sm approve-queue skipped every cross-area metric. An op with no area is now
    searched for in every area's files and in the cross-area joins, entities and metrics. Exactly one
    match is applied; an entity or metric whose declared name equals the op's name exactly wins over
    others that only share its file slug (revenue ($) and revenue (%)). No match, or more than one (two joins between the same tables), fails the whole
    batch with an error naming the op and the candidates, and nothing in the batch is written;
    from_column / to_column (or area) picks one. A join op whose named area doesn't hold the join
    also finds it in another area. An op that names an area behaves as before otherwise: a missing
    target is reported under skipped and the rest of the batch applies.

Fixed

  • A second-order metric can sit on a base metric that spans several tables, and its table names
    are quoted for the engine (#525).
    OUTERAGG({base}) with inner_grain used to be refused when
    the base metric read more than one table, and the synthesized statement wrote table and column
    names bare, so a mixed-case table failed on PostgreSQL. The synthesized inner query now joins the
    table the base metric measures to each other source table by the one relationship declared
    directly between them. That table is the base's primary_table (matched to its source table
    with or without the schema), else the one table that declares every column the base's
    aggregates read; a primary_table that is not a source table, or a base whose measured columns
    span tables or can't be placed, is refused and asks for primary_table. It is also refused,
    with the reason, when there is no such edge, more than one, or the join would fan out the base
    aggregate's rows. A bare inner_grain column must be declared by exactly one of the tables; a
    table.column entry (orders.order date, line items.id) must name a source table and a
    column it declares. Identifiers are quoted only where the engine needs it (including words one
    engine reserves, such as index on SQLite or key on MySQL), so a plain lower-case name is
    emitted as before; a reserved lower-case name on Snowflake or Oracle is quoted in upper case
    ("LIMIT"), the name those engines store it under.
    Tables with the same name from different schemas in one base are refused. An inner_grain entry
    naming a declared column (order date) is that column, quoted; the base metric's own subject
    area is read first for the table's columns.

Fixed

  • The receipt names the table behind a subquery or CTE alias, and reads a FILTER clause (#521).
    COUNT(*) FILTER (WHERE s.x > 0) over a subquery s listed the reference as s.x, and the
    aggregate said it names no column. A column qualified by a derived table or a CTE is now followed
    through a projection that passes one column through unchanged (bare or renamed) to its table, so
    the reference reads public.t.x. A computed projection (x / id AS r) keeps the alias it was
    written with. A qualifier is resolved in the query it was written in, so an alias reused in a
    nested query no longer borrows the other query's table. An aggregate's FILTER (WHERE …) columns
    count as columns it reads, the same as a CASE inside it, so a single-table COUNT(*) FILTER (…)
    is resolved and one over a join can report a fan. A CTE name binds as the scope gate binds it (a
    schema-qualified table, a later sibling or a non-recursive self-reference is the table), and a
    column list (WITH s(y, x), AS s(y, x)) renames outputs by position. A subquery's columns are
    not credited to the outer query's aliases: a FILTER holding a subquery, an aggregate holding
    one and an aggregate computed inside a nested SELECT report undetermined (a fan proven on the
    outer columns still reports multiplied). A body with a star keeps the written alias.
    What the scope gate refuses is unchanged, and a value read from a subquery is still not matched to
    a metric.

Added

  • The model explorer can reject in bulk, beside every bulk approve (#531). Each "✓ Approve all N
    proposed" and "✓ Approve N" now has a "✕ Reject all N proposed" / "✕ Reject N" next to it. It acts
    on exactly the items shown (the sub-tab, kind group, status filter and search), skipping anything
    already approved or rejected, confirms with the count and kind ("Reject 104 metrics?") and asks for
    one reason, which is required. Each queued reject op carries that reason, which reaches
    curation_log.jsonl with the op; the model itself does not store it. Like approvals, the rejects
    are queued and applied only when the feedback block is submitted. A single reject still carries no
    reason.

Fixed

  • "Approve all" on the Metrics tab names the kind on each op (#531). The Metrics tab's bulk
    approve queued ops with no kind, so the Queued tab showed "approve undefined" and the curation
    engine skipped them as "op needs kind + name". They now say metric, as the Review tab's do.

Fixed

  • The interpreter resolver no longer picks a Python that can never be brought up to date (#529).
    connect_resolve.py checks each candidate for pip and for the agami-core version it carries, read
    from installed metadata. A candidate with no pip and an agami-core that is missing or older than the
    plugin is skipped, with a reason in candidates_scored; a pip-less one whose library is current is
    still chosen, and $AGAMI_PYTHON is never skipped but gets a warning. The output says
    chosen_because. sm and py apply the same rule to the Pythons on PATH, so an activated
    pip-less venv no longer shadows a working one; a configured interpreter that fails it gives way to
    a PATH one that passes, and is kept (so sm can say what is wrong) when none does. When the library a Python runs is older than the plugin, connect_resolve
    (as a library_older_than_plugin anomaly), py and the scripts' library loader warn on stderr,
    naming both versions, the interpreter and the fix (sm install, or AGAMI_PYTHON). sm says
    " has no pip" instead of a bare "couldn't install". The plugin version is read from
    plugin.json by one shared helper, with the cache folder's name as the fallback.

Fixed

  • Validator: fewer false positives, entity mappings checked everywhere, collision messages name
    the table (#520).
    metric_binding_unknown_column read the aliases of a binding's own subquery
    as missing columns: SELECT AVG(s.r) FROM (SELECT …, SUM(pts) / SUM(maxpts) AS r …) s warned
    about r. It now leaves out a column a derived table or CTE the SELECT reads projects (through
    a SELECT * too, when the star is over another derived table) and a select-list alias used in
    the same SELECT's HAVING or ORDER BY; a typo inside the subquery, one read through a star over a
    base table, and one an unused CTE happens to project still warn. The loader's rule
    for hiding a metric that reads an excluded or stale column is unchanged. An entity mapping to a
    table outside its own area was never checked, nor were cross_subject_area_entities; both now
    resolve against the whole model, with new warnings entity_mapping_unknown_table (no area
    defines the table) and entity_mapping_column_not_found (the table has no such column). They are
    warnings so a model that validated before still writes. cross_area_entity_collision printed
    ?.?.<table> for a table the area only listed, and so warned when two areas mapped the same
    table; it now resolves the table across the model (on the connection the area lists it on, when
    two connections define it), compares the resolved table, and prints
    schema.table with each area's name (the connection too when there is more than one).

Fixed

  • A subject area named where a table was wanted also names that area's tables like it (#567).
    get_datasource_schema(dataset_names=["orders"]) answered "'orders' is a subject area, not a
    table: pass it as area" and nothing else, so in a model whose tables carry a suffix (orders_v)
    the agent spent a second call finding the table. The hint now ends "or did you mean 'orders_v'?"
    with the same names in did_you_mean, matched against that area's own tables only, so a
    like-named table elsewhere is never guessed and never crowds the area's own out. The activity
    log's missed entry for a name answered with any wrong-kind hint (an area or a column) now
    carries hinted: true, so an empty suggestion list no longer reads as "the caller was given
    nothing".

  • The forked transport no longer relays a child's whole stderr on an other failure (#583).
    Every sanitized failure is rebuilt by the parent from the exit code, because stderr is shared with
    anything a library logs during the run. other (exit 6) had no fixed sentence, so it fell through
    to the relay meant for the authored configuration errors (exit 2 and 3), and a library's line,
    values or paths included, reached failure.message. It now gets the generic sentence, or a
    SQLSTATE sentence (#561) relayed only as an exact match; a stream holding neither goes to the
    server log.

  • A datasource.yaml key this agami-core does not read is reported, not silently dropped (#575).
    The loader reads a fixed list of top-level keys, so a model written for a newer agami-core that
    declared a field the running one did not know lost it without a word, and a deploy fed by that
    load stored the model without it. The model still loads and the field is still ignored, but sm validate lists the keys as an unknown_datasource_key warning and the deploy prints them, saying
    the model may need a newer agami-core. The load itself stays silent, since the executor loads the
    model in a child process whose stderr is its output.

  • On Redshift, discovery finds the tables of a read-only user (#558). sm discover, and an
    introspection run without a table list, listed schemas from information_schema.schemata, which on
    Redshift shows only the schemas the user owns. A read-only role owns none, so discovery reported
    "no tables discovered" for a catalog it could read in full; passing --tables worked, which hid
    it. Redshift now lists the schemas the user has USAGE on (pg_namespace with
    has_schema_privilege), leaving out the system schemas. Other engines are unchanged.