Skip to content

0.12.0

Choose a tag to compare

@ekut ekut released this 04 Nov 11:53
· 41 commits to main since this release
3fb65f6

Ossie schema 0.2.0.dev0. Tested with dactopus-ossie-clickhouse 0.13.0
(the canon and the blind questions) and 0.12.0 (CI), dbt v2 (dbt-oss
2.0.11) and dbt v1 (dbt-core 1.11 with dbt-clickhouse 1.10) on
ClickHouse 26.9, the WooCommerce package also on 26.7. Needs
dbt build --full-refresh once.

order_lines, line_categories, line_brands and refunds of both
packages and of the commerce union gain a column: an incremental run
stops on the changed schema, and a deployment runs
dbt build --full-refresh once. The same full refresh records the
settings the new check below needs. No input table changes. Metric names
of the commerce model change.

Changed

  • orders, order_lines and refunds of the Shopify and WooCommerce
    packages built by an earlier release record no settings, so their
    first incremental run stops with the message of the check below. The
    full refresh this release needs for the new columns builds them anew
    and records the settings, so one dbt build --full-refresh covers
    both.

  • The dbt user needs ALTER MODIFY COMMENT on orders, order_lines
    and refunds of both packages. The WooCommerce package needed it on
    refunds already.

  • The 18 commerce metrics that read order lines, their categories and
    brands, or refunds together with orders.in_sales_reports are metrics
    of those datasets, as the Ossie spec requires: it rejects an aggregate
    over fields of two datasets, and its relational layer does not expose
    the metrics of the model itself. A question names them
    dataset.metric, and result columns are named so too. This breaks
    saved questions, dashboards and agent prompts that ask for the old
    names; the numbers are unchanged:

    Old New
    items_sold, items_net, line_revenue, items_returned, line_returned_amount, line_net_revenue the same name on order_lines: order_lines.line_revenue
    category_items_sold, category_revenue, category_items_returned, category_returned_amount, category_net_revenue the same name on line_categories: line_categories.category_revenue
    brand_items_sold, brand_revenue, brand_items_returned, brand_returned_amount, brand_net_revenue the same name on line_brands: line_brands.brand_revenue
    refund_count, refunds_total the same name on refunds: refunds.refunds_total

    The model has no metrics of its own any more.

  • order_lines, line_categories, line_brands and refunds carry
    in_sales_reports (Bool), the flag of the line's or refund's order
    (orders.in_sales_reports), so that the metrics above count the rows
    of one dataset. At Shopify it is true. At WooCommerce the packages
    compute it from the order's status, time placed and payment and
    completion dates, by the formula of orders that now sits in the
    macros woocommerce_report_time and woocommerce_in_sales_reports.
    The commerce union lists the column too.

  • orders.refund_rate, orders.average_order_value,
    orders.items_per_order and orders.repeat_order_share divide by
    NULLIF(..., 0): a question that matches no counted order gets NULL,
    where it divided by zero.

  • tests/check_contracts.py fails on a metric that reads fields of more
    than one dataset. AGENTS.md and CONTRIBUTING.md state the rule,
    and docs/commerce.md lists the metrics by dataset.

  • The model instructions said every metric but orders.all_orders and
    orders.all_orders_total counts only the orders the store's sales
    reports count. The metrics of customers count the orders the store's
    customers report counts, which at WooCommerce includes an order on hold
    that is not paid yet. The
    instructions and the descriptions of customers say so;
    CONTRIBUTING.md and AGENTS.md state the rule: a dataset built from
    orders says which orders it counts, in the model's terms, and then
    names the store report that counts the same.

Added

  • An incremental run of the Shopify and WooCommerce packages stops, before
    it reads or deletes a row of orders, order_lines or refunds, when
    the variables that decide the values of the rows already built differ
    from the ones the tables were built with.
    Until now a changed variable rebuilt only the orders the run reread and
    left the others as they were, a mixture that no test caught (the
    one-label tests caught a new label, after the run had rewritten rows).
    The message names the table, each differing variable with its old and
    new value, and asks for dbt build --full-refresh once. Guarded at
    Shopify: shopify_timezone, shopify_source_id. At WooCommerce:
    woocommerce_timezone, woocommerce_date_type,
    woocommerce_excluded_statuses, woocommerce_order_statuses,
    woocommerce_currency, woocommerce_price_decimals,
    woocommerce_order_number_meta_key, woocommerce_source_id. The input
    database, the table prefix and the lookback days are not guarded.
  • The values are kept in the comment of orders, order_lines and
    refunds, as built with {...} (JSON, keys in alphabetical order),
    written after a first build or a full refresh. At WooCommerce the
    comment of refunds also holds its sync position, and each macro
    that writes the comment keeps the segments it does not own
    (woocommerce_set_comment). The README of each package has a section,
    Changing a setting.
  • Tests order_lines_in_sales_reports_match_orders and
    refunds_in_sales_reports_match_orders in both packages: no line or
    refund has a flag different from its order's. At WooCommerce they warn
    until DACTOPUS_STRICT_TESTS=error, as the package's other tests that
    compare two tables.
  • line_categories.category_tree_items_sold, category_tree_revenue,
    category_tree_items_returned, category_tree_returned_amount and
    category_tree_net_revenue, and
    product_categories.category_tree_product_count: the category metrics
    over every row of the line or product, the categories above its own
    included. Filtered by one category (category_path), they count the
    category with its subcategories, each line or product once. The
    descriptions advised order_lines.line_revenue or
    products.product_count filtered by line_categories.category_path
    or product_categories.category_path for that: a filter on the many
    side of a relationship, which dactopus-ossie-clickhouse answers and
    the Ossie spec's relational and wide-table layers do not express. The
    advice now names the new metrics; orders.average_order_value
    filtered by a category keeps the old form, which has no other. The
    tables do not change.