Repository navigation
0.12.0
Ossie schema 0.2.0.dev0. Tested with dactopus-ossie-clickhouse 0.13.0
(the canon and the blind questions) and 0.12.0 (CI), dbt v2 (dbt-oss
2.0.11) and dbt v1 (dbt-core 1.11 with dbt-clickhouse 1.10) on
ClickHouse 26.9, the WooCommerce package also on 26.7. Needs
dbt build --full-refresh once.
order_lines, line_categories, line_brands and refunds of both
packages and of the commerce union gain a column: an incremental run
stops on the changed schema, and a deployment runs
dbt build --full-refresh once. The same full refresh records the
settings the new check below needs. No input table changes. Metric names
of the commerce model change.
Changed
-
orders,order_linesandrefundsof the Shopify and WooCommerce
packages built by an earlier release record no settings, so their
first incremental run stops with the message of the check below. The
full refresh this release needs for the new columns builds them anew
and records the settings, so onedbt build --full-refreshcovers
both. -
The dbt user needs
ALTER MODIFY COMMENTonorders,order_lines
andrefundsof both packages. The WooCommerce package needed it on
refundsalready. -
The 18 commerce metrics that read order lines, their categories and
brands, or refunds together withorders.in_sales_reportsare metrics
of those datasets, as the Ossie spec requires: it rejects an aggregate
over fields of two datasets, and its relational layer does not expose
the metrics of the model itself. A question names them
dataset.metric, and result columns are named so too. This breaks
saved questions, dashboards and agent prompts that ask for the old
names; the numbers are unchanged:Old New items_sold,items_net,line_revenue,items_returned,line_returned_amount,line_net_revenuethe same name on order_lines:order_lines.line_revenuecategory_items_sold,category_revenue,category_items_returned,category_returned_amount,category_net_revenuethe same name on line_categories:line_categories.category_revenuebrand_items_sold,brand_revenue,brand_items_returned,brand_returned_amount,brand_net_revenuethe same name on line_brands:line_brands.brand_revenuerefund_count,refunds_totalthe same name on refunds:refunds.refunds_totalThe model has no metrics of its own any more.
-
order_lines,line_categories,line_brandsandrefundscarry
in_sales_reports(Bool), the flag of the line's or refund's order
(orders.in_sales_reports), so that the metrics above count the rows
of one dataset. At Shopify it is true. At WooCommerce the packages
compute it from the order's status, time placed and payment and
completion dates, by the formula ofordersthat now sits in the
macroswoocommerce_report_timeandwoocommerce_in_sales_reports.
The commerce union lists the column too. -
orders.refund_rate,orders.average_order_value,
orders.items_per_orderandorders.repeat_order_sharedivide by
NULLIF(..., 0): a question that matches no counted order gets NULL,
where it divided by zero. -
tests/check_contracts.pyfails on a metric that reads fields of more
than one dataset.AGENTS.mdandCONTRIBUTING.mdstate the rule,
anddocs/commerce.mdlists the metrics by dataset. -
The model instructions said every metric but
orders.all_ordersand
orders.all_orders_totalcounts only the orders the store's sales
reports count. The metrics ofcustomerscount the orders the store's
customers report counts, which at WooCommerce includes an order on hold
that is not paid yet. The
instructions and the descriptions ofcustomerssay so;
CONTRIBUTING.mdandAGENTS.mdstate the rule: a dataset built from
orders says which orders it counts, in the model's terms, and then
names the store report that counts the same.
Added
- An incremental run of the Shopify and WooCommerce packages stops, before
it reads or deletes a row oforders,order_linesorrefunds, when
the variables that decide the values of the rows already built differ
from the ones the tables were built with.
Until now a changed variable rebuilt only the orders the run reread and
left the others as they were, a mixture that no test caught (the
one-label tests caught a new label, after the run had rewritten rows).
The message names the table, each differing variable with its old and
new value, and asks fordbt build --full-refreshonce. Guarded at
Shopify:shopify_timezone,shopify_source_id. At WooCommerce:
woocommerce_timezone,woocommerce_date_type,
woocommerce_excluded_statuses,woocommerce_order_statuses,
woocommerce_currency,woocommerce_price_decimals,
woocommerce_order_number_meta_key,woocommerce_source_id. The input
database, the table prefix and the lookback days are not guarded. - The values are kept in the comment of
orders,order_linesand
refunds, asbuilt with {...}(JSON, keys in alphabetical order),
written after a first build or a full refresh. At WooCommerce the
comment ofrefundsalso holds its sync position, and each macro
that writes the comment keeps the segments it does not own
(woocommerce_set_comment). The README of each package has a section,
Changing a setting. - Tests
order_lines_in_sales_reports_match_ordersand
refunds_in_sales_reports_match_ordersin both packages: no line or
refund has a flag different from its order's. At WooCommerce they warn
untilDACTOPUS_STRICT_TESTS=error, as the package's other tests that
compare two tables. line_categories.category_tree_items_sold,category_tree_revenue,
category_tree_items_returned,category_tree_returned_amountand
category_tree_net_revenue, and
product_categories.category_tree_product_count: the category metrics
over every row of the line or product, the categories above its own
included. Filtered by one category (category_path), they count the
category with its subcategories, each line or product once. The
descriptions advisedorder_lines.line_revenueor
products.product_countfiltered byline_categories.category_path
orproduct_categories.category_pathfor that: a filter on the many
side of a relationship, which dactopus-ossie-clickhouse answers and
the Ossie spec's relational and wide-table layers do not express. The
advice now names the new metrics;orders.average_order_value
filtered by a category keeps the old form, which has no other. The
tables do not change.