Skip to content

Releases

0.12.2

Choose a tag to compare

@ekut ekut released this 09 Nov 02:24
e23aad8

Ossie schema 0.2.0.dev0. Tested with dactopus-ossie-clickhouse 0.14.0,
dbt v2 (dbt-oss 2.0.11) and dbt v1 (dbt-core 1.11 with
dbt-clickhouse 1.10) on ClickHouse 26.9, the WooCommerce package also on
26.7.

The GA4 and Sensor.Community packages need dbt build --full-refresh
once: their incremental tables record their label from this release on,
and a table built by an earlier one records none, so its first
incremental run stops and asks for it. The Shopify and WooCommerce
packages need none: their tables keep the record 0.12.0 wrote.

Changed

  • One settings guard for every package: macros/settings.sql, the same
    file in each, run on all models from the package's dbt_project.yml
    (+pre-hook, +post-hook), ahead of a model's own hooks on dbt v1 and
    v2; it does nothing on a model that is not incremental. The variables
    it guards are meta.guarded_vars there, with their defaults, and the
    variables any run may change are meta.free_vars. It replaces the
    Shopify macros shopify_settings, shopify_check_settings and
    shopify_save_settings and the WooCommerce macros
    woocommerce_settings, woocommerce_check_settings,
    woocommerce_save_settings and woocommerce_check_built. The record
    and the message stay as they were.
  • The incremental tables of the GA4 package (events, sessions,
    purchases) and of the Sensor.Community package (readings) record
    ga4_source_id and sensor_community_source_id: a run with another
    label stops before it reads or deletes a row, where the one-label tests
    caught it after the run had rewritten rows. Their dbt user needs
    ALTER MODIFY COMMENT on these tables. Their READMEs have a section,
    Changing a setting.
  • A variable a later release starts to guard is read as its default from
    a table whose record lacks it: only a deployment that set another value
    builds anew.
  • The phrase of the message, was not built with the settings of this run, is what a tool that deploys the packages can tell the stop by
    (docs/deployment.md); a change of it will be in this changelog.

Added

  • tests/check_contracts.py fails on a package with an incremental model
    or a time zone that does not run the guard from its dbt_project.yml,
    on a copy of macros/settings.sql that differs, on a variable of
    vars in neither meta.guarded_vars nor meta.free_vars, on a
    default there other than the one in vars, and on a time_zone or
    source_id variable that is not guarded.
  • tests/check_settings.py runs a package's build with another value of
    each guarded variable and checks that the run stops with the message,
    names that variable alone and leaves the tables as they were; that a
    record without a variable reads it as its default; and that a full
    refresh records the new values. The CI job of every package runs it in
    place of steps of its own.

0.12.1

Choose a tag to compare

@ekut ekut released this 07 Nov 05:40
a95319a

Ossie schema 0.2.0.dev0. Tested with dactopus-ossie-clickhouse 0.14.0,
dbt v2 (dbt-oss 2.0.11) and dbt v1 (dbt-core 1.11 with
dbt-clickhouse 1.10) on ClickHouse 26.9, the WooCommerce package also on
26.7.

Changed

  • CI installs dactopus-ossie-clickhouse 0.14.0 from PyPI instead of 0.12.0 from
    git.
  • The commerce model's instructions no longer tell an agent that a store
    missing from a result has a zero for a ratio or an average, such as the
    average order value or the refund rate: it has a zero count or sum there
    and no ratio or average.

0.12.0

Choose a tag to compare

@ekut ekut released this 04 Nov 11:53
3fb65f6

Ossie schema 0.2.0.dev0. Tested with dactopus-ossie-clickhouse 0.13.0
(the canon and the blind questions) and 0.12.0 (CI), dbt v2 (dbt-oss
2.0.11) and dbt v1 (dbt-core 1.11 with dbt-clickhouse 1.10) on
ClickHouse 26.9, the WooCommerce package also on 26.7. Needs
dbt build --full-refresh once.

order_lines, line_categories, line_brands and refunds of both
packages and of the commerce union gain a column: an incremental run
stops on the changed schema, and a deployment runs
dbt build --full-refresh once. The same full refresh records the
settings the new check below needs. No input table changes. Metric names
of the commerce model change.

Changed

  • orders, order_lines and refunds of the Shopify and WooCommerce
    packages built by an earlier release record no settings, so their
    first incremental run stops with the message of the check below. The
    full refresh this release needs for the new columns builds them anew
    and records the settings, so one dbt build --full-refresh covers
    both.

  • The dbt user needs ALTER MODIFY COMMENT on orders, order_lines
    and refunds of both packages. The WooCommerce package needed it on
    refunds already.

  • The 18 commerce metrics that read order lines, their categories and
    brands, or refunds together with orders.in_sales_reports are metrics
    of those datasets, as the Ossie spec requires: it rejects an aggregate
    over fields of two datasets, and its relational layer does not expose
    the metrics of the model itself. A question names them
    dataset.metric, and result columns are named so too. This breaks
    saved questions, dashboards and agent prompts that ask for the old
    names; the numbers are unchanged:

    Old New
    items_sold, items_net, line_revenue, items_returned, line_returned_amount, line_net_revenue the same name on order_lines: order_lines.line_revenue
    category_items_sold, category_revenue, category_items_returned, category_returned_amount, category_net_revenue the same name on line_categories: line_categories.category_revenue
    brand_items_sold, brand_revenue, brand_items_returned, brand_returned_amount, brand_net_revenue the same name on line_brands: line_brands.brand_revenue
    refund_count, refunds_total the same name on refunds: refunds.refunds_total

    The model has no metrics of its own any more.

  • order_lines, line_categories, line_brands and refunds carry
    in_sales_reports (Bool), the flag of the line's or refund's order
    (orders.in_sales_reports), so that the metrics above count the rows
    of one dataset. At Shopify it is true. At WooCommerce the packages
    compute it from the order's status, time placed and payment and
    completion dates, by the formula of orders that now sits in the
    macros woocommerce_report_time and woocommerce_in_sales_reports.
    The commerce union lists the column too.

  • orders.refund_rate, orders.average_order_value,
    orders.items_per_order and orders.repeat_order_share divide by
    NULLIF(..., 0): a question that matches no counted order gets NULL,
    where it divided by zero.

  • tests/check_contracts.py fails on a metric that reads fields of more
    than one dataset. AGENTS.md and CONTRIBUTING.md state the rule,
    and docs/commerce.md lists the metrics by dataset.

  • The model instructions said every metric but orders.all_orders and
    orders.all_orders_total counts only the orders the store's sales
    reports count. The metrics of customers count the orders the store's
    customers report counts, which at WooCommerce includes an order on hold
    that is not paid yet. The
    instructions and the descriptions of customers say so;
    CONTRIBUTING.md and AGENTS.md state the rule: a dataset built from
    orders says which orders it counts, in the model's terms, and then
    names the store report that counts the same.

Added

  • An incremental run of the Shopify and WooCommerce packages stops, before
    it reads or deletes a row of orders, order_lines or refunds, when
    the variables that decide the values of the rows already built differ
    from the ones the tables were built with.
    Until now a changed variable rebuilt only the orders the run reread and
    left the others as they were, a mixture that no test caught (the
    one-label tests caught a new label, after the run had rewritten rows).
    The message names the table, each differing variable with its old and
    new value, and asks for dbt build --full-refresh once. Guarded at
    Shopify: shopify_timezone, shopify_source_id. At WooCommerce:
    woocommerce_timezone, woocommerce_date_type,
    woocommerce_excluded_statuses, woocommerce_order_statuses,
    woocommerce_currency, woocommerce_price_decimals,
    woocommerce_order_number_meta_key, woocommerce_source_id. The input
    database, the table prefix and the lookback days are not guarded.
  • The values are kept in the comment of orders, order_lines and
    refunds, as built with {...} (JSON, keys in alphabetical order),
    written after a first build or a full refresh. At WooCommerce the
    comment of refunds also holds its sync position, and each macro
    that writes the comment keeps the segments it does not own
    (woocommerce_set_comment). The README of each package has a section,
    Changing a setting.
  • Tests order_lines_in_sales_reports_match_orders and
    refunds_in_sales_reports_match_orders in both packages: no line or
    refund has a flag different from its order's. At WooCommerce they warn
    until DACTOPUS_STRICT_TESTS=error, as the package's other tests that
    compare two tables.
  • line_categories.category_tree_items_sold, category_tree_revenue,
    category_tree_items_returned, category_tree_returned_amount and
    category_tree_net_revenue, and
    product_categories.category_tree_product_count: the category metrics
    over every row of the line or product, the categories above its own
    included. Filtered by one category (category_path), they count the
    category with its subcategories, each line or product once. The
    descriptions advised order_lines.line_revenue or
    products.product_count filtered by line_categories.category_path
    or product_categories.category_path for that: a filter on the many
    side of a relationship, which dactopus-ossie-clickhouse answers and
    the Ossie spec's relational and wide-table layers do not express. The
    advice now names the new metrics; orders.average_order_value
    filtered by a category keeps the old form, which has no other. The
    tables do not change.

0.11.2

Choose a tag to compare

@ekut ekut released this 04 Nov 05:55
70520e2

Ossie schema 0.2.0.dev0. Tested with dactopus-ossie-clickhouse 0.13.0,
the other versions as in 0.11.0.

Changed

  • The instructions of the three models keep only the rules across their
    datasets and fit in 2,300 characters (commerce had 8,367,
    web_analytics 3,683, iot 2,870). dactopus-ossie-clickhouse puts
    them into its MCP server's instructions, which Claude Code cut at
    4,096 characters: an agent saw a quarter of the commerce rules there.
    A rule about one dataset, field or metric moved to its description, and
    one said twice is said once. The tables do not change and no rebuild
    is needed. tests/check_contracts.py fails on instructions over 2,300
    characters.

0.11.1

Choose a tag to compare

@ekut ekut released this 04 Nov 02:55
ea63b4f

Ossie schema 0.2.0.dev0. Tested with the same versions as 0.11.0.

Changed

  • The descriptions of the three models name the source packages without
    repository paths (sources/shopify and the like). The tables do not
    change and no rebuild is needed.
  • CI leaves a change of text alone out of the package and union jobs: a
    dbt_project.yml whose only changed line is its version:, an entity
    whose description and ai_context alone changed
    (tests/reach.py). The validate job still runs on any change to an
    entity. A release's version lines no longer run every package.

0.11.0

Choose a tag to compare

@ekut ekut released this 04 Nov 00:54
99bc53e

Ossie schema 0.2.0.dev0. Tested with dactopus-ossie-clickhouse 0.12.0,
dbt v2 (dbt-oss 2.0.11) and dbt v1 (dbt-core 1.11 with
dbt-clickhouse 1.10) on ClickHouse 26.9, the WooCommerce package also
on 26.7. The models write expressions in the sql form, which
dactopus-ossie-clickhouse reads since 0.11.0: an older version, or an
Ossie tool that reads only dialects, does not load them. No table
changes and no rebuild is needed.

Changed

  • The models of the three domains write expressions as
    expression: {sql: ...} instead of
    expression: {dialects: [{dialect: ANSI_SQL, ...}]}, the form of the
    apache/ossie examples and spec since apache/ossie#640. Reading it
    needs dactopus-ossie-clickhouse 0.11.0 or newer. The numbers of the
    models do not change and no rebuild is needed.

0.10.0

Choose a tag to compare

@ekut ekut released this 03 Nov 04:03
e9e129b

Ossie schema 0.2.0.dev0. Tested with dactopus-ossie-clickhouse 0.12.0,
dbt v2 (dbt-oss 2.0.8) and dbt v1 (dbt-core 1.11 with
dbt-clickhouse 1.10) on ClickHouse 26.9, the WooCommerce package also
on 26.7. The unions' views read the tables they add up with the rights
of the user who built them, so a user that reads a model needs SELECT
on the union's database alone; where row policies on those tables hide
rows from the reader, the numbers through a union change.
deployment.yml names the zone of the days of a package that fixes it.
No table changes and no rebuild is needed: a union changes on its next
build.

Added

  • zone in deployment.yml: the zone of the days of a package that
    fixes it in its model, so a deployment no longer has to be told it.
    The Sensor.Community package declares zone: UTC; Shopify and
    WooCommerce take the zone from the variable with the role time_zone,
    and GA4's days are in the property's zone; none of the three has the
    key. zone and time_zone exclude each other, and a union's file has
    no zone.
    tests/check_deployment.py requires an IANA name and counts in every
    daily table the rows whose date is not the day of their time in
    zone, with or without --time-zone, and fails on any such row or on
    a --time-zone that differs
    (#60).

  • tests/<source>/canon/ and canon_windows/ for Shopify and
    WooCommerce: the canonical tables of entities/commerce.yaml (the
    columns the model reads, in the order of the contracts) over
    fixture.sql, and over fixture.sql with windows.sql, one CSV file
    a table, with tests/commerce/tables.sql, their DDL in standard SQL.
    The questions and their expected rows are in
    tests/<source>/questions.yaml. tests/run_canon.py loads the files
    into ClickHouse and asks the questions through dactopus-ossie-clickhouse,
    so the model's numbers can be checked without dbt or the platform's raw
    rows; --engine duckdb asks them through DuckDB's community extension
    ossie instead. Its 0.3.0 does not read the metrics of a dataset
    (apache/ossie#591), so the model does not load there as it is:
    tests/run_canon.py loads a copy with them moved to the model's metrics. With that
    copy, 72 of the 95 questions give the same
    numbers, 19 are refused, and 4 differ: it leaves out the rows a
    many-to-one relationship finds no match for (by products.status, the
    lines of products the products table does not have; by
    customers.first_order_month, the guests' orders). tests/export_canon.py writes
    the files from a built package, and CI fails when a build differs from
    them.

Changed

  • The distinct counts that pair source_id with the counted column
    (orders.buyer_count, product_categories.product_count,
    product_brands.product_count, sessions.users,
    sessions.user_conversion_rate, readings.reporting_devices) join the
    two: COUNT(DISTINCT source_id || '/' || CAST(customer_id AS VARCHAR)).
    COUNT(DISTINCT source_id, customer_id) is not standard SQL, and DuckDB
    rejects it. The numbers are unchanged.
  • The questions of tests/shopify/check_numbers.py and
    tests/woocommerce/check_numbers.py moved to questions.yaml, with
    their derivations as comments; the two scripts read it and need
    pyyaml (uv run installs it). The questions and their numbers are
    unchanged.
  • The example databases and labels in docs/deployment.md name what
    the store is, not its platform: databases main_store, outlet_store,
    wholesale_store, labels usa_store, wholesale_store.
  • The unions' views read the tables of the stores, properties and
    networks with the rights of the user who built them
    (+sql_security: definer, +definer: CURRENT_USER). A user with
    SELECT on a union's database reads its model without a grant on the
    databases it adds up, and those stay closed to it. Queries through a
    view go under the builder's row policies on those tables, no longer
    the reader's: this changes the numbers of a deployment whose row
    policies there hide rows from the reader. A union changes on its next
    build
    (Several stores, properties or networks).

0.9.0

Choose a tag to compare

@ekut ekut released this 02 Nov 05:39
adfb1a8

Ossie schema 0.2.0.dev0. Tested with dactopus-ossie-clickhouse 0.12.0,
dbt v2 (dbt-oss 2.0.8) and dbt v1 (dbt-core 1.11 with
dbt-clickhouse 1.10) on ClickHouse 26.9, the WooCommerce package also
on 26.7. Several stores, properties or networks in one deployment: every
table starts with the label source_id, orders carries platform,
and the unions add up the tables of several databases. Every column has
a declared type, and some types change: an incremental run stops on the
changed schema, and a deployment runs dbt build --full-refresh once.
No input table changes.

Added

  • Every table of every package has a first column source_id, the
    label of the store, property or network its rows come from: the
    variable ga4_source_id, shopify_source_id, woocommerce_source_id
    or sensor_community_source_id, LowCardinality(String). The default
    is the package's name (shopify); a label other than 1 to 64 of
    a-z, 0-9 and _ stops the build. A dbt test on every model fails
    on a row of another label, as an incremental run after a new label
    leaves, and names --full-refresh. deployment.yml maps the role
    source_id to the variable, and tests/check_deployment.py requires
    it.
  • The Ossie models have a field source_id in every dataset. It comes
    first in every primary_key and on both sides of every relationship,
    and every count of distinct values counts them within a label
    (orders.buyer_count, product_categories.product_count,
    product_brands.product_count, readings.reporting_devices,
    sessions.users, sessions.user_conversion_rate): the tables of
    several stores, each built with its own label, keep their orders and
    customers apart. In one database the label is constant and every
    number stays the same. Its description and synonyms (store, shop;
    property, site; network) let an agent find it, and each model's
    instructions say how to answer over several: for a store, property or
    network the question names or describes, an agent lists the labels and
    takes the one that names it (for a store, also the one store in a
    currency the question names), and asks when none does; a question that
    names none means all of them; identifiers repeat across them, so an
    agent groups by one with its source_id; "today" is the date where the
    model is served, while each one's dates are in its own zone. GA4 users
    count within a property and are not matched across properties, and a
    sensor network's silent devices count from its own latest contact.
  • orders has a column platform, shopify or woocommerce, after
    currency, and the commerce model a field orders.platform: an agent
    over stores of both platforms finds which store a question about
    "the Shopify store" means, and which platform's report rules apply to
    a store. The unions carry it.
  • Unions: unions/commerce, unions/web_analytics and unions/iot, dbt
    projects that add up the tables of several stores, properties or
    networks, each built by its package into a database of its own under
    its own label. A union writes a view of each dataset of the domain's
    Ossie model, UNION ALL over the databases of
    commerce_union_databases, web_analytics_union_databases or
    iot_union_databases, with the columns the model reads and their
    types as a dbt contract. A commerce union takes Shopify and
    WooCommerce stores alike. The build stops on a database whose column
    types differ from the contract and on the union's own database among
    those it adds up; its test fails when a label is in two databases.
    deployment.yml maps the role input_databases to the variable.
    tests/check_contracts.py checks the unions' contracts against the
    Ossie models and the packages, that source_id comes first in every
    key and relationship of the Ossie models, that each package tests the
    key of each dataset (Sensor.Community's readings with
    sensor_community_unique_key, which replaces the test
    readings_one_per_sensor_time_measure), and that the copies of
    macros/union.sql and of each package's macros/freshness.sql are
    the same; tests/check_union.py checks that each label's answers over
    a union are its database's own.

Fixed

  • WooCommerce: an incremental run over a table whose columns a release
    changed stops before its pre-hooks delete a row. dbt's own check of
    the columns comes after the hooks: the run stopped with order_lines
    and refunds short of the rows of its window, refunds empty on the
    fixture.
  • WooCommerce: category and brand names (line_categories.category,
    category_path, line_brands.brand and the same fields of
    product_categories and product_brands) are decoded from the HTML
    escaping WordPress stores them with: Home & Garden, not
    Home &amp; Garden. No table changes: the next run rebuilds the
    category and brand tables whole.

Changed

  • CI runs a job only when a file it reads changed (the filters in
    ci.yml): a pull request that changes only documentation runs none,
    one that changes one package runs that package's jobs and the unions
    job. The Ossie models are validated together in a job of their own.
    The one check main requires is CI passed, which fails when a job
    failed or was cancelled; a skipped job does not block a merge.

  • CI runs on ubuntu-24.04, not ubuntu-latest, and keys the uv cache
    by ci.yml: before, the key was constant and the cache was never
    refreshed.

  • AGENTS.md keeps what an agent applies while writing and is not in
    CONTRIBUTING.md: the rules of source_id, contracts and unions, the
    model format, ClickHouse traits, which page a change ships with, and
    the development setup. Layout, scope, the authoring rules and the
    checks before a pull request are read from CONTRIBUTING.md, whose
    canon rule gains the two details only AGENTS.md had: a source that
    has less of a field still provides it when the meaning is the same,
    and no canonical field goes in "for completeness".

  • The README is split. Each package's guide (loading, settings, known
    differences from the source's own reports) is sources/<source>/README.md;
    what each Ossie model answers and its limits is docs/<domain>.md;
    the packages in a dbt project, labels, time zones, serving and unions
    are docs/deployment.md; layout, what a source package is, out of
    scope and the authoring rules are in CONTRIBUTING.md. The README keeps
    the overview, one quick start and versioning. Notes on how the
    packages were checked against test stores, and the migration note for
    snapshots loaded before 0.5.1, are gone from the guides: the checked
    numbers are stated, the history is this changelog. The install
    recipes take dactopus-ossie-clickhouse from PyPI.

  • The commerce model's instructions: one deployment can hold several
    stores, under their labels, in different currencies. An agent asks
    every amount by orders.currency, answers in the one currency or in
    each, and never adds up two; amounts of customers go by
    customers.source_id. orders.currency and customers.net_amount
    say the same. A customer of two stores counts in each.

  • README, Known differences from WooCommerce Analytics: buyers in a
    month, where the reports also count a customer with only a refund.

  • Every package declares the type of each column it writes, a dbt
    contract in models/schema.yml, and a build stops on any other. The
    columns an Ossie model reads come first in each table, with the same
    types in every package that serves the model: the tables of Shopify
    and WooCommerce add up with UNION ALL. tests/check_contracts.py
    checks it in CI. No value changes; these types do:

    • Amounts are Decimal(38, 8) and never NULL: orders.total,
      subtotal, discount_total, shipping_total, tax_total,
      refunded_total, net_total; customers.net_total; line_total
      and returned_amount of order_lines, line_categories and
      line_brands; refunds.refunded_amount; sales_events.amount.
      Shopify's were Decimal(18, 4) (customers.net_total
      Decimal(38, 4)), and Shopify's order_lines.discount_total
      follows; WooCommerce's were Nullable(Decimal(38, 8)), all but
      orders.refunded_total.
    • orders.customer_id is Nullable(Int64), customers.customer_id
      Int64: Shopify's were Nullable(UInt64) and UInt64,
      WooCommerce's orders.customer_id Int64.
    • Shopify's quantities are Int64: orders.item_quantity;
      quantity and current_quantity of order_lines,
      line_categories and line_brands. They were UInt32 and
      UInt64.
    • WooCommerce's orders.order_time, refunds.refund_time and
      sales_events.event_time are DateTime('UTC'), not
      DateTime64(6, 'UTC'): WooCommerce keeps whole seconds.
    • Flags are Bool: orders.is_cancelled and
      customers.second_order_within_*_days (Nullable(Bool)) of both
      packages, WooCommerce's orders.in_sales_reports and is_direct of
      line_categories, product_categories and
      product_categories_all. They were UInt8.
    • Shopify's order_lines.title is String, not Nullable(String).

    The columns of every table come in a new order, the model's first.
    With source_id (Added), every package runs once with
    --full-refresh in place of the next incremental run, which stops.

  • GA4: on_schema_change: fail, as in the other packages; dbt requires
    it of an incremental model with a contract.

  • WooCommerce: an order or refund in another currency without a rate
    the package knows now fails in its model, with a message that names
    the README's Known limits, where before the not_null tests on its
    amounts failed.

  • The commerce model reads its Bool flags alone, without = 1
    (orders.is_cancelled, orders.in_sales_reports,
    customers.second_order_within_*_days and the
    customers.second_order_rate_*_days metrics, is_direct of
    line_categories and `product...

Read more

0.8.0

Choose a tag to compare

@ekut ekut released this 30 Oct 05:34
f0891d7

Ossie schema 0.2.0.dev0. Tested with dactopus-ossie-clickhouse 0.12.0,
dbt v2 (dbt-oss 2.0.8) and dbt v1 (dbt-core 1.11 with
dbt-clickhouse 1.10) on ClickHouse 26.9. Customers for Shopify and
WooCommerce. orders gains columns, and the WooCommerce
orders.customer_id changes type: an incremental run stops on the
changed schema, and a deployment runs dbt build --full-refresh. No
input table changes.

Added

  • commerce.yaml: customers, one row per customer with an order the
    store's report of new and returning customers counts, with
    first_order_date, first_order_month (the cohort),
    last_order_date, number_of_orders and net_total, what the
    orders earned after refunds and cancellations (at WooCommerce, orders
    on hold included, as the Customers report's total spend). Metrics
    customer_count, net_amount and average_customer_value. orders
    joins it on customer_id, so orders break down by cohort. A customer
    record without such an order has no row, and the table holds no
    email, name or address. Both packages build it from orders on every
    run.
  • Retention windows: customers.second_order_within_30_days,
    _60_days, _90_days and _365_days, whether the second order came
    within that many days of the first, and the metrics
    second_order_rate_30_days to second_order_rate_365_days, the
    share among customers whose window is over. A window stays empty
    until its last day is before the day of the last load of orders
    (loaded_at at Shopify, synced_at at WooCommerce, in the store's
    time zone): it moves only when orders are loaded, so an
    incremental run over unchanged input changes nothing.
  • orders.days_since_previous_order and months_since_first_order
    (-1 on the first order, then calendar months, as Shopify's
    months_since_first_purchase), between the dates the customer's
    sequence goes by: processed_at at Shopify, the date placed at
    WooCommerce, in the store's time zone. The metric
    orders.average_days_between_orders averages the first.
  • Singular test orders_numbered_in_sequence in both packages: each
    customer's orders are numbered 1 to n.
  • Fixture cases for the customers and the windows
    (tests/<source>/windows.sql, check_numbers.py --windows), and CI
    steps that feed WooCommerce guest orders in parts and compare
    customers with a full refresh.

Changed

  • Shopify: cancelled orders take their place in the customer's sequence
    (customer_order_number, new_vs_repeat), as Shopify's report of new
    and returning customers counts them: on the development store a
    customer's only order, cancelled, is New, and the next one Returning.
    Over the fixture orders.repeat_order_share goes from 5/7 to 6/9.
  • WooCommerce: every order has a customer_id, the customer WooCommerce
    Analytics counts. A guest's order belongs to the account that has an
    order billed to the same email, in any letter case (the smallest such
    account where there are several); else to the guest, one per email,
    numbered by the smallest id of their orders with a minus sign
    (-112); a guest's order without an email is a customer of its own.
    Guests are counted and numbered.
  • orders.customers is renamed orders.buyer_count, no longer named
    like a dataset. This breaks saved questions and dashboards that ask
    for the old name.
  • orders.repeat_order_share, a share of orders, gives up the synonyms
    "repeat purchase rate" and "repeat rate": they usually ask about
    customers and go to the windows' rates. An agent asked for a repeat
    purchase rate now answers with a window and names it.
  • The README's known differences from Shopify's reports and from
    WooCommerce Analytics follow the new numbering. On the development
    store Shopify's customers, new and returning by month, each customer's
    total sales and orders, and orders by months_since_first_purchase
    equal the reports'. At WooCommerce the customers differ from Analytics
    where an account's order is billed to another email: the model joins
    that email's guest orders to the account. The README's commerce
    section lists the customers' metrics.
  • ROADMAP.md: Customers is done and leaves Next. Discount codes, which
    code brought the revenue, is planned after it, under Later.

0.7.2

Choose a tag to compare

@ekut ekut released this 27 Oct 06:51
7c743ad

Ossie schema 0.2.0.dev0. Tested with dactopus-ossie-clickhouse 0.12.0,
dbt v2 (dbt-oss 2.0.8) and dbt v1 (dbt-core 1.11 with
dbt-clickhouse 1.10) on ClickHouse 26.9. The Shopify input gains a
column, and refunds and sales_events need the rebuild the first entry
describes; every other table builds the same as in 0.7.1.

Fixed

  • Shopify: a refund is dated by its processedAt, the date Shopify's
    sales reports count it on, instead of its createdAt
    (#45).
    The two are equal for a refund made in the admin. A refund imported
    with a past processedAt, as a store that moves its history to
    Shopify records its returns, counted on the day of the import in
    refunds.refund_date and sales_events.event_date, and its month's
    net sales were too high. The refunds input table has a new column,
    processed_at, which a loader must deliver; the README's export
    query reads it. A deployment adds it to the input, giving the refunds
    already loaded their created_at:

    ALTER TABLE shopify_raw.refunds ADD COLUMN processed_at DateTime('UTC') DEFAULT created_at AFTER created_at;
    ALTER TABLE shopify_raw.refunds MATERIALIZE COLUMN processed_at SETTINGS mutations_sync = 1;
    ALTER TABLE shopify_raw.refunds MODIFY COLUMN processed_at REMOVE DEFAULT;

    A store with imported refunds then exports its orders again from
    before the first import, so that those refunds arrive with their
    processedAt. Last, dbt build --full-refresh --select refunds+
    rebuilds refunds and sales_events: an incremental run keeps the
    old dates of the refunds it does not reread, with no error.

  • The descriptions of refunds.refund_date and refund_time say which
    date they hold: the date the store's reports date the refund by, at
    Shopify the date it was processed, for an imported refund its past
    date and not the day it was recorded, as order_date says for orders.

  • The Shopify and WooCommerce packages set join_use_nulls = 0 in their
    queries. Their LEFT JOINs count on a missing match giving 0 (an order
    without gift cards, a refund without lines); a server or profile that
    sets it to 1 gave NULL totals and the build failed on
    sales_events.amount. tests/profiles.yml sets 1, so CI builds
    against the setting the packages must not depend on.