Skip to content

v1.3.0: fan-out joins honor tuple uniqueness tests

Latest

Choose a tag to compare

@matbcassis matbcassis released this 12 Jun 07:46

fan_out_joins no longer flags a join whose key set is already covered by a model-level uniqueness guarantee.

Before this release the check only recognized a column-level unique test, so a model that declares uniqueness on a column tuple through dbt_utils.unique_combination_of_columns was still flagged, even when downstream models joined it on the whole tuple. Tuples attached through a YAML anchor alias were missed for the same reason: model-level tests were never read.

What changed:

  • Model-level data_tests / tests are now parsed. dbt_utils.unique_combination_of_columns (and the older unique_combination) tuples are read from both the classic combination_of_columns: form and the newer arguments: combination_of_columns: form. YAML anchors and aliases are expanded by the loader, so an anchored test is read the same as an inline one.
  • A join key is suppressed only when every join clause that uses it covers a uniqueness guarantee on the joined model: a column-level unique test or the tested PK among the clause keys, or a unique-combination tuple that is a subset of the clause keys. A join on the whole tuple or a superset cannot fan out and is not flagged. A join on a strict subset of the tuple still can, so it stays flagged.

Verified on three public projects: Cal-ITP fan_out_joins 6 to 2, with Mattermost (6) and GitLab (26) unchanged, so no real fan-out is suppressed. New regression fixture test-fixtures/fan-out-joins/ and scripts/tests/test_fan_out_joins.py.