Skip to content

Commit 491ff46

Browse files
committed
Fix: Add deduplication to event_details CTE to prevent row fan-out
The event_details CTE was reading from market_details_raw without deduplication. Since market_details_raw contains temporal snapshots with multiple rows per event_ticker (indicated by last_updated_ts), the LEFT JOIN on event_ticker was causing a fan-out, producing duplicate ticker rows that violated the unique_combination_of_columns test and cascaded incorrect duplicates into kalshi_market_trades. Added QUALIFY with ROW_NUMBER() to keep only the latest row per event_ticker based on last_updated_ts DESC, ensuring 1:1 join cardinality and preventing duplicate ticker rows.
1 parent 4f4ca39 commit 491ff46

1 file changed

Lines changed: 1 addition & 0 deletions

File tree

dbt_subprojects/daily_spellbook/models/_projects/kalshi/kalshi_market_details.sql

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@ with markets as (
5454
strike_period,
5555
last_updated_ts
5656
from {{ source('kalshi', 'market_details_raw') }}
57+
qualify row_number() over (partition by event_ticker order by last_updated_ts desc) = 1
5758
)
5859

5960
select

0 commit comments

Comments
 (0)