Create stats view - #110
Merged
Merged
Conversation
anxolin
force-pushed
the
trade-stats
branch
from
September 10, 2020 08:51
f2d868f to
21f8dcf
Compare
jeff-dude
added a commit
that referenced
this pull request
May 26, 2026
…_details (#9689) `kalshi.markets_raw.volume_fp` can freeze at creation-time for actively traded markets because the market-feed `markets-sync` cron queries Kalshi with `min_created_ts` and never re-fetches existing markets. The `kalshi_market_details` spell then filters on `volume_fp >= 100` against that stale value, dropping markets that look idle on the raw side but trade heavily in reality (e.g. KXIPLGAME-26MAY15CSKLSG-CSK: 21k trades, 20.5M contracts on 5/15 but markets_raw.volume_fp = 0 since 5/8). This silently dropped 50-91% of recent Kalshi trades from the downstream `kalshi.market_trades` spell — which INNER JOINs market_details on ticker — starting 2026-05-08, the day market-feed PR #110 routed the ticker WebSocket stream into the new `kalshi.market_updates_raw` table instead of refreshing markets_raw. Fix: source live volume_fp from `market_updates_raw` (per-tick state stream), apply the >= 100 filter against the coalesced value, and project the live value downstream. Existing fallback semantics preserved for tickers with no updates in the incremental window. Validated against prod: rescues 469,719 tickers and ~6.75B contracts of trade volume currently being filtered out. Also adds `market_updates_raw` to `sources/kalshi/_sources.yml` and documents the markets_raw freshness caveat on its description. A `--full-refresh` of `kalshi.market_details` (and a downstream rebuild of `kalshi.market_trades`) is needed after merge to recover the ~2 weeks of dropped trades; the incremental cadence alone won't pick up tickers whose `updated_time` is older than the incremental window. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: jeff-dude <102681548+jeff-dude@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a new analytics table for identifying the origin of the volume