Skip to content

Task 02h: FFIC trade-history backfill (Phases 1+2)#8

Merged
MaksymDS merged 4 commits into
masterfrom
task02h/ffic-trade-backfill
Apr 27, 2026
Merged

Task 02h: FFIC trade-history backfill (Phases 1+2)#8
MaksymDS merged 4 commits into
masterfrom
task02h/ffic-trade-backfill

Conversation

@MaksymDS
Copy link
Copy Markdown
Contributor

Summary

Phase 1+2 of FFIC trade-history backfill. Recovered 446,512 trades across 10 previously-empty FFIC markets, raising FFIC trade coverage from 8/24 to 18/24 markets.

Phase 1 — Diagnostics

Diagnosed 24 FFIC markets across never_run, ran_returned_zero, ran_indexer_failed, partial categories. Report: reports/TASK_02H_FFIC_DIAGNOSTICS.md.

Phase 2 — Subgraph backfill

10/10 markets succeeded. Recovery includes:

  • fficd-003 cluster (4 markets, 322K trades): Iran/Khamenei deadline markets
  • fficd-004 cluster (3 markets, 82K trades): Maduro/Venezuela
  • fficd-005 (1 market, 7.5K trades): Bitcoin ETF
  • fficd-006 (1 market, 453 trades): Gene Hackman
  • fficd-007 (1 market, 14K trades): Biden pardons SBF

Report: reports/TASK_02H_PHASE2_RESULTS.md.

Bug fixes

  1. orderFilleds → enrichedOrderFilleds schema mismatch on master (cherry-picked from task02d).
  2. PostgreSQL 32,767 parameter limit in wallet upsert (chunked at 10K rows).

Out of scope

6 ran_indexer_failed markets remain — 4 fficd-001 election markets ($153M-$1.5B) and 2 fficd-003 top markets ($174M, $269M). Phase 3 will recover the 2 fficd-003 markets via Polygonscan logs API. Election markets deferred to future revision.

Files

  • reports/TASK_02H_FFIC_DIAGNOSTICS.md
  • reports/TASK_02H_PHASE2_RESULTS.md
  • reports/POLYGONSCAN_COLLECTION_STATUS.md
  • scripts/diagnose_ffic_missing.py
  • fflow/collectors/subgraph.py (cherry-pick enrichedOrderFilleds + wallet chunking fix)
  • fflow/config.py (extra="ignore", AliasChoices for anthropic_api_key)

After merge

Branch stays open for Phase 3 (Polygonscan logs API for fficd-003 indexer-failed markets). Phase 3 will be appended to this branch and PR.

🤖 Generated with Claude Code

MaksymDS and others added 4 commits April 27, 2026 18:10
…dexer_failed, 8 ok

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ected

Fixes applied:
- subgraph: chunk wallet upsert at 10K rows (PostgreSQL 32767 param limit)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… enriched, 3 blockers identified

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@MaksymDS MaksymDS merged commit c952023 into master Apr 27, 2026
1 check failed
@MaksymDS MaksymDS deleted the task02h/ffic-trade-backfill branch May 1, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant