feat(filings): Wave 1 PR B — Form 4 insider transactions ingest

[cipher813]

Summary

Wave 1 PR B of the institutional data-revamp arc. Expands EDGAR coverage beyond existing 8-K + 10-K/Q with Form 4 (Section 16 insider transactions) — the biggest institutional alpha signal still missing from the substrate.

Form 4 is filed by every officer, director, and 10%+ beneficial owner within 2 business days of each transaction. Net insider activity is a real institutional signal:

• Cluster buying by multiple insiders → strong bullish
• CEO/CFO buys vs RSU vests → distinguished from forced exercises
• Sales after earnings → may signal weakness
• 10b5-1 plan disclosures → reduce noise from scheduled sales

13F + longer-form text filings (14A, S-1, 13D-G) defer to PR B.1+.

What's in

New module rag/pipelines/ingest_form4.py:

Form4Transaction row shape (frozen dataclass, 20 fields)

Group	Fields
Identity	ticker, issuer_cik, accession_number, filed_date, schema_version
Reporting owner	name, cik, is_director, is_officer, is_ten_percent_owner, officer_title
Transaction	date, code (A/D/M/S/P/F/G/J), security_title, shares, price_per_share, acquired_disposed_code, value_usd (computed), shares_owned_after, direct_or_indirect, is_derivative
Provenance	fetched_at

parse_form4_xml() — pure XML parser

• Handles both non-derivative AND derivative transaction tables
• Multi-transaction filings emit multiple rows
• Tolerates direct-text vs <value>-wrapped transactionCode (both forms occur in actual EDGAR XML)
• Returns [] on parse failure (logged) — caller skips, doesn't crash

_search_form4_filings() — EDGAR discovery

Shared CIK lookup + submissions API pattern from ingest_8k_filings.py. Rate-limited at ~8 req/sec under EDGAR's 10/sec ceiling.

write_form4_parquet() — S3 writer

• s3://alpha-engine-research/data/insider_transactions/{filed_date}.parquet
• One parquet per filed_date holds all insider transactions across all tickers filed that day
• Idempotent overwrite

ingest_for_tickers() — orchestrator

End-to-end: discovery → download → parse → write. Returns stats dict. Groups transactions by filed_date so one parquet write per day, not per filing.

CLI

python -m rag.pipelines.ingest_form4 --tickers AAPL,MSFT --lookback-days 90

What's deferred to PR B.1+

• 13F (institutional positioning) — quarterly XML schema with holdings tables; bigger payloads + ticker_to_cik reverse lookup
• 14A / DEF 14A (proxy statements) — RAG-only text ingest mirroring 10-K/Q pattern
• S-1 / S-4 (registration statements) — RAG-only
• 13D / 13G (5%+ ownership disclosures) — structured XML, similar shape to Form 4

Test plan

• +16 unit tests (tests/test_ingest_form4.py):
  • XML parsing: single sale / multi-transaction (2 non-deriv + 1 deriv) / missing optional fields → None / malformed XML returns [] / empty filing returns []
  • transactionCode unwrapping (direct text + <value>-wrapped)
  • DataFrame: empty list → empty df with schema / round-trip preserves all fields
  • S3 writer: key format / write-read round-trip via in-memory mock / empty list writes empty-schema parquet
  • Orchestrator: end-to-end discovery → download → parse → write / non-Form-4 filings skipped at discovery / dry-run skips writes / one bad download isolated (batch continues)
  • SCHEMA_VERSION pinned to 1 + present on every row
• Full data suite: 945 passing (1 skipped) in 4s

🤖 Generated with Claude Code