Skip to content

feat(verify+auditor): OSAP factor-exposure proxy contract codification (closes #217 + #218)#221

Merged
dackclup merged 1 commit into
mainfrom
claude/cool-cannon-Ztmeg
May 23, 2026
Merged

feat(verify+auditor): OSAP factor-exposure proxy contract codification (closes #217 + #218)#221
dackclup merged 1 commit into
mainfrom
claude/cool-cannon-Ztmeg

Conversation

@dackclup
Copy link
Copy Markdown
Owner

Summary

Closes #217 + #218 in one focused diff — both issues are the same root cause (Phase 4h OSAP factor-exposure proxy contract not codified anywhere) attacked from opposite sides of the audit surface.

The 2026-05-23 cron #3 surfaced a false-positive escalation chain: stock-detail-auditor read the universe-wide identical osap_signals dict (502 tickers × same dict) as cron-wide data corruption and woke incident-commander to a P1 that downgraded to P3 after walking the schema + frontend + blending pipeline. Root cause — the documented Phase 4h factor-exposure proxy contract (compute/features/osap_replicate.py:14-35, locked 2026-05-18) was nowhere in the agent prompts or the verify-helper. Both directions of drift (regression OR Phase 4i+ graduation) were silent.

What changed

File Change
.claude/agents/stock-detail-auditor.md Step 3 gains "Documented patterns — NOT broken_data" 3-row recognition table; new documented_proxy verdict; escalation row routing universe-wide scope-note-contract patterns to methodology-scientist (NOT incident-commander)
.claude/skills/verify-production-output/helper.py New section_l_osap_proxy_invariant() — classifies OSAP surface into phase4h_proxy / blending_regression / graduated / schema_drift / unknown; wired into Section A-L main() runner
tests/test_verify_helper.py +6 Section L cases (empty/skip + no-OSAP/skip + proxy-pass + regression-fail + graduated-warn + drift-fail + None-coverage-gap exclusion); 12 → 18 tests in this file
CLAUDE.md + AGENTS.md Lockstep in-flight entry per §Conventions

Live data verification

Section L on the 2026-05-23 cron #3 output:

=== Section L — OSAP factor-exposure proxy invariant ===
  ✓ tickers with osap_signals: 502
  ✓ unique signal sets: 1
  ✓ distinct blended scores: 428
  ✓ mode: phase4h_proxy (1 signal set × 428 blended scores)

Section L passes; Summary 0 failures, 1 warnings (unchanged from pre-PR baseline).

Failure modes Section L catches

  1. blending_regression — uniform signals AND uniform blended scores. Future code change that drops the per-ticker scalar multiplication in compute/scoring/osap_blend.py will FAIL the helper instead of passing silently.
  2. graduated — varying signals (Phase 4i+ true per-stock replication). WARNS until the osap_replicate.py:14-35 scope-note is updated to declare the graduation explicitly — forces an intentional contract bump.
  3. schema_driftosap_signals key set differs across tickers (broken uniformity without graduating cleanly). FAIL.

Scope

No compute / schema / scoring / valuation / frontend code change — agent prompt + helper script + tests + lockstep docs only.

Test plan

  • ruff check clean on touched files
  • pytest tests/ -m "not network" → 1115 passed (7 skipped pyqlib/ipca env-missing, 22 deselected network)
  • python .claude/skills/verify-production-output/helper.py on live 2026-05-23 cron feat(phase-1): universe + prices + momentum stub #3 data → Section L emits mode=phase4h_proxy (1 signal set × 428 blended scores); overall 0 failures, 1 warnings (unchanged baseline)
  • stock-detail-auditor.md prompt update reviewed for clarity
  • CLAUDE.md + AGENTS.md lockstep verified (both touched per §Conventions)
  • CI green
  • Spot-check on next cron feat(phase-2): SEC EDGAR fundamentals + per-stock detail pages #4 — Section L still passes; if a future cron sees mode=graduated without an intentional Phase 4i+ scope-note update, the WARN will surface it

Generated by Claude Code

closes #217 + #218)

The 2026-05-23 cron #3 surfaced a false-positive escalation chain: the
stock-detail-auditor read the universe-wide identical osap_signals dict
(502 tickers × same dict) as cron-wide data corruption and woke
incident-commander to a P1 that downgraded to P3 after walking the
schema + frontend + blending pipeline. Root cause: the documented
Phase 4h factor-exposure proxy contract (compute/features/osap_replicate.py:14-35,
locked 2026-05-18) was nowhere in the agent prompts or the verify-helper.
Both directions of drift — regression OR Phase 4i+ graduation — were
silent. This commit closes the loop on both sides.

(a) stock-detail-auditor (issue #217)
  - Step 3 gains a "Documented patterns — NOT broken_data" callout
    with a 3-row recognition table:
      proxy_active + blended_varies → documented_proxy
      proxy_active + uniform_blended → broken_data (osap_blend.py bug)
      graduated → escalate to methodology-scientist
  - Third verdict type `documented_proxy` added alongside real_outlier
    and broken_data
  - Escalation table gains a row routing universe-wide scope-note-
    contract patterns to methodology-scientist (NOT incident-commander)

(b) verify-production-output/helper.py Section L (issue #218)
  - section_l_osap_proxy_invariant() classifies the OSAP surface into
    phase4h_proxy / blending_regression / graduated / schema_drift / unknown
  - Wired into the Section A-K main() runner (now A-L)
  - Live data: 2026-05-23 cron #3 passes with mode=phase4h_proxy
    (1 signal set × 428 blended scores on 502 tickers)
  - Guards both directions: a future osap_blend.py regression that
    drops the per-ticker scalar multiplication FAILS; a Phase 4i+
    graduation that lands without updating the scope-note WARNs

(c) tests/test_verify_helper.py
  - 6 new Section L cases: empty/skip + populated-no-OSAP/skip +
    phase4h_proxy/pass + blending_regression/fail + graduated/warn +
    schema_drift/fail + None-coverage-gap exclusion
  - 12 → 18 tests in this file; full suite 1115 passed locally

No compute / schema / scoring / valuation / frontend code change —
agent prompt + helper script + tests + lockstep docs only.
@vercel
Copy link
Copy Markdown

vercel Bot commented May 23, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
quantrank Building Building Preview, Comment May 23, 2026 9:35am

@dackclup dackclup marked this pull request as ready for review May 23, 2026 10:14
@dackclup dackclup merged commit eba0fde into main May 23, 2026
4 checks passed
@dackclup dackclup deleted the claude/cool-cannon-Ztmeg branch May 23, 2026 10:14
dackclup added a commit that referenced this pull request May 27, 2026
…AUDE.md (#271)

Refactors a user-shared research report (Master Prompt + 6 phase sub-prompts
+ CLAUDE.md template) into the existing doc surface without creating a new
.claude/skills/agentic-6-phase/ skill. The report's underlying logic is
already implemented in the 18 subagents + CLAUDE.md §Auto-routing; what
was genuinely missing was a 6-phase mapping table a new session can scan
in < 30 sec on top of the 9 phases.

Scope (2 substance files + 1 INFLIGHT entry):

- WORKFLOW.md — new section "Agentic 6-Phase Cadence" between §"Tools
  You'll Use Daily" and §"Phase Overview". Mapping table (Step × Fire
  trigger × Subagent(s) × Done when) over Planning → Code Gen →
  Integration → Test → Deploy → Monitor + 5 cadence invariants. Reuses
  the 18 standing subagents — no new agent files. Session-start protocol
  cites schema 0.10.5-phase4.5e (PRs #264 + #265; cron #4 still at
  0.10.4, next cron Wed 2026-05-27 re-renders at 0.10.5), defense
  layer 33 declared = 7 vetoes + 26 annotates, tag v1.3.0-phase4.5e,
  CVE baseline 15 open (0C / 6H / 7M / 2L) after PR #194 patch +
  PR #226 triage.
- CLAUDE.md — new §Conventions bullet "Session-start phase
  identification" (~5 lines) pointing readers at PHASE_STATUS.md
  §"Current state" + WORKFLOW.md §"Agentic 6-Phase Cadence" using the
  standing 18 subagents.
- PHASE_STATUS_INFLIGHT.md — new in-flight entry per PR #237 side-file
  lockstep convention.

Out of scope (deliberately NOT done per user direction 2026-05-27):

- NO .claude/skills/agentic-6-phase/ — overhead exceeds benefit
- NO Master Prompt / phase sub-prompts copied into the repo
- NO edits to any of the 18 subagent files under .claude/agents/
- NO AGENTS.md substance edit — the cadence is Claude-Code-subagent-
  specific; cross-tool agents would route differently. INFLIGHT entry
  satisfies §Conventions "ship with every PR" lockstep.

docs-reviewer verdict (2026-05-27, agent id a2c87ed3679f55fe5):
NEEDS-CROSS-REF-FIX — both items applied in this commit:
1. CVE attribution: "after PR #226 triage" → "after PR #194 patch +
   PR #226 triage" (PR #194 closed the 10 advisories; PR #226
   documented the resulting state)
2. Step 4 fire-trigger col: "Sections A-J" → "Sections A-L"
   (Section L added by PR #221 OSAP proxy invariant; internal match
   with the same row's Done-when col)

All else passes: 4 cited numbers, 18 agent names, 3 cross-refs, token
budget (WORKFLOW ≤ 1 page, CLAUDE ≤ 5 lines), Rule 16 + Rule 18 no
contradiction.

Pre-existing SKILL.md schema-version table gap (rows for 0.10.5-phase4.5e
PR #264 + valuation_output_anomalous rename PR #265 missing) escalated
to schema-sentinel as separate doc-only PR per docs-reviewer recommendation
— not blocking on this scope.

Verification:
- ruff check . — N/A (no Python)
- python -m compute.output.schema_check — N/A (no schemas)
- pytest tests/ -m "not network" — N/A (no test surface)
- docs-reviewer subagent — PASS after the 2 fixes above

Co-authored-by: Claude <noreply@anthropic.com>
dackclup added a commit that referenced this pull request May 30, 2026
…#326 §Gotchas (#327)

Full docs-reviewer substance pass on CLAUDE.md → 5 MUST-FIX + 4 SHOULD-FIX, all fixed. CLAUDE.md only (+ PHASE_STATUS_INFLIGHT.md lockstep entry); no code/schema/compute/frontend change.

MUST-FIX: §Phase status "In flight" marked PR #312 (tasteful-motion) as "THIS PR" → merged (e602485); "Recently merged" frozen at #310 → drained #311#326 (16 entries, SHAs verified); "Next deliverables" Issue #67 flip listed pending → DONE (PR #294, USE_SECTOR_COE=True); "Section A-H/A-J" → "A-L" (helper is A-L since PR #221).

SHOULD-FIX: §Stack "Phase 3b on this PR" → "(merged)" + edgartools 5.31→5.32; §Gotchas compute/main.py line refs re-anchored (840→879 · 1965-66→2084-85 · 717→728 · 785→805 · 972→1025); +2 §Gotchas for PR #326 invariants (sidebar data-rail↔globals.css pre-paint lockstep; AreaChart re-park debounce ≥300ms).

Confirmed no drift: schema 0.10.11-phase4.6 · skills 46 · agents 19 · hooks 3. docs-reviewer re-check: DOCS-CLEAN (all 16 SHAs match, 5 line refs accurate, 2 gotchas code-backed). Lockstep via PHASE_STATUS_INFLIGHT.md side-file (PR #237).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

stock-detail-auditor: add factor-exposure proxy heuristic for OSAP signals

2 participants