Predict Raven

English is the primary README. 中文版见 docs/README.zh-CN.md.

Last updated: 2026-08-24

predict-raven is an open-source forecasting agent framework: it lets an AI agent estimate the probability of real-world events, gather and weigh evidence continuously, and act on the result. The same forecasting intelligence powers a family of products today:

• Autonomous prediction-market trading — the first autonomous, continuously-running trading agent on Polymarket. It estimates fair probabilities, compares them to market-implied odds, and trades the edge under hard, service-layer risk controls.
• Market-blind public forecasting — transparent, Brier-scored probabilities for all 48 teams at the 2026 World Cup, deliberately produced without ever reading a market price. Live at forecasting-agent.com, scored in public on the track record page.
• Hosted Forecasting Engine — the same iterative forecaster as hosted products: an interactive research console at /engine, the Raven Delta news-impact engine at /delta, and a forecast API + MCP service (probability + reasoning + evidence as JSON, plain text, or PDF). See Forecasting Engine (hosted).
• Autonomous paper trading — a simulation-only twin of the trading agent running unattended in the cloud: market-blind position evaluations three times a day, net-edge exits, and a daily self-review with a Brier skill score against the market. See Autonomous paper trading.
• Delta PM — news-driven US-stock shadow trading — an autonomous analyst→PM pipeline over US equities: a real news feed, two gates (does it matter / is it already priced in), a price-blind valuation thesis, and a fully audited shadow trade on tokenized US-stock perp market data. See Delta PM.
• Multi-model forecast fleet — seven independent $10k paper books, one per frontier model, running the identical harness so equity and Brier skill can be compared across models. See Multi-model forecast fleet.

Naming note: Raven Delta (/delta) and Delta PM are different products. Raven Delta is the interactive surface — you paste one news item and read the impact analysis. Delta PM is the autonomous shadow portfolio manager — it consumes a live news feed end-to-end and books audited paper trades on its own.

Watch live:

• World Cup forecasts (market-blind): forecasting-agent.com/world-cup · track record — EN / 简中 / 繁中
• Forecasting Engine console: forecasting-agent.com/engine (browsing is public; new runs are invite-gated) — EN / 中文 toggle
• Raven Delta news-impact engine: forecasting-agent.com/delta (invite-gated) — EN / 中文 toggle
• Paper-trading book review: forecasting-agent.com/live-predict-raven (invite-gated, live VM snapshot) — EN / 中文 toggle
• Delta PM decision-audit page: forecasting-agent.com/live-delta-pm (invite-gated, live VM ledger) — EN / 中文 toggle
• Delta PM operator console: forecasting-agent.com/pm (token-gated) — EN / 中文 toggle
• On-chain positions / fills (Polymarket profile): 0x6664...614e

The historical trading decision-log page was retired in the 2026-07 cleanup; the live-trading line's public record is the on-chain profile above. autopoly-pizza-spectator.vercel.app now redirects to the World Cup site.

The forecaster's web-search layer silently degraded and contaminated past outputs. Flagged by the user for a dedicated fix next session; actionable checklist lives in docs/agent-handoff.md (P0).

• What broke: both search paths scraped keyless DuckDuckGo, and DDG now serves HTTP-200 CAPTCHA pages that parsed as "search succeeded, 0 results" — runs were told "no evidence exists" when the search never ran. Measured 2026-08-22: 4 of 6 queries blocked.
• Contamination window (corrected 2026-08-23 after a VM archive audit): the paper-trading agent is clean — all 33 of its engine dossiers record provider: "claude" (native WebSearch), so its Brier scores never depended on the DDG scraper; the earlier "entire lifetime contaminated" claim was wrong. Actually affected: runs on the DeepSeek/Kimi provider path (raven app provider toggle, forecast-api runs pinned to deepseek, local experiments) since 2026-07-03, and the final live-trading day (2026-06-10), where 6–8 of 12 pulse news queries returned empty per run (vs 1/16 on 06-07). World Cup forecasts and Delta PM are not affected.
• Fixed so far: DuckDuckGo removed entirely; search now runs on Exa (EXA_API_KEY, preferred) or Tavily, and fails loudly with no key (PR #108). Production deployed 2026-08-23: the VM's paper-agent/forecast-api run the fix with a working key (in-container Exa call verified). Pending: VM archive audit + re-scoring the contaminated reports.

Everything in this repo is built around one capability: an AI agent that can research a question about the future, weigh cited evidence, and defend a probability. Its purest expression is the reusable iterative forecaster in packages/forecast-engine; several products run that engine directly, always as an isolated child process:

                    packages/forecast-engine
              (iterative evidence forecaster)
                            │
        spawned as an isolated process per question
       (scripts/forecast/cli.ts — process isolation is
        also what enforces market-blindness: the child
        sees only the question and settlement rules)
                            │
     ┌──────────────────────┼──────────────────────────┐
     ▼                      ▼                          ▼
 /engine research     Forecast API + MCP     paper-trading books
 console              (services/            (services/paper-agent:
 (apps/raven)          forecast-api)         the Tokyo $10k book +
                                             the 7-book model fleet)

The other product lines apply the same evidence-first discipline through their own purpose-built pipelines:

• Polymarket live trading — Market Pulse research + a service-layer risk engine (services/orchestrator / services/executor): the AI independently estimates event probabilities, gathers evidence, compares against market-implied odds, and issues trading instructions that combine edge with capital return efficiency.
• World Cup blind forecasts — a statistical core (live Elo prior, Davidson three-way model, Monte-Carlo bracket simulation) plus bounded, source-cited Bayesian evidence updates (scripts/world-cup, packages/sports-*).
• Raven Delta — a news→impacted-stocks analyzer with its own LLM provider seam, modeled on the engine's adapters (apps/raven-delta).
• Delta PM — a two-gate news judgment, a price-blind valuation thesis, and a PM decision engine with per-decision audit records (services/delta-pm).

1. Superhuman reasoning on complex tasks — Agents now match or exceed human-level reasoning on complex problems. Most of the time, the human edge is better information sources rather than reasoning, and engineering can close that gap. The core analytical capability is already in place.
2. Broad coverage and fast reaction time — An Agent can monitor thousands of markets 24/7 and spot pricing dislocations no individual could track. When news breaks, the Agent responds in seconds; a human needs at least three minutes. Opportunities like this appear across countless markets.
3. Prediction markets are still a blue ocean — Most participants in political and tech prediction markets lack a clear pricing model and broadly fear inventory management and adverse-selection risk. Systematic Agent trading faces very little competition in these areas. Even in sports, there is plenty beyond moneyline markets.

• Every order the Agent places and its decision reasoning are published on the website
• The Agent runs continuously in the cloud — not as ad-hoc local scripts — with no human in the loop
• Runs on @polymarket/clob-client-v2 with pUSD as the default collateral; V2 cutover is 2026-04-28 11:00 UTC, see docs/internal/plan/2026-04-28-v2-cutover-runbook.md for the runbook

The same agent powers a probability-research product that is deliberately decoupled from the trading side: it forecasts events without reading any betting or prediction-market price, so the output is an independent estimate rather than a re-statement of the market consensus.

The 2026 World Cup deployment is the public showcase — 87 questions (champion, group winners, group matches, knockout qualifiers) for all 48 teams:

• Statistical prior: live Elo ratings feed a Davidson three-way model for single matches; tournament questions run 100,000 Monte-Carlo simulations over the official bracket.
• Bayesian update: key evidence (injuries, lineups, form, venue/altitude/weather) is converted into a bounded adjustment on the prior — at most ±8 percentage points per match, and nothing moves without a cited source.
• Public scoring: every forecast is Brier-scored in public after the match settles; wrong calls stay on the record. The track record page benchmarks the blind forecasts after the fact against Polymarket's implied probabilities at forecast time — Mock PNL, Brier skill score vs the market, calibration (ECE), and hit rate — for both the group stage and the knockout round of 32.

Market data is used only for event structure and settlement mapping (slug / conditionId / resolution rules); price fields are stripped at cache-write time. Code lives in scripts/world-cup/, packages/sports-data/, packages/sports-model/, and apps/web/app/world-cup/. This is probability research, not betting advice.

The evidence-gathering core is packaged as a reusable iterative event forecaster in packages/forecast-engine: given a binary question, it frames the event, runs bounded evidence rounds with cited sources, and produces a traceable probability plus a decision-first report. Correctness guardrails are built in — a framing audit, a model self-estimated prior, independence-aware evidence aggregation (cluster discounting), a mandatory disconfirmation pass, consequential source verification, and an explicit saturated status when a probability pins at the engine's floor/ceiling. A market-blind mode (FORECAST_MARKET_BLIND=1) bans betting-market prices from prompts, search, and evidence weighting.

Three hosted surfaces run this engine on a cloud VM, all under forecasting-agent.com:

Engine runs are metered: a daily quota plus file-backed invite codes with per-code limits and usage metering.

A simulation-only twin of the trading agent (services/paper-agent) runs a $10k paper book unattended on the same VM — zero private keys, zero order endpoints; its only network surface is public market-data reads:

• Market-blind evaluations, 3×/day — each position is re-forecast by an isolated engine process that sees only the market question and settlement rules — never the position, cost basis, or order book.
• Net-edge exits + model-free stop-loss — a position is closed when its fee-adjusted edge turns negative; a hard stop-loss overrides the model. Saturated winners (price pinned at the ceiling on the position's favorable side) are held to resolution instead of ceiling-sold.
• Realistic fills — simulated orders walk the real order book (real slippage) and pay real per-market CLOB fees, with entry fees folded into round PnL; exits use a hybrid 50% market + 50% maker-limit split.
• Daily reflection — the agent writes a daily self-review: counterfactual exit alpha, Brier calibration, and a Brier skill score vs the market — the standing answer to "is the agent actually beating the market?".

The book is reviewed publicly at forecasting-agent.com/live-predict-raven (invite-gated): a live snapshot pulled from the VM at request time, the equity curve, the closed-round table, and the effective run parameters.

Delta PM takes the forecasting discipline into US equities: an autonomous analyst→PM pipeline that reads a real news feed and decides, for each item, whether it is worth a trade — then books that trade on a shadow ledger (paper only, zero exchange credentials, no order endpoints). Market data comes from tokenized US-stock perpetuals on Hyperliquid; the tradable universe is ~20 large-cap US tech names.

The pipeline runs in stages, each with its own contract:

• Two gates before any analysis money is spent — gate 1: does this news matter for the universe at all? gate 2: is it already priced in, judged by first-seen timing of the story crossed with the actual price reaction since then. Most items die here at near-zero LLM cost.
• Price-blind thesis — for survivors, an analyst pass writes a valuation thesis (impact path, EPS linkage, stated assumptions) without seeing the price reaction, so the thesis can't rationalize the chart.
• PM decision — a deterministic decision layer sizes the trade only if the residual edge (thesis-implied move minus what the market already moved, β-adjusted against an RTH-aligned benchmark) clears a threshold, under tiered position caps, a −20% per-position hard stop, and a −25% portfolio halt.
• Full audit chain — every decision (including every rejection) records the edge decomposition, each sizing guard it passed through, and veto tags. The chain is reviewable end-to-end at /live-delta-pm (invite-gated), pulled live from the VM ledger.

Phase 0 (shadow mode) has been running unattended on the cloud VM since 2026-08-23. Code: packages/delta-pm-contracts (machine contracts), services/delta-pm (engine, port 8792), apps/delta-pm-console (operator console, port 3400, public at /pm behind a token gate); operations manual at docs/delta-pm-operations.md. Not investment advice; no real orders are placed.

The paper-trading agent doubles as a model benchmark harness: seven independent $10k paper books run the identical code, rules, and evaluation cadence — the only variable is the model doing the forecasting (Claude Fable / Opus / Sonnet, GPT-5.6 ×2 via the Codex CLI provider, Kimi K3, DeepSeek v4-flash). Ledgers are fully isolated per book; evaluation slots are staggered around the clock.

The standing question it answers: with the harness held constant, which model actually forecasts better? Each book accrues its own equity curve and Brier skill score against the market, so the comparison is apples-to-apples in a way one-off benchmark runs are not.

The fleet runs unattended on a separate always-on server (live since 2026-08-23). A monitor tracks each provider's quota and burn-rate waterlines and posts a daily digest; fleet tooling lives in scripts/fleet. There is no public page for the fleet yet.

Driven entirely through an AI Agent (Claude Code / Codex / OpenClaw) in natural language. No commands to memorise.

Prerequisite: install either Claude Code or Codex CLI, git clone this repo, and start the Agent inside the repo directory before going through the 4 steps below.

Say to the Agent:

install the dependencies for predict-raven

Expected: the Agent runs pnpm install + pnpm build and tells you whether the environment is ready. If you don't have Node.js / pnpm yet, it'll install those first. No Docker, no real wallet required at this stage.

Predict-Raven supports multiple capital-management modes, including social login (Google, Telegram) and OKX Agentic Wallet.

In private-key mode, get your Polymarket wallet credentials from polymarket.com → Settings → Export Wallet. Create a new .env.live-test (use .env.example as the template) and fill in these 5 fields:

• WALLET_PROVIDER=private-key
• PRIVATE_KEY — the wallet private key
• FUNDER_ADDRESS — the Polymarket proxy wallet address
• SIGNATURE_TYPE — signature type (0 or 1)
• CHAIN_ID — 137 (Polygon mainnet)

OKX Agentic Wallet mode does not need PRIVATE_KEY, but you must log in with onchainos wallet login/verify first and set WALLET_PROVIDER=onchainos, FUNDER_ADDRESS (the Polymarket deposit/proxy wallet with collateral/allowance), SIGNATURE_TYPE=3, and CHAIN_ID=137.

Then say:

configure my wallet

Expected: the Agent reads your .env.live-test, confirms the wallet can talk to Polymarket, and prints the wallet address and current balance. If any field is missing, it tells you exactly which one.

Say:

recommend some trades, no actual orders

Expected: the Agent lists a few suggested trades — each with the market, side, stake size, and its estimated edge and capital return efficiency. The full reasoning is also written to disk as markdown so you can review it later. No orders are placed in this step, so you can run it end-to-end even without USDC in the wallet.

Say:

run the pulse with real money

Expected: the Agent places real orders based on the recommendations from step 3 and tells you which ones filled and which got rejected.

For concrete pnpm commands (forecast:*; the old pulse:* names remain as compatibility aliases), env vars, and archive directories, see docs/diagrams/dev-reference.md.

The real-money Polymarket line is organised in four layers; data flows top to bottom. (For how the other product lines relate to the shared forecasting core, see System Design above.)

┌─────────────────────────────────────────────────────────────┐
│  Layer 1 · Research / Pulse                                 │
│  Fetches Polymarket listings, produces the Pulse pool       │
│  Output → runtime-artifacts/reports/pulse/...               │
└───────────────────────────┬─────────────────────────────────┘
                            ▼
┌─────────────────────────────────────────────────────────────┐
│  Layer 2 · Decision / Runtime                               │
│  orchestrator turns Pulse + position context → decisions    │
│  Primary: pulse-direct │ Legacy: provider-runtime           │
└───────────────────────────┬─────────────────────────────────┘
                            ▼
┌─────────────────────────────────────────────────────────────┐
│  Layer 3 · Execution / Risk                                 │
│  Service-layer hard risk trimming → executor order / sync   │
│  FOK market · ≤15% per trade · ≤80% expo · ≥30% halt        │
└───────────────────────────┬─────────────────────────────────┘
                            ▼
┌─────────────────────────────────────────────────────────────┐
│  Layer 4 · State / Archive / UI                             │
│  DB / local state / runtime-artifacts archive / apps/web    │
└─────────────────────────────────────────────────────────────┘

The system is not tied to a single AI framework. Swapping between Codex / Claude Code / OpenClaw is a one-line change:

AGENT_RUNTIME_PROVIDER=codex        # options: codex / claude-code / openclaw

Custom Agents are plugged in via a template command configured through <PROVIDER>_COMMAND. See .env.example for examples and placeholders.

Model capability is named in three tiers after the Norse Norns, so "which model" is one memorable choice instead of a raw model id scattered across env vars:

This is a thin alias / mapping layer (@autopoly/norns), not a rewrite. Anywhere a model id is read, a tier name may be used instead and is resolved to a concrete model for that provider family. Raw model ids and empty defaults pass through unchanged, so every existing config keeps working — behaviour changes only when a tier name is explicitly used. Each tier also carries soft depth knobs (token budget, evidence/pass counts) that drivers can scale by.

Where it applies:

• Trading engine / provider-runtime: CODEX_MODEL / CLAUDE_CODE_MODEL / OPENCLAW_MODEL accept a tier name (e.g. CLAUDE_CODE_MODEL=skuld), resolved per provider family (codex → openai, claude-code / openclaw → anthropic).

The tier table is the single source of truth in packages/norns/src/index.ts; all model ids are env-overridable.

There are currently two decision strategies, selected via the AGENT_DECISION_STRATEGY environment variable:

Pulse markdown → Regex/table parsing → PulseEntryPlan
                                        ↓
Current positions → reviewCurrentPositions → hold/reduce/close
                                        ↓
           monthlyReturn sort (top 4) → 20% batch cap
                                        ↓
                   composePulseDirectDecisions → TradeDecisionSet

No external LLM process is needed. Entry candidates are extracted directly from Pulse's structured sections, sorted by monthlyReturn = edge / monthsToResolution, the top 4 are taken, and total staking in a single round is capped at 20% of bankroll.

Spawns an external process (Codex / OpenClaw / Claude Code CLI), passes Pulse + position context to the LLM, and parses stdout into a TradeDecisionSet. Still functional, but no longer the default path.

Core principle: risk controls do not rely on prompt engineering — they are service-layer hard rules. No matter which provider or decision strategy runs upstream, anything entering the orchestrator / executor pipeline is bound by the same constraints: Agent reasoning errors, bad data, and model overreach cannot bypass them. Three tiers of defence plus Pulse-level preflight checks trim everything before orders go out; individual positions that cross the line are force-stopped; a system-wide drawdown breach halts trading immediately, and only an admin can resume (fail-closed).

• Must come from a real fetch_markets.py fetch — no mock fallback
• Stale Pulse (>120 minutes) or too few candidates (<1) is treated as a risk state; no new open in that round
• open actions' token_id must originate from the Pulse candidate set

Full rules: docs/risk-controls.md.

Full template: .env.example

Organised into four groups:

If your Polymarket credentials live in an adjacent repo, you can set ENV_FILE=../pm-PlaceOrder/.env.aizen. For real-money testing, stick to a dedicated .env.live-test.

The Polymarket order path supports two signer modes.

Private-key mode needs:

• WALLET_PROVIDER=private-key
• PRIVATE_KEY — wallet private key (prefer a Polymarket proxy wallet over your main wallet)
• FUNDER_ADDRESS — the Polymarket proxy wallet address (the one that holds collateral)
• SIGNATURE_TYPE — 0 or 1, depending on wallet type
• CHAIN_ID — 137 (Polygon mainnet)

OKX Agentic Wallet / OnchainOS mode needs:

• WALLET_PROVIDER=onchainos (okx-agentic remains a compatibility alias)
• ONCHAINOS_BIN — defaults to onchainos
• FUNDER_ADDRESS — Polymarket deposit/proxy wallet address with collateral/allowance
• SIGNATURE_TYPE=3 — deposit wallet / POLY_1271
• CHAIN_ID=137

Keep these in separate per-purpose files, none of which are committed:

• .env.live-test — real-money live-trading credentials
• .env.<wallet-name> (e.g. .env.pizza) — split by wallet name to avoid mixing them up

Every preflight prints the current ENV_FILE, wallet address, and collateral amount. If any of them do not match, it aborts immediately so you never accidentally trade on the wrong wallet.

vendor/manifest.json pins the following external repos to specific commits:

Run pnpm vendor:sync to sync them into vendor/repos/. A plain pnpm build does not need vendor, but the pulse / trial / live paths must sync first.

All run artifacts are written to runtime-artifacts/ (already in .gitignore), rooted at ARTIFACT_STORAGE_ROOT.

Failure archives (per the AGENTS convention) go to run-error/ with the failing stage, core context, root-cause summary, and next-step command.

• AGENTS.md / CLAUDE.md — Agent collaboration conventions (required reading)
• docs/risk-controls.md — Full write-up of the hard risk rules
• docs/delta-pm-operations.md — Delta PM operations manual
• .env.example — Environment variable template
• docs/diagrams/onboarding-architecture.md — Architecture diagram + module map
• docs/diagrams/trading-modes-flowchart.md — Trading mode flowchart
• docs/diagrams/dev-reference.md — Command cheatsheet / dependency matrix / deployment shapes
• docs/internal/plan/2026-06-09-world-cup-special-plan.md — World Cup forecasting product plan

Historical handoff docs and one-off exploration notes are archived under docs/archive/README.md.