Heimdall

A quant portfolio management terminal: build strategies with an alpha layer on top of a beta (broad market index) layer, backtest them fast on real Norgate data, mix multiple strategies together, optimize allocations under risk constraints, and see the same code path run live.

Named for the Norse watchman who sees across all realms — the goal is full visibility into a strategy, both ex ante (forecasted risk/exposures before the fact) and ex post (realized performance attribution after the fact).

Status

Functional. Backend engine, data pipeline, and all five frontend sections (Overview, Strategy Console, Portfolio Builder, Risk & Optimization, Live Trading) are wired up and verified against real Norgate data. Live Trading connects to Interactive Brokers paper accounts only — see Roadmap for what's still open.

Quickstart

Prerequisites: Python 3.12+, Node 22+, and the Norgate Data Updater running locally (Windows only — see Data & the Norgate cache). A Norgate Data subscription is required for real price/index data — see that section for what runs without one.

# Backend
cd backend
python -m venv .venv
.venv/Scripts/pip install -e ".[dev]"     # .venv/bin/pip on macOS/Linux
cp .env.example .env                       # optional — only needed for live trading
.venv/Scripts/python -m uvicorn app.main:app --port 8000

# Frontend (separate terminal)
cd frontend
npm install
npm run dev

Open http://localhost:3000. The frontend expects the backend at http://localhost:8000/api by default — override with NEXT_PUBLIC_API_URL if needed.

Or run the whole stack in Docker (backend + frontend + the IB Gateway in paper mode; Linux containers, no host venv/node/Gateway needed):

cp .env.example .env        # optional: IBKR credentials for the containerized Gateway
docker compose up --build   # backend on :8000, UI on :3000, Gateway console on :6080

/app/data (settings, saved strategies/portfolios, the Norgate Parquet cache, the blotter) lives in a native Docker volume, not a host bind mount — every read/write stays inside the Linux VM instead of crossing the Windows↔container filesystem bridge, which measured 300-800x slower per file access than native (see docs/ENGINE_ISSUES.md §12). Norgate sync runs via the container's own daily auto-sync (talks to NDU over HTTP — see below); a host-side scripts/sync_norgate_cache.py run is no longer a supported path for the Docker setup specifically (run it inside the container instead: docker compose exec backend python scripts/sync_norgate_cache.py ...), though running the whole backend natively (not in Docker, per the Quickstart above) still works exactly as before — there's no bridge to worry about outside Docker. The MCP server runs inside the container too — see docs/MCP.md for how a local agent registers it. The IB Gateway now runs inside the stack (paper mode) — see Live trading in Docker below for how login/2FA works.

Everything above gets you backtesting, strategy building, portfolio construction, and risk/ optimization. Two pieces are optional and only matter if you need them:

• Live paper trading needs an Interactive Brokers Gateway — either the one bundled in the Docker stack (see below) or TWS/Gateway running on the host — see backend/README.md.
• A real historical risk-free rate series (scripts/sync_risk_free_rates.py) is a nice-to- have one-time backfill from Norgate (%3MTCM); everything falls back to a flat Settings value without it.

Live trading in Docker (login & 2FA)

docker compose up starts an ibgateway service: IB Gateway (paper) inside a container via IBC (the standard automation layer). You don't log into the container interactively — the pieces are:

1. Credentials — put IBKR_USERNAME / IBKR_PASSWORD (your IBKR paper-account login) in the project-root .env (copy .env.example). IBC submits them automatically at container start, so the Gateway logs itself in.
2. The 2FA prompt — open http://localhost:6080 (the novnc browser console; password = IBKR_VNC_PASSWORD from .env) and you'll see the Gateway's desktop. When IBKR pushes the two-factor prompt, click Accept there. That's the whole login flow: .env creds + one click in the browser console.
3. Key store / settings persist in data/ibgateway (mounted, gitignored). If your account uses IBKR's Secure Login System, download the .jks from IBKR Client Portal → Settings → Secure Login System and drop it into that folder before first start.

With empty credentials the container still boots — it just sits at the login screen, which you can log into manually through the same console. The backend is pre-wired to the container Gateway (HEIMDALL_IBKR_HOST=ibgateway, port 4004). Host-installed TWS/Gateway remains a supported alternative — see backend/README.md. Details, port map, and caveats (trusted IPs, read-only 2FA) are in docs/FUTURE_IDEAS.md.

What's here

Section	Route	What it does
Overview	/	Dashboard: AUM/exposure/revolver header stats, gross-vs-net-vs-benchmark chart, cost waterfall, NAV/financing chart, net-of-cost analytics, saved strategies
Strategy Console	/strategies	Write an on_rebalance strategy in Python — against a fixed symbol list or a dynamic index universe — run it against real Norgate data, and save it to the strategy library. See docs/WRITING_STRATEGIES.md for the full guide: the contract, what's in scope, performance dos/don'ts, and worked examples
Portfolio Builder	/portfolio	Mix multiple strategy sleeves (pick saved strategies or write ad-hoc code) with an allocation dial each, see the blended portfolio net of costs vs. benchmark
Risk & Optimization	/risk	Bounds-constrained mean-variance/risk-budgeted/Treynor-Black optimizer across sleeves, with risk-profile presets, an efficient frontier chart, stress testing, and net-of-cost performance
Live Trading	/live	Connect to an IBKR paper account, see current holdings vs. target weights, and place holdings-aware rebalancing orders through the same strategy/portfolio code path used in backtests
Settings	/settings	Universe, benchmark, rebalance timing, leverage cap, mode, and the cost/financing model defaults (AUM, commission, slippage, MER, financing rate/spread, collateral haircut)
MCP (agent API)	python -m app.mcp	Model Context Protocol server for local AI assistants: list strategies/portfolios, check data coverage, backtest strategies or saved portfolios, optimize, pull tear sheets — read-only by default (stdio); write tools (save/update/delete strategies, update settings) opt-in behind HEIMDALL_MCP_ENABLE_WRITE_TOOLS with an append-only audit log. See docs/MCP.md

Cost & financing model

Every backtest is run net of a real institutional cost model, not just gross returns:

• Trading costs: commission + slippage, charged in bps of turnover at each rebalance.
• MER (management fee): accrued daily against NAV, same convention as a mutual fund.
• Financing: any gross exposure beyond 1.0x NAV is treated as financed via a modeled revolver, collateralized at 102% — the standard US equity securities-lending margin and the Canadian mutual-fund regulatory minimum for repo collateral — at a SOFR-proxy base rate plus a spread.
• TER (total expense ratio) is defined here as MER + realized trading expense ratio + realized financing expense ratio, annualized — deliberately explicit since the term is used inconsistently across the industry.

All of it is configurable in Settings (with sensible institutional defaults) and overridable per API call. See docs/ARCHITECTURE.md for the full mechanics and the research behind the default numbers.

Dynamic (point-in-time) universe

A strategy can target a fixed symbol list, or a Norgate index/watchlist — in which case the engine restricts each day's snapshot to that day's actual point-in-time constituents (survivorship-bias-free), not today's list. Pick this in the Strategy Console, Portfolio Builder, or Risk & Optimization via the "Index universe" toggle.

Strategy Library

Save a strategy once in the Strategy Console (name, code, and either fixed symbols or a dynamic universe), then pick any number of saved strategies as sleeves in Portfolio Builder or Risk & Optimization — no more re-pasting code per sleeve. Backed by GET/POST/PUT/DELETE /api/strategy-library.

See docs/API.md for the full endpoint reference and docs/ARCHITECTURE.md for the design decisions behind the engine.

Why

Most backtesting tools either fake speed by ignoring realistic execution timing, or bolt analytics on as an afterthought. Heimdall is built around a few constraints instead:

• No lookahead by construction. Norgate provides EOD data only. A signal computed on the close of day N can only act at the close (MOC/auction) of day N+1, and only earns a return starting N+1 → N+2. This is enforced in BacktestEngine.run, not just documented as a convention.
• Alpha is measured against beta, always. Every strategy return stream is decomposed against its benchmark: active return, tracking error, information ratio, hit rate, drawdown — not just standalone Sharpe/CAGR. See ex_post.py.
• One code path, backtest and live. A strategy is a function from data → target weights (the Strategy protocol in strategy.py). Backtest and live execution are swappable adapters underneath that same function, not two separate implementations that can drift apart.

Project structure

backend/
  app/
    data/       # DataFeed protocol + adapters: NorgateFeed (live), DuckDBFeed (Parquet cache),
                #   CompositeFeed (cache-first w/ live fallback)
    engine/     # Strategy protocol, BacktestEngine (N -> N+1 timing + cost/NAV walk),
                #   strategy code runner, sleeves.py (multi-sleeve blending)
    analytics/  # ex_post.py (realized metrics), ex_ante.py (Ledoit-Wolf shrinkage covariance +
                #   forecasted TE/VaR/CVaR), optimizer.py (mean-variance, risk-budgeted,
                #   Michaud resampling), fundamental_law.py, attribution.py (Brinson-Fachler),
                #   crowding.py, liquidity.py, stress_testing.py, treynor_black.py,
                #   costs.py (CostModel: commission/slippage/MER/financing/collateral)
    live/       # IBKR paper-trading execution adapter (ibkr_client.py wraps ib_async),
                #   holdings-aware rebalancing, client-id pooling
    reporting/  # Tear-sheet export (PDF via reportlab/matplotlib, Excel via openpyxl)
    api/        # FastAPI routes — one module per resource, schemas.py for shared response shapes
    core/       # Settings persistence, strategy/portfolio library persistence, blotter store,
                #   app config
  scripts/      # Norgate smoke tests + the cache sync ETL job
  tests/        # pytest suite (synthetic data, no live Norgate connection required)
frontend/
  src/app/
    page.tsx, strategies/, portfolio/, risk/, live/, settings/   # the six sections
    components/    # NavShell, PerformanceChart & MultiSeriesChart (lightweight-charts),
                   #   AllocationDial, FrontierChart, WaterfallChart, StrategyPicker, StatTile
  src/lib/api.ts   # fetch helpers, reads NEXT_PUBLIC_API_URL
docs/
  ARCHITECTURE.md      # design decisions, trust model, timing model, cost model
  API.md               # endpoint reference
  WRITING_STRATEGIES.md  # how to write on_rebalance code: contract, scope, performance, examples

Data & the Norgate cache

The norgatedata Python library talks to the Norgate Data Updater over local IPC — that updater only runs on Windows, and repeated IPC calls are too slow for a backtest loop. Heimdall splits acquisition from consumption.

Windows: scripts/setup/download_norgate_data_updater.ps1 downloads the Data Updater installer and launches it (skips the download if it's already installed, checked via the Windows registry; pass -Force to reinstall anyway). Installing needs no login — a paid, active Norgate Data subscription is only needed afterward, to actually sync real data through it.

• Acquisition (Windows-only): backend/scripts/sync_norgate_cache.py pulls EOD bars and index membership via norgatedata and writes them to Parquet under backend/data/cache/. Run it once per close — matches the EOD-only rebalance timing anyway.
• Consumption (portable): DuckDBFeed reads that Parquet cache as one bulk multi-symbol SQL scan; CompositeFeed (what every API route actually uses, see data/default_feed.py) tries the cache first and falls back to live Norgate on a miss. This is what lets the engine/backend run on Linux — only the sync step needs Windows.

cd backend
.venv/Scripts/python scripts/sync_norgate_cache.py --symbols SPY AAPL MSFT \
  --watchlist "Dow Jones Industrial Average Current & Past" \
  --index "Dow Jones Industrial Average"

Testing

cd backend
.venv/Scripts/python -m pytest -q          # unit tests, synthetic data, no Norgate required
.venv/Scripts/python -m ruff check .        # lint

cd frontend
npx eslint .                                 # lint
npx tsc --noEmit                             # typecheck

Roadmap

See docs/ROADMAP.md for the full history of what's shipped and docs/ENGINE_ISSUES.md / docs/FUTURE_IDEAS.md for open work. As of 2026-08-08, the audit's Phase E performance pass is done (see docs/AUDIT_PLAN.md), the MCP server ships with all three phases — read-only tools, write tools (HEIMDALL_MCP_ENABLE_WRITE_TOOLS, append-only audit log), and live gating (HEIMDALL_MCP_ENABLE_LIVE_TOOLS, two-step rebalance confirmation) — with a global background-job concurrency cap, and the Phase F path-traversal guards + auto-sync filesystem lock are closed. What remains open: rate limiting (Phase F — deliberately deferred while localhost-only), plus:

• Full process isolation for strategy code — the execution-timeout work above bounds slow code (a wall-clock deadline between rebalance days, plus a background-job runtime limit), but neither can forcibly kill a single on_rebalance call that itself hangs (an actual infinite loop). That needs subprocess-based execution with a real OS-level kill signal — a bigger architectural change, only worth it if this is ever exposed beyond a single trusted operator (see docs/ARCHITECTURE.md).
• A GICS fundamental factor model as a v2 beyond the PCA statistical factor model, if PCA's purely statistical decomposition proves insufficient in practice (see docs/FUTURE_IDEAS.md #5).

Disclaimer

This software is provided as-is, with no warranty of any kind, and no guarantee of fitness for any purpose — see the Apache 2.0 license's warranty and liability terms below, which govern in full. It is a research/engineering tool, not a broker, advisor, or regulated financial product. Nothing in this repository — code, documentation, sample strategies, backtest output, or Live Trading functionality — constitutes investment, tax, legal, or other professional advice in any jurisdiction, and none of it should be relied on to make a real financial decision. Backtested and simulated results do not represent actual trading and have material limitations; past performance (real or simulated) is not indicative of future results. Live Trading only ever connects to Interactive Brokers paper-trading ports (see app/live/ibkr_client.py's PAPER_PORTS guard) — using it against real capital is outside its designed and tested scope and is done entirely at your own risk. The authors and contributors accept no liability for any loss, financial or otherwise, arising from the use of this software.

Norgate Data is an independent, third-party data vendor and is used here only because it was the author's personal pick for a survivorship-bias-free EOD data source — this project is not sponsored by, affiliated with, or endorsed by Norgate Investor Services Pty Ltd. "Norgate Data" and "Norgate" are trademarks of their respective owner, referenced here solely to describe interoperability. No Norgate data is bundled with or redistributed by this repository (see Data & the Norgate cache) — a subscription and the Norgate Data Updater are required to pull real data yourself, subject to Norgate's own terms of service.

License

Apache 2.0 — see LICENSE. Note that a Norgate Data subscription (and its Windows-only Data Updater) is required to pull real price/index data; that's a separate paid third-party service, not covered by this repo's license.