feat(market): workbench page + market-data HTTP consolidation#133
Merged
luokerenx4 merged 11 commits intomasterfrom Apr 20, 2026
Merged
feat(market): workbench page + market-data HTTP consolidation#133luokerenx4 merged 11 commits intomasterfrom
luokerenx4 merged 11 commits intomasterfrom
Conversation
Turns the Diary page into a dashboard: heartbeat feed stays as the main column (now left-aligned instead of centered), with a right-side Brain panel surfacing Alice's current frontal lobe and emotion. Each panel has a click-to-expand dialog that lists the full commit history as text snapshots (no diff — Alice rewrites the whole field each time, so diff would be noise). On narrow screens the sidebar flattens above the feed and both panels default to collapsed. Data comes from a new read-only /api/brain/state endpoint that reads data/brain/commit.json directly, so brain changes triggered inside chat sessions show up alongside heartbeat-driven ones.
…lf-referring notes The emotion module was write-only in practice — no downstream code read `emotion` to shape decisions, and because Agent SDK routes tools through tool-search once the catalog grows, the AI never reached for updateEmotion on its own. A field that nobody reads and a tool nobody calls rots into dead architecture. Meanwhile the frontal lobe was poisoning later rounds: its tool descriptions told her to store market views, portfolio health, and predictions — all world-referring content that goes stale between rounds but gets injected back into the system prompt as "Current Brain State" like it's ground truth. She'd then reason on top of her own yesterday's forecast as if it were an established fact. The fix is a principle, not more schema: the frontal lobe accepts only self-referring content (commitments, attention targets, self-constraints, operating stances) — anything the world cannot falsify. Facts, predictions, and narrative are banned because they rot; the rule-shaped content that remains stays valid until she chooses to change it. Changes: - Remove emotion from BrainState / BrainCommitType / Brain methods; restore() filters legacy emotion commits so existing commit.json still loads - Rewrite updateFrontalLobe description with explicit ✅/❌ lists and a fully self-referring example (replacing the old market-narrative one) - Inject as "Notes you wrote to yourself / (written Nh ago)" instead of "Current Brain State" — the time-distance cue breaks the authoritative framing that made stale notes feel like present truth - Add "Before You Check" section to default heartbeat watch list asking her to sense market mood fresh each round (regeneration, not storage) - Strip Emotion panel from Diary sidebar; Frontal Lobe subtitle now shows "N versions · written Nh ago" so the user sees the same staleness cue Alice does
ChatPage: during streaming, scrollIntoView fires every render and the async `scroll` event raced it — user wheel/touch got overwritten before userScrolledUp could flip. Added synchronous wheel/touchmove listeners that lock the flag before the next auto-scroll can run, and lowered the at-bottom threshold from 80 to 20 for finer detection. PushApprovalPanel: added fmtNum helper. Strings pass through; numbers go through toFixed(8) + trim to avoid both IEEE 754 noise leaking into the display and toLocaleString's default 3-decimal truncation that turned crypto-scale quantities into '0'. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The 5 price-class fields on the Order class (lmtPrice, auxPrice, trailStopPrice, trailingPercent, cashQty) were number/UNSET_DOUBLE while totalQuantity and filledQuantity were already Decimal. Precision was dropped at the field slot itself — before reaching the wire — so no UI-layer formatting could recover it. Extends the existing Decimal pattern to those 5 fields so the whole chain (Zod → UTA stage → git staging → JSON roundtrip → UI) carries decimal strings end-to-end, matching totalQuantity. - packages/ibkr: Order fields switch to Decimal = UNSET_DECIMAL. comm.ts's makeField/makeFieldHandleEmpty grow a Decimal branch that uses .toFixed() (not .toString() — the latter emits scientific notation for small values and would break the TWS text wire). The 5 decodeFloat calls in order-decoder swap to decodeDecimal; the protobuf decoder wraps incoming numbers. - domain: UTA stage* wraps Decimal(String(x)); TradingGit.rehydrateOrder extends to all 5 fields (legacy number-typed persisted state still rehydrates cleanly). Guards switch to .equals(UNSET_DECIMAL). - tool: Zod widens to union([number, string]) + refine(positive) so the AI can send high-precision strings when it needs to. summarizeOrder emits Decimal.toFixed() strings. - brokers: Alpaca sends decimal strings (its REST accepts them; preferred over .toNumber() to avoid IEEE 754 at the SDK boundary). CCXT SDK takes number, so .toNumber() at the call site. IBKR is pass-through since the wire lib now handles Decimal natively. MockBroker updated. - tests: +13 packages/ibkr unit tests locking wire encode/decode round-trip incl. IEEE 754 traps (0.1 + 0.2), satoshi scale, UNSET sentinel. +3 UTA e2e precision tests. +3 paper-TWS e2e tests in packages/ibkr/tests/e2e verifying the full round-trip against a real IB server. The paper-TWS e2e caught a bug the unit layer missed: after the slot type switch, client/orders.ts::placeOrder sent cashQty via makeField (no UNSET handling), so an unset Decimal cashQty encoded as its sentinel string (~1.7e38) got interpreted by TWS as a real cashQty → error 10244. Switched to makeFieldHandleEmpty. This is exactly the kind of protocol-layer semantic drift that justifies paper-TWS e2e. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…orders One-shot script. Pre-Decimal migration, Order's price-class fields defaulted to UNSET_DOUBLE = Number.MAX_VALUE and got JSON-serialised as 1.7976931348623157e+308 into data/trading/<account>/commit.json. Post-migration those fields are Decimal with a different UNSET_DECIMAL (~1.7e38), so the old number sentinel rehydrates into a valid ~1.7e308 Decimal that slips past every `.equals(UNSET_DECIMAL)` check and surfaces as "XLE BUY $179769..." in the Recent Trades panel. Cleaned 47 fields across 4 account commit logs (auxPrice=9, trailStopPrice=12, trailingPercent=12, cashQty=10, lmtPrice=4). Idempotent. Run is a no-op if disk is already clean. Keeping it in scripts/ rather than inlining a runtime compat shim in rehydrateOrder — legacy translation at every load would be a permanent tax for a one-time transition. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collapse the three parallel market-data entry points (in-process executor, embedded 6901 server, and potential web-connector duplication) into a single HTTP surface mounted on Alice's main port at /api/market-data-v1/*. - opentypebb mountToHono gains defaultCredentials fallback so server-side provider keys flow through without forcing UI to send X-OpenBB-Credentials - src/server/opentypebb.ts becomes a pure mountOpenTypeBB helper; the standalone server Plugin is removed along with its reconnect branch - marketData.apiServer config (and the related Embedded API Server toggle in MarketDataPage) is gone — default is simply correct, no switch - Alice tools and UI now share one QueryExecutor via ctx.bbEngine Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…onfig Step 1 of the Market workbench page. Scope: sidebar entry, search box, and price candlestick. Backend: - Pull the cross-asset-class heuristic search out of tool/market.ts into a dedicated domain service (aggregate-search.ts). The AI tool and the new /api/market/search HTTP route both call it — one source of truth. - Teach opentypebb's mountToHono to accept a resolveProvider callback, and use it in mountOpenTypeBB to auto-inject the per-asset-class provider from config.marketData.providers when ?provider= is omitted. UI and Alice share provider configuration. Frontend: - New /market page with a debounced (300ms) SearchBox — results show assetClass badges (equity / crypto / currency / commodity) and support arrow-key navigation. - KlinePanel built on TradingView lightweight-charts: candlestick + volume histogram pane, 1M/3M/1Y/5Y/All timeframes. Dispatches historical URL by assetClass. Commodity shows a placeholder until canonical-symbol resolution lands in step 2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Server-side date filtering was unreliable — FMP's /stable/historical-price-eod/full endpoint ignores start_date/end_date and always returns its full window, so 1M/3M/1Y buttons were all showing identical data. Switch to a single fetch per symbol and reshape the timeframe buttons into a client-side visible-range zoom on lightweight-charts' timeScale. Timeframe switching is now instant and costs zero API calls. Also thread the upstream provider through the response into the header as a small badge alongside bar count and date span, so users can see which source the chart is drawn from (fmp vs yfinance vs ...). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three bugs in the opentypebb HTTP layer that surfaced while building the Market page: 1. FMP's historical OHLC endpoints take `from`/`to`, not the canonical `start_date`/`end_date` from the standard query schema. The FMP helper passed the canonical names through unchanged and FMP silently returned its default 5-year window — so any timeframe filtering on equity/crypto/currency/index historical + commodity-spot-price was a no-op. Rename the keys inside getHistoricalOhlc before serializing. 2. amakeRequest embedded the request URL verbatim in OpenBBError messages, which for FMP (and any provider using apikey-in-querystring auth) leaked the credential into logs and user-visible errors. Redact a standard set of secret query-param names — apikey, api_key, token, access_token, key, secret, password — to "***" before building any message. 3. amakeRequest caught fetch-layer failures and threw "Request failed: URL" with the original error tucked into a non-rendered `.original` field, so logs showed the URL but not WHY — DNS, TLS, socket reset all looked identical. Include a short description of the cause in the thrown message text. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Now that FMP honors date ranges, swap the one-shot full-history fetch for on-demand pulls keyed to (symbol, interval, timeframe) — matching how Alice's own K-line calculator tools fetch. Switching either control refetches, rather than client-side zooming over a fixed 5y payload. New interval selector exposes 1m / 5m / 1h / 1d (1m is premium on some FMP tiers; the error surfaces clearly when hit). Added shorter 1D / 5D timeframes so intraday intervals have a sensible range partner. UX: the two button groups were previously unlabeled and wrapping into two rows, reading as a puzzle to non-trader users. Labeled them Interval and Range with muted caps, added gap between groups, and hover tooltips explaining what each group controls. Intraday intervals flip the time axis to show hours/minutes; daily stays date-only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
aggregateSymbolSearch returned [equity..., crypto..., currency..., commodity] in that order, so queries whose best match was a commodity (e.g. "gold", whose canonical id is exactly "gold") were drowned in 20 equity hits before the user ever saw it. Same pattern hid "xau" → gold alias behind GOLD FIELDS and friends. Score each result against the query — exact match on symbol/id/name, alias equality, prefix, word-boundary match, substring — and sort globally. Ties preserve upstream order, so within a score tier equity SEC ordering stays as a sensible fallback. Regex uses a word-boundary check so "gold" no longer ranks GOLDMAN SACHS above SPDR GOLD TRUST. The same function backs both the AI tool and the /api/market/search route, so the ranking fix lands in both surfaces. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR batches several related improvements on the dev branch:
Market workbench (primary — new user-facing surface)
/marketpage with an aggregated symbol search and TradingView lightweight-charts K-line.goldnow put commoditygoldat the top instead of being drowned by 20 equity hits. Asset-class badge on each result; 300ms debounce; arrow-key navigation.1m / 5m / 1h / 1d) × range (1D / 5D / 1M / 3M / 1Y / 5Y / All). Intraday flips the time axis to hours/minutes. Provider badge + bar count + date span shown in the header. Commodity candles TBD.Market-data HTTP consolidation
/api/market-data-v1/*. One executor, one CORS policy, one place for defaults.mountToHonogainsdefaultCredentialsand aresolveProvidercallback; Alice wiresconfig.marketData.providersin, so the UI never has to pass?provider=— server-side config is authoritative.opentypebb bug fixes (surfaced while wiring the UI)
from/toand were silently ignoringstart_date/end_date, returning a fixed 5-year window regardless of query. Fetcher now renames the keys before serialising, which restores date-range filtering across equity/crypto/currency/index historical + commodity spot.amakeRequestredacts secret query params (apikey/token/etc.) before embedding URLs in error messages, and includes the caught cause's description so logs distinguish DNS / TLS / socket failures.Also riding along (previously committed to dev)
refactor(trading): Order price fields migrated to Decimal end-to-end, with a legacy UNSET_DOUBLE sentinel stripper for persisted orders.fix(ui): chat scroll no longer sticks during streaming; Push panel shows correct precision.refactor(brain)+feat(diary): frontal lobe constrained to self-referring notes (emotion dimension removed); Diary gets a Brain sidebar showing frontal lobe + emotion state.Test plan
npx tsc --noEmitcleanpnpm test— 1088 tests / 56 files passcurl /api/market/search?query=gold→ commoditygoldat position 0curl /api/market/search?query=xau→ commodity gold via alias matchcurl /api/market-data-v1/equity/price/historical?symbol=AAPL(no?provider=) → returns data using configured FMP defaultcurl /api/market-data-v1/equity/price/historical?symbol=AAPL&interval=1d&start_date=2025-03-21&end_date=2025-04-21→ 21 bars (previously returned 1254 due to FMP param bug)/market: search apple → click AAPL → K-line renders; switching interval + range refetches🤖 Generated with Claude Code