OpenEval v0.1.2

OpenEval v0.1.2 is a performance, stability, and dashboard-UX release. At ~1,300 local sessions, warm Collection loads drop from ~2.3 s to ~0.15 s, idle Live polls shrink from ~2.3 MB to 71 bytes, and the Collection table can now browse the entire session history instead of the newest 80.

Performance

• One shared full-history parse + fingerprint cache — Collection, home, and Timeline used to run three independent full-history parses per page load; they now share a single pass, and a corpus fingerprint (per-file path/mtime/size from the discovery walk) serves warm loads without re-parsing at all. Warm /collection ~2.34 s → ~0.15 s, home ~1.63 s → ~0.1 s; Rescan revalidates the fingerprint instead of forcing a recompute. (#32)
• Live parser fast paths — strict ISO-timestamp fast path, single-pass token sets, cursor line scanning: claude-format corpora parse 11–22% faster, codex 7–17%, with byte-identical output (116/116 sha256-verified corpora + 700k-string fuzz). (#26)
• Scan pipeline — archived-session merge reads went from 144.9 ms to 8.1 ms (17.8×) via a paths-first two-step read; prepared-statement cache in the SQLite layer; in-process session cache upgraded to a 4,000-entry LRU; one stat per file and one directory walk per pass, now shared between discovery and the recompute path. (#26, #35)
• aggregate() single pass — fourteen full-array filter passes folded into the main loop plus a per-call pricing memo; warm scan path 7–16% faster. (#28)
• Pricing memo at the source — model→rate resolution memoized inside lib/pricing.ts (~8×: 26.3 → 3.2 ms per 100k lookups), covering every hot caller. (#34)
• Timeline & judge insights — binary-search marker windows and sliding-window medians: impact stage 43–63 ms → ~4 ms, metric series ~10×, judge window sampling 171 ms → 1.9 ms on a 50k-point stress corpus. (#31, #33)
• Live polling — a content signature lets unchanged 10-second polls return a 71-byte response instead of the full ~2.3 MB payload; per-session reference identity is preserved across polls so unchanged rows skip re-rendering, and the redundant mount poll is gone. (#30)

Stability

• Transient filesystem errors during a parse are no longer cached as permanent null tombstones — previously a session could silently vanish from every dashboard until its file changed. (#26)
• Torn or garbage cache rows now read as misses instead of crashing consumers, and a corrupt live-cache DB is renamed aside and rebuilt automatically. (#26)
• Four flaky tests pinned (week-boundary clock, killed-run settings poison, pid-reuse workdirs, shared test-DB races) and the live API tests are now isolated from the operator's home directories. (#26, #36)

Dashboard UX

• Loading skeletons for every primary route — navigation no longer freezes on the previous page while a scan runs — and error boundaries for /collection and /collection/timeline, which previously rendered silent zeros on scan failure. (#27)
• Collection sessions table: load-more through the full history (limit-aware retention replaces the old per-source 100-session cap — all ~1,300 sessions reachable), sort pills, harness/model filters, a prominent error banner, and a "scanned Xs ago" freshness indicator. (#29, #35)
• Live: "updated Xs ago" indicator and flicker-free polling. (#30)

Verification

Every performance change shipped with byte-identical output-equivalence proof (sha256 over golden corpora, scan-layer hash gates, differential fuzzing), the full node:test suite (297 tests), zero-warning lint, and end-to-end route checks against a real ~1,300-session corpus.

Known note: the bundled OpenRouter pricing catalog has drifted for three models since 2026-07-15 (glm-5.2, deepseek-v4-flash, kimi-k2.7-code). Refreshing rates changes inferred cost estimates, so it is deferred to a deliberate follow-up rather than bundled into equivalence-gated changes.

Full changelog: v0.1.1...v0.1.2

OpenEval v0.1.1

OpenEval v0.1.1 bundles the post-v0.1.0 bugfix line with the release-readiness tranche.

Highlights:

• More accurate transcript, token, model, pricing, and cost provenance handling.
• Stronger harness probing, live-session rendering, artifact boundaries, and tmux capture reliability.
• Report bundles, local judge settings, theme/mobile fixes, and regression coverage.
• 19 shipped evaluation cases with selftest and accuracy-audit coverage.

The launch film, poster, and social preview are included as release assets.

OpenEval v0.1.0

OpenEval v0.1.0

OpenEval is a free, local-first evaluation dashboard for agent CLIs. It turns runs, live traces, grader evidence, cost, performance, and accuracy data into an inspectable record you can use to decide what to ship.

Highlights

• Run repeatable evaluation cases across Claude Code, Codex, ncode, and descriptor-defined harnesses.
• Combine deterministic checks, trace contracts, visual evidence, optional LLM judges, and manual review.
• Inspect live sessions with honest measured, inferred, missing, and malformed provenance.
• Compare runs by quality, cost, tokens, duration, and case-level regressions.
• Audit oracle coverage, known-bad rejection, evidence tiers, and weak proof surfaces.
• Keep operator history, transcripts, workdirs, and SQLite data local by default.

Launch film

This release includes the final 29.5-second OpenEval launch film, a video poster, and a feed-readable 1280×640 social preview as downloadable release assets. Production source stays outside the product repository.

The application was developed with Noumena Code, Claude Code (including Fable 5 and Sonnet 5), and OpenAI Codex. Fable 5 contributed substantially to OpenEval itself. GPT-5.6 Sol (High) led the launch-film edit, composition, visual QA, and release pass, with additional iteration support from GPT-5.6 Luna (xhigh).

Production used Codex, ImageGen, TouchDesigner, HyperFrames, GSAP, CDP Recorder, and FFmpeg.

Huge thanks to @KingBootoshi from Right to Intelligence.

Release assets

• Launch film
• Video poster
• Social preview
• Changelog: CHANGELOG.md
• License: MIT