v0.0.10

No Developer ID or Authenticode certificate and no notarization yet.

macOS: Apple Silicon (M1 or later) only; Intel Macs are not supported. The app is ad-hoc signed, so the
first open is blocked. Move Factr-I.app to Applications, open it once and close the dialog, then open
System Settings > Privacy & Security and click Open Anyway (macOS 15 and later: right-click > Open no
longer works).

Windows: x64 only, unsigned; SmartScreen warns (More info > Run anyway).

Linux: x64 AppImage, unsigned. Run chmod +x Factr-I-*.AppImage. On Ubuntu 24.04 install FUSE 2
(sudo apt install libfuse2t64) and start it with ./Factr-I-*.AppImage --no-sandbox (AppArmor blocks
the Chromium sandbox for AppImages).

Full Changelog: v0.0.9...v0.0.10

v0.0.9

No Developer ID or Authenticode certificate and no notarization yet.

macOS: Apple Silicon (M1 or later) only; Intel Macs are not supported. The app is ad-hoc signed, so the
first open is blocked. Move Factr-I.app to Applications, open it once and close the dialog, then open
System Settings > Privacy & Security and click Open Anyway (macOS 15 and later: right-click > Open no
longer works).

Windows: x64 only, unsigned; SmartScreen warns (More info > Run anyway).

Linux: x64 AppImage, unsigned. Run chmod +x Factr-I-*.AppImage. On Ubuntu 24.04 install FUSE 2
(sudo apt install libfuse2t64) and start it with ./Factr-I-*.AppImage --no-sandbox (AppArmor blocks
the Chromium sandbox for AppImages).

What's Changed

• docs: add Windows build guide by @hir0-pixel in #3
• docs(readme): link Factr-I to Refactr and add the pilot CTA by @zordhalo in #4
• ChatGPT sign-in: browser approval, no code to type by @hir0-pixel in #6

New Contributors

• @hir0-pixel made their first contribution in #3
• @zordhalo made their first contribution in #4

Full Changelog: v0.0.8...v0.0.9

v0.0.8

No Developer ID or Authenticode certificate and no notarization yet.

macOS: Apple Silicon (M1 or later) only; Intel Macs are not supported. The app is ad-hoc signed, so the
first open is blocked. Move Factr-I.app to Applications, open it once and close the dialog, then open
System Settings > Privacy & Security and click Open Anyway (macOS 15 and later: right-click > Open no
longer works).

Windows: x64 only, unsigned; SmartScreen warns (More info > Run anyway).

Linux: x64 AppImage, unsigned. Run chmod +x Factr-I-*.AppImage. On Ubuntu 24.04 install FUSE 2
(sudo apt install libfuse2t64) and start it with ./Factr-I-*.AppImage --no-sandbox (AppArmor blocks
the Chromium sandbox for AppImages).

Full Changelog: v0.0.7...v0.0.8

v0.0.7

Known issue: bots (the Bots section) do not work in this release or in 0.0.6. A bot chat stays on "Waking up..." because the app sends every bot to the single default engine, so bot-to-bot messaging cannot run either. A fix is planned for 0.0.8. Ordinary chats, projects, voice dictation and the default Factr-I bot are not affected.

---

No Developer ID or Authenticode certificate and no notarization yet.

macOS: Apple Silicon (M1 or later) only; Intel Macs are not supported. The app is ad-hoc signed, so the
first open is blocked. Move Factr-I.app to Applications, open it once and close the dialog, then open
System Settings > Privacy & Security and click Open Anyway (macOS 15 and later: right-click > Open no
longer works).

Windows: x64 only, unsigned; SmartScreen warns (More info > Run anyway).

Linux: x64 AppImage, unsigned. Run chmod +x Factr-I-*.AppImage. On Ubuntu 24.04 install FUSE 2
(sudo apt install libfuse2t64) and start it with ./Factr-I-*.AppImage --no-sandbox (AppArmor blocks
the Chromium sandbox for AppImages).

Full Changelog: v0.0.6...v0.0.7

v0.0.6

No Developer ID or Authenticode certificate and no notarization yet.

macOS: Apple Silicon (M1 or later) only; Intel Macs are not supported. The app is ad-hoc signed, so the
first open is blocked. Move Factr-I.app to Applications, open it once and close the dialog, then open
System Settings > Privacy & Security and click Open Anyway (macOS 15 and later: right-click > Open no
longer works).

Windows: x64 only, unsigned; SmartScreen warns (More info > Run anyway).

Linux: x64 AppImage, unsigned. Run chmod +x Factr-I-*.AppImage. On Ubuntu 24.04 install FUSE 2
(sudo apt install libfuse2t64) and start it with ./Factr-I-*.AppImage --no-sandbox (AppArmor blocks
the Chromium sandbox for AppImages).

Full Changelog: v0.0.5...v0.0.6

Factr-I 0.0.5

Factr-I 0.0.5

0.0.5 is a fixes release for the desktop app and the engine's start-up, sign-in and provider handling. It fixes the known issue attached to 0.0.4 (first-run sign-in failing in the desktop app) and adds a round of onboarding, approvals, observability and UI fixes. It is not a score release and carries no new benchmark numbers.

Benchmarks

No benchmark gate was re-run on this binary. The engine is unchanged in behaviour from 0.0.4 apart from the gateway, start-up, provider-resolution and config fixes listed below. Benchmark gates for 0.0.5 were verified by in-repo isolation tests and a comparison of the pre-release engine against main (the benchmark start, --provider openai --model gpt-6-luna serve, resolves to the same provider, model, auth state and status, and sends the same request bodies). The 0.0.4 results stand as published with that release; nothing is claimed beyond that for 0.0.5.

What changed since 0.0.4

Sign-in and onboarding

• First-run sign-in works for every method in the desktop app (the 0.0.4 installers failed with "Connected, but Factr-I still cannot resolve a usable provider", a 404 feature_not_requested from the desktop gateway). Onboarding saves the provider you signed in with, never the first row of a model list, and errors read as plain text.
• A fresh install without Ollama no longer resolves to an API-key provider the backend does not know. A saved provider the engine cannot serve no longer stops serve; the engine starts with no provider and says why, and it never serves a different provider instead.
• "Local / custom endpoint" works: a local or private-network endpoint needs no key, a remote one reads its key from model.key_env, model.api_key or OPENAI_COMPAT_API_KEY.
• Saving an OpenAI API key in onboarding no longer fails with "Could not save OpenAI" (an API-key-only user is saved as openai-api, a ChatGPT sign-in stays openai-codex).
• A config.yaml that cannot be read is named in onboarding (file and line) with "Open the folder" and "Back up and start fresh".
• The app no longer borrows (and, by refreshing, logs out) a Claude Code or Codex CLI login by default.

Provider and credential handling

• A custom endpoint's key saved from the app goes to .env and config.yaml holds only a ${VAR} pointer.
• A ChatGPT pick with no valid ChatGPT login and an OpenAI API key present is refused with a clear message instead of billing the key.
• Provider failures read as plain messages (no provider connected, rejected key, missing model, rate limit, server error); the raw text stays under Details.

Approvals

• With the working folder at home or another broad location, a plain delete (rm, unlink, shred, > file) now asks first. "Always allow" is bound to the folder it was given in. Project folders and approvals.mode: off are unchanged.
• The app defaults to and shows Manual approvals and offers only Manual and Off (the engine does not accept smart).

Observability

• A tool turn counts every model request in the run ledger and Usage; silent_failures no longer fires after normal chats; subagent rows close; /usage answers from the ledger.

Desktop and packaging

• Engine restart mid-reply ends the turn with a Retry button instead of spinning; About > Uninstall works in the packaged app and lists exactly what it removes.
• Error toasts no longer pile up; provider retry progress is a brief info toast; Stopped replies keep their partial text.
• UI work: right sidebar tabs (Browser, Files, Terminal, Changes), a lighter dark theme, one greeting in an empty chat, settings and Tools & Keys readability fixes, Esc closes provider setup, Run history entry.
• The macOS app is built with the release hardening (Chromium remote-debugging switches removed, fuses off) and verified with codesign --verify --deep --strict and npm run verify:app. The release build no longer embeds the builder's file paths.

The full list is in docs/CHANGELOG.md.

Known limits

• Installers are unsigned: macOS is ad-hoc signed and not notarized (first open needs System Settings > Privacy & Security > Open Anyway on macOS 15 and later), Windows shows SmartScreen. macOS is Apple Silicon only.
• The Windows and Linux installers are built by the release workflow and have not been launched or tested by the authors. The Windows build in particular is untested with the hardened settings (Electron fuses off), including the terminal pane's ConPTY path. Testing and the QA runs for this release were on macOS.
• API keys saved from the app are stored in a plain .env file with mode 0600. This keeps secrets out of config.yaml but is not encryption.
• Engine-core behaviour (agent loop, providers, prompts, classification and learning) was intentionally left unchanged. Known rough edges there, for example the wording of a malformed-stream error, are not fixed in this release.
• A sign-in against a live ChatGPT account is covered by a manual test on the macOS build, not by automated tests. Other providers' sign-in paths are covered by gateway tests only.
• Wall time, memory and cost were not re-measured for 0.0.5.

Installing

No Developer ID or Authenticode certificate and no notarization yet.

• macOS: Apple Silicon (M1 or later) only; Intel Macs are not supported. The app is ad-hoc signed, so the first open is blocked. Move Factr-I.app to Applications, open it once and close the dialog, then open System Settings > Privacy & Security and click Open Anyway (macOS 15 and later: right-click > Open no longer works).
• Windows: x64 only, unsigned; SmartScreen warns (More info > Run anyway).
• Linux: x64 AppImage, unsigned. Run chmod +x Factr-I-*.AppImage. On Ubuntu 24.04 install FUSE 2 (sudo apt install libfuse2t64) and start it with ./Factr-I-*.AppImage --no-sandbox (AppArmor blocks the Chromium sandbox for AppImages).

Downloads and verification

• Desktop installers (macOS arm64 .dmg and .zip, Windows x64 .exe, Linux x86_64 .AppImage) are built by the release workflow from the tag (commit c1fba38). SHA256SUMS lists every installer attached. Check with shasum -a 256 -c SHA256SUMS.
• The macOS app's bundled engine reports Factr-I 0.0.5 (c1fba38) from factr --version. A binary you build yourself reports the commit you built.
• No separate engine binaries are attached to this release (0.0.4 attached the benchmarked builds; 0.0.5 was not benchmarked).
• Upgrading from 0.0.4: install over it. The app only checks GitHub for updates and never installs one itself.

Factr-I 0.0.4

Factr-I 0.0.4

Known issue in the desktop installers attached to this release: first-run sign-in can fail in the desktop app with "Connected, but Factr-I still cannot resolve a usable provider" (a 404 feature_not_requested from the desktop gateway), and a fresh install without Ollama can resolve to the wrong provider. The engine binaries and the benchmark results are not affected. The fix is on main (development version 0.0.5-dev.1) and will ship in 0.0.5; the 0.0.4 desktop installers are not updated, because this release is the benchmarked build and is kept unchanged. Signing in from Settings > Providers uses a different request path that the gateway tests show is not affected (covered by routing tests only, not by a live account other than ChatGPT/Codex on the fixed build).

Factr-I 0.0.4 is a cost-and-time release for long-context labelling, plus a lighter start-up. It is not a score release and makes no claim to win anything. Scored result against 0.0.3 and the two other harnesses: not significantly different from 0.0.3 on any of the six benchmarks (paired tests, p from 0.39 to 1.0); significantly more accurate than both Prime Agent 0.9.8 and Hermes 2026.9.24 on OOLONG and OOLONG-Pairs, than Prime Agent 0.9.8 on Aider polyglot and than Hermes 2026.9.24 on GAIA; tied with both on TBLite and LongCoT-Mini, and tied with Prime Agent 0.9.8 on GAIA and with Hermes 2026.9.24 on Aider polyglot; no significant loss on any benchmark. It is not a win on all six.

Same model for every arm (gpt-6-luna, medium reasoning effort), same protocol as 0.0.3. The two other harnesses are the same two that were run earlier under the same conditions, in the same order as before (Prime Agent 0.9.8, Hermes 2026.9.24); they were not re-run for this release. One run per benchmark, except LongCoT-Mini, which was run twice on the same binary (first run = paired-test headline, second = labelled replicate; both reported, with their mean).

Results (0.0.4 / 0.0.3 / Prime Agent 0.9.8 / Hermes 2026.9.24; paired exact tests)

Benchmark (n)	0.0.4	0.0.3	Prime Agent 0.9.8	Hermes 2026.9.24	vs Prime Agent 0.9.8	vs Hermes 2026.9.24	vs 0.0.3
OOLONG-Pairs (20), F1	73.8% F1	67.5% F1	32.7% F1	29.8% F1	win (p<0.0001)	win (p<0.0001)	no significant difference (p=0.39)
OOLONG (50)	45/50 = 90.0%	45/50 = 90.0%	30/50 = 60.0%	17/50 = 34.0%	win (p=0.0003)	win (p<0.0001)	no significant difference (p=1.0)
Aider polyglot (225), pass@1	194/225 = 86.2%	192/225 = 85.3%	165/225 = 73.3%	184/225 = 81.8%	win (p=0.0002)	tie (p=0.15)	no significant difference (p=0.85)
GAIA (165)	66/165 = 40.0%	67/165 = 40.6%	62/165 = 37.6%	47/165 = 28.5%	tie (p=0.63)	win (p=0.0013)	no significant difference (p=1.0)
TBLite (100)	68/100 = 68.0% (first pass 66)	69/100 = 69.0% (first pass 67)	64/100 = 64.0%	60/100 = 60.0%	tie (p=0.57)	tie (p=0.15)	no significant difference (p=1.0)
LongCoT-Mini (507)	98/507 = 19.3%	104/507 = 20.5% (second run 98)	89/507 = 17.6%	90/507 = 17.8%	tie (run 1 p=0.29; replicate p=1.0)	tie (run 1 p=0.44; replicate p=1.0)	no significant difference (p=0.52)

Baselines are carried forward, not rerun. The Prime Agent and Hermes columns come from earlier runs; those harnesses were NOT run again for 0.0.4.

• Which runs and when: Prime Agent 0.9.8 and stock Hermes 2026.9.24 were run from 2026-09-30 to 2026-10-02 by result-file times (two at a time; long-context sets 2026-10-01 13:32 to 2026-10-02 01:51, GAIA 2026-10-01 05:00 to 07:38, TBLite 2026-09-30 23:15 to 2026-10-01 12:45, Aider polyglot first result 2026-10-01 03:05) and first appeared in the 0.0.1 report (2026-10-05). The same numbers were reused in the 0.0.2 report (2026-10-06) and the 0.0.3 report (2026-10-08) and are reused again here. Factr-I 0.0.1 to 0.0.3 ran later, 2026-10-04 to 2026-10-08; 0.0.4 ran 2026-10-09 16:05 to 2026-10-10 08:47.
• Why not rerun: a full six-benchmark rerun of both harnesses costs days of compute and model usage, and keeping their versions pinned keeps every release comparable to the same baseline. A rerun was not needed to show what 0.0.4 changes.
• Setup (as in the earlier reports): model gpt-6-luna through the ChatGPT/Codex login, reasoning effort medium, default service tier, the same for every arm; Apple Silicon Mac, macOS arm64 (Apple M2 Max, 12 cores, 64 GB, macOS 27.0.1; Docker Desktop (engine 29.8.0, client 29.8.1) with 12 CPUs and 15.6 GiB for the TBLite containers), with TBLite run as a Linux build in Docker (amd64 emulation); benchmark data revisions: OOLONG: Hugging Face oolongbench/oolong-synth rev f0d59eaf0febf130664cfceb710436c8e3216b2b (test split, Yahoo, 131072); OOLONG-Pairs: mit-oasys/oolong-pairs rev d1e1522b86ac0c169bbc890b0471408aaa29e8fa; LongCoT-Mini: LongHorizonReasoning/LongCoT rev 7f9948570f5239f0f3697c6c9793b4866860c681 (507 easy-tier items; verifier package vendored, version 0.1.0); long-context plan lock hash 977f4aabe1a00883303633213fa8ba49dbc575555990e841758a038e315fe990; GAIA: 165 validation questions, plan lock hash dc63cc607c64584b52a0af99bb4c45554578b78be8d8986666d8f8db21517f95; Aider polyglot: benchmark checkout commit 7e0611e77b54e2dea774cdc0aa00cf9f7ed6144f (225 exercises, 6 languages); TBLite: OpenThoughts-TBLite v2.0, 100 tasks via Harbor, as downloaded for the original baseline runs (no registry revision id was recorded). Same samples, order and byte-identical prompts for every arm; per-sample timeouts OOLONG 1500 s, OOLONG-Pairs 2400 s, LongCoT-Mini 1500 s, GAIA 1500 s, polyglot 1500 s per turn with two tries (test output shown on the retry); infrastructure failures rerun at most twice; one task at a time within a lane; scoring uses the final answer text only, and an empty or refused reply counts as no answer.
• Config differences from each harness's defaults: the benchmark toolset (terminal, file, code execution, todo; web tools only on GAIA, where all three arms' web access was pinned to one shared cached search service). Factr-I's own labelled differences (tool surface, nudges, environment line, memory cap, concurrency) are listed under Disclosures. Any further per-harness configuration: Prime Agent v0.9.8 (commit 7d442aafa985f9342134fac16c2ef41f03fb45c1) and stock Hermes v2026.9.24 (commit f97608f178d1ffeca59860195ab7da295f7c8e5f) were official releases with no source changes, each task starting from a fresh private home; reasoning effort medium (set explicitly for GAIA and the long-context sets, the harness default of medium for Aider polyglot and TBLite); Hermes config sha256: long-context sets 4f8e26ec271c99e9be68d19dc55f4be8d2990ab1a02e78db000907956cc03fb0, GAIA 7f84d47d55d4b637c51b4f336e489211627f63146bb273cdd39a8456bc7ef61f.
• Whose numbers: every number here, including the other harnesses' scores, memory, calls and tokens, is a Factr-I project measurement made with its own runners, scorers and samplers, not a figure published by those harnesses' authors.
• Method and run counts: paired exact tests (exact McNemar on items; sign-flip on OOLONG-Pairs F1). One run per harness per benchmark; Factr-I 0.0.4 on LongCoT-Mini twice (a replicate). A single run carries run-to-run noise, so small differences are not findings.
• Please rerun and tell us: the setup, prompts and scorers are described in the reports; you are invited to rerun any row and report a discrepancy as an issue on the repository.
• Staleness warning: if either harness has shipped a newer version since, its column here may understate it, and a comparison against a stale baseline is weaker than a fresh one. Read those columns as "those versions, on those dates". No logos or endorsement are implied; the names are used only to identify the versions measured.

Aider polyglot pass@2: 224/225 (99.6%) for 0.0.4, against 225/225 for 0.0.3 and 218/225 for Prime Agent 0.9.8 and 223/225 for Hermes 2026.9.24; one exercise failed both attempts; the test files were not modified by the agent in any exercise (0 tampered).

0.0.4 against 0.0.3 (same tests): OOLONG-Pairs mean F1 73.8 vs 67.5 (+6.3 points, p=0.39, 20 questions); OOLONG 45/50 vs 45/50 (p=1.0); GAIA 66/165 vs 67/165 (p=1.0); TBLite 68/100 vs 69/100 after infrastructure reruns, 66 vs 67 on the first pass (p=1.0, first pass p=1.0); Aider polyglot pass@1 194/225 vs 192/225 (p=0.85); LongCoT-Mini run 1 98/507 vs 104/507 (p=0.52), replicate 89, mean of the two runs 93.5 vs 101 for 0.0.3's two runs (nominally lower, within run-to-run noise). None of these differences is statistically significant.

What the results do and do not show

• This release changes cost and time on one kind of workload. It was not built to move accuracy, and a score difference of a few items against 0.0.3 is within run-to-run noise (about 0.3 points on record-level labelling accuracy, about 6 to 9 items on LongCoT-Mini between two runs of the same binary; for 0.0.4 itself the two LongCoT-Mini runs differ by 9 items (98 and 89; 59 correct in both, 39 only in run 1, 30 only in the replicate; paired p=0.34), against 6 items for 0.0.3's two runs (104 and 98). The other five benchmarks were run once, so they have no replicate).
• Significant wins over both Prime Agent 0.9.8 and Hermes 2026.9.24 on OOLONG (p=0.0003 and p<0.0001) and OOLONG-Pairs (both p<0.0001); a win over Prime Agent 0.9.8 on Aider polyglot (p=0.0002) with a tie against Hermes 2026.9.24 (p=0.15); a win over Hermes 2026.9.24 on GAIA (p=0.0013) with a tie against Prime Agent 0.9.8 (p=0.63); ties with both on TBLite (p=0.57 and 0.15) and LongCoT-Mini (run 1 p=0.29 and 0.44; replicate p=1.0 and 1.0). No significant loss against either harness on any benchmark, and no significant difference from 0.0.3 on any.
• Wins, ties and any losses against the two other harnesses are read from the table above only; the 0.0.3 pattern is not assumed to carry over. "Beats the other har...

Factr-I 0.0.3

Factr-I 0.0.3

Factr-I 0.0.3 is significantly more accurate than Prime and Hermes on OOLONG and OOLONG-Pairs, ahead of Prime on Aider polyglot and of Hermes on GAIA, statistically tied with both on TBLite and LongCoT-Mini (nominally ahead on LongCoT-Mini in both of two runs), with no significant loss on any of the six, and with lower peak memory on every benchmark (ratios of medians from about 1.1x vs Hermes on polyglot to about 7x vs Prime on OOLONG, OOLONG-Pairs, GAIA and LongCoT-Mini); it improves LongCoT-Mini over 0.0.2 (p = 0.007).

It is not a win on all six benchmarks and should not be described as one. Memory note: the TBLite ratios compare Factr-I's whole-container process tree with the other arms' harness process trees and are not like-for-like, so they are not used in the range above. Same model for every arm (gpt-6-luna, medium reasoning effort), same protocol; Prime Agent 0.9.8 and stock Hermes 2026.9.24 were run earlier under the same conditions. One run per benchmark, except LongCoT-Mini, which was run twice on the same binary (first run = paired-test headline, second = labelled replicate; both reported, with their mean).

Results (0.0.3 / 0.0.2 / Prime / Hermes; paired exact tests)

Benchmark (n)	0.0.3	0.0.2	Prime	Hermes	vs Prime	vs Hermes
OOLONG-Pairs (20), F1	67.5%	76.6%	32.7%	29.8%	win (p = 0.0009)	win (p = 0.0002)
OOLONG (50)	90.0% (CI 78.6-95.7)	92.0%	60.0%	34.0%	win (p < 0.0001)	win (p < 0.0001)
Aider polyglot (225), pass@1	85.3% (CI 80.1-89.4)	86.2%	73.3%	81.8%	win (p = 0.0004)	tie (p = 0.23)
GAIA (165)	40.6% (CI 33.4-48.2)	41.2%	37.6%	28.5%	tie (p = 0.54)	win (p = 0.0022)
TBLite (100)	69 after two infrastructure reruns (first pass 67)	68 (first pass 67)	64	60	tie (p = 0.44)	tie (p = 0.064)
LongCoT-Mini (507)	run 1 20.5% (CI 17.2-24.2); replicate 19.3% (CI 16.1-23.0); mean 19.9%	16.0%	17.6%	17.8%	tie (run 1 p = 0.082; replicate p = 0.31)	tie (run 1 p = 0.16; replicate p = 0.45)

Aider polyglot pass@2: 100% for 0.0.3 (Prime 96.9%, Hermes 99.1%).

0.0.3 against 0.0.2 (same tests): OOLONG-Pairs p = 0.22, OOLONG p = 1.0, GAIA p = 1.0, TBLite p = 1.0, polyglot p = 0.85, LongCoT-Mini p = 0.0067 (up). The LongCoT-Mini gain over 0.0.2 is real; the other five are unchanged within noise.

What the results do and do not show

• Significant wins over both rivals on OOLONG and OOLONG-Pairs, over Prime on polyglot and over Hermes on GAIA; statistical ties on TBLite, LongCoT-Mini, GAIA against Prime and polyglot against Hermes. No benchmark is a significant loss.
• LongCoT-Mini: nominally ahead of both rivals in both runs (104 and 98 correct against 89 and 90) but not significantly. Run-to-run noise is about 6 items: 70 items were correct in both runs, 34 only in run 1 and 28 only in the replicate. The benchmark prompt tells the model not to use tools and Factr-I follows that instruction; no switch to override it exists.
• Post-hoc, exploratory view next to the headline, not instead of it (rule fixed before the 0.0.3 results: exclude the items whose prompts forbid tools and the chess family where Prime had a pre-installed library, leaving 302 items): 0.0.3 scored 63 (replicate 58) against Prime 41 (p = 0.0038; replicate 0.021) and Hermes 46 (p = 0.019; replicate 0.11).
• The LongCoT-Mini gain over 0.0.2 is not proven to come from this release's one change (below): the evidence for that change is a model-free reproduction and a robustness run on synthetic library tasks (14/16 against 12/16, within noise).
• OOLONG and OOLONG-Pairs are in-distribution: the improvement loop used held-out data from that benchmark family. The other four are out-of-distribution, with this disclosure: 0.0.3's design drew on aggregate failure analyses of the scored 0.0.1 and 0.0.2 runs (counts and general causes only, no per-item tuning); no scored item was used for development.
• OOLONG-Pairs is nominally lower than 0.0.2 (p = 0.22) because 0.0.3 gave no answer on two of the 20 questions (18 of 20 answered against 20 of 20); the other 18 are within a few points. The cause is not established.
• One run per benchmark (two for LongCoT-Mini): differences of a few items are within run-to-run noise.

Memory and cost

Peak process-tree memory per item, median / 95th percentile in MiB (0.0.3 | 0.0.2 | Prime | Hermes):

Benchmark	0.0.3	0.0.2	Prime	Hermes
OOLONG-Pairs	86.3 / 98.3	78.6 / 98.0	626.7 / 679.3	230.6 / 248.8
OOLONG	87.5 / 120.3	93.1 / 110.8	617.5 / 629.7	221.4 / 250.8
LongCoT-Mini	86.1 / 194.0	50.2 / 172.1	609.1 / 667.1	225.8 / 332.4
GAIA	86.1 / 273.3	51.9 / 313.8	637.4 / 677.0	234.2 / 448.9
TBLite (container total for Factr-I, not like-for-like)	26.6 / 180.4	26.9 / 201.4	778.3 / 811.8	241.4 / 306.0
Aider polyglot	382.5 / 722.1	381.4 / 721.6	825.6 / 1061.5	415.6 / 665.8

On LongCoT-Mini and GAIA the median peak is 86 MiB against 50 and 52 for 0.0.2. The cause was then measured with a model-free test: the short-lived child process that builds the new per-session Python environment at engine start adds a transient peak of about 42 MiB with uv on the path (the engine alone stays at about 28.5 MiB) and about 112 MiB without uv, for about 2 seconds, which dominates the peak of short tasks. A lighter version of that child (no pip installer, smaller peak, about 0.1 s instead of 2 s) has been developed for a later version and is not part of this build. Even so, 0.0.3 uses about 7 times less peak memory than Prime and about 2.6 times less than Hermes on LongCoT-Mini.

Cost, mean per item, model calls / wall seconds (0.0.3 | 0.0.2 | Prime | Hermes):

Benchmark	0.0.3	0.0.2	Prime	Hermes
OOLONG-Pairs	42.5 / 185	36.9 / 200	9.8 / 60	7.6 / 65
OOLONG	53.4 / 171	60.1 / 171	5.2 / 25	3.1 / 19
LongCoT-Mini	2.9 / 39	3.0 / 39	3.2 / 55	2.6 / 49
GAIA	7.8 / 54	7.4 / 50	6.9 / 45	6.2 / 40
TBLite	9.5 / 116	9.6 / 121	8.2 / 109	9.7 / 122
Aider polyglot	8.9 / 62	8.9 / 61	7.9 / 44	8.7 / 68

The price in model calls and wall time is concentrated in OOLONG-Pairs and OOLONG (about 4 to 17 times the calls and 3 to 9 times the wall time of Prime and Hermes); elsewhere it is comparable, and on LongCoT-Mini 0.0.3 is faster than both. Wall times are not strictly comparable: up to 5 Factr-I engines ran at once.

Illustrative API-equivalent cost

Tokens per item and a dollar illustration at assumed public per-million-token rates for gpt-6-luna of $0.10 uncached input, $0.01 cached input and $0.50 output. The rates come from third-party price trackers and are NOT verified against OpenAI's own page; the runs used the ChatGPT/Codex login, so no per-token bill exists. Treat the dollars as illustrative. Cached input is cache-read tokens; uncached is the rest of the input; output includes reasoning tokens; the same definitions apply to every arm (Factr-I's logged input includes the cache, Prime's and Hermes's exclude it, and this was corrected for). Factr-I's REPL sub-model calls are included in its totals; Prime's sub-agent sessions (about 1.6% of its tokens) and Hermes's small title-generation calls are not. No request exceeded 272K input tokens for Factr-I or Prime (Hermes's request sizes were not recorded), so no long-prompt surcharge applies to anything measured.

API-equivalent dollars per 100 items (0.0.3 | Prime | Hermes):

Benchmark	0.0.3	Prime	Hermes
OOLONG-Pairs	$1.32	$0.70	$0.73
OOLONG	$2.07	$0.39	$0.28
GAIA	$0.25	$0.46	$0.22
TBLite	$0.50	$0.28	$0.61
Aider polyglot	$0.39	$0.58	$0.50
LongCoT-Mini	$0.15	$0.37	$0.17

0.0.3 is much more expensive than Prime and Hermes on OOLONG (about 5 to 7 times per item, driven by REPL sub-model calls: 17.8k output tokens per item against 0.5k and 0.3k) and about 1.8 times as expensive on OOLONG-Pairs, although it is the cheapest per point of score there. It is cheaper than Prime on GAIA, polyglot and LongCoT-Mini, and about level with Hermes on GAIA, polyglot, LongCoT-Mini and TBLite. Per correct answer (illustrative): OOLONG $0.023 against $0.0065 (Prime) and $0.0082 (Hermes); GAIA $0.0062 against $0.0122 and $0.0077; TBLite $0.0072 against $0.0044 and $0.0101; polyglot $0.0046 against $0.0079 and $0.0061; LongCoT-Mini $0.0074 (run 1) against $0.0207 and $0.0096.

What changed since 0.0.2

• One writable per-session Python environment (with the system site-packages visible): the REPL, the shell's python and pip, the environment line and the install hint all use it, so a package the model installs through the shell is importable in the REPL on the next cell. Before, on hosts with a non-writable system Python, the hint-directed install failed. The model may install packages from PyPI during a run through the shell; nothing is pre-installed or named. FACTR_GUARD_SESSION_VENV=0 turns it off; an explicit FACTR_REPL_PYTHON bypasses it.
• The environment is built on a background thread (engine ready time stays about 0.15 s); the first shell command or REPL cell waits for it (at most 15 s, then falls back to the system interpreter); it is removed at exit and stale ones are swept at start.
• Import refresh before each REPL cell, one-command install hint, install-failure message, and a fix so the environment line is built after the environment exists (its tool list is consistent).
• Windows and CI: tests made platform-neutral, the Windows shell tool description shortened under its token cap, CI timeouts. Public CI is green on macOS, Linux and Windows (the Windows test step is non-blocking).

Gate record and disclosures

• The synthetic-suite per-cell tolerance was breached by an intermediate candidate (7c8f1c7; tools-forbidden cell -4, level-6 free -4, level-5 free ...

Factr-I 0.0.2

Factr-I 0.0.2

Factr-I 0.0.2 is significantly more accurate than Prime and Hermes on OOLONG and OOLONG-Pairs, significantly ahead of Prime on Aider polyglot and of Hermes on GAIA, statistically tied with both on TBLite and LongCoT-Mini, with no significant loss on any of the six, and uses less memory on every benchmark (about 1.1x to 28x), at the price of more model calls and wall time on the long-context sets.

It is not a win on all six benchmarks, and it should not be described as one. Same model for every arm (gpt-6-luna, medium reasoning effort), same protocol; Prime Agent 0.9.8 and stock Hermes 2026.9.24 were run earlier under the same conditions. The numbers below are one run per benchmark.

Results (Factr-I 0.0.2 / Factr-I 0.0.1 / Prime / Hermes; paired exact tests)

Benchmark (n)	0.0.2	0.0.1	Prime	Hermes	Reading
OOLONG-Pairs (20), F1	76.6%	76.9%	32.7%	29.8%	significantly higher than both
OOLONG (50)	92.0% (CI 81.2-96.8)	88.0%	60.0%	34.0%	significantly higher (p = 0.0001 / < 0.0001)
Aider polyglot (225), pass@1	86.2% (CI 81.1-90.1)	89.8%	73.3%	81.8%	significantly above Prime (p = 0.0002); tied with Hermes (p = 0.087); pass@2 225/225
GAIA (165)	41.2% (CI 34.0-48.8)	34.5%	37.6%	28.5%	tied with Prime (p = 0.45); significantly above Hermes (p = 0.0015)
TBLite (100)	68	61	64	60	tied
LongCoT-Mini (507)	16.0% (CI 13.0-19.4)	11.0%	17.6%	17.8%	tied (p = 0.42 / 0.39); up from 0.0.1 (p = 0.004)

Peak memory per task, median MiB (0.0.2): OOLONG-Pairs 79, OOLONG 93, GAIA 52, TBLite 27, LongCoT-Mini 50, polyglot 381. Prime and Hermes medians are in the 0.0.1 report (Prime 609-826, Hermes 216-416). Lower on every benchmark, from about 1.1x lower than Hermes on polyglot to about 28x lower than Prime on TBLite.

Cost, mean per item, model calls / wall seconds (0.0.2 | 0.0.1 | Prime | Hermes):

Benchmark	0.0.2	0.0.1	Prime	Hermes
OOLONG-Pairs	36.9 / 200	41.2 / 151	9.8 / 60	7.6 / 65
OOLONG	60.1 / 171	55.7 / 167	5.2 / 25	3.1 / 19
LongCoT-Mini	3.0 / 39	2.3 / 39	3.2 / 55	2.6 / 49
TBLite	9.6 / 122	11.9 / 130	8.2 / 109	9.7 / 122
GAIA	7.3 / 52	7.1 / 42	6.9 / 45	6.2 / 40
Aider polyglot	8.9 / 61	10.0 / 79	7.9 / 44	8.7 / 68

The price in model calls and wall time is concentrated in OOLONG-Pairs and OOLONG (about 4 to 20 times the calls and 3 to 9 times the wall time of Prime and Hermes); on the other benchmarks it is comparable, and on LongCoT-Mini 0.0.2 is below Prime and Hermes in wall time. Wall times are not strictly comparable: Factr-I ran with other jobs running concurrently.

What the results do and do not show

• Not a win on all six. A statistically significant win over both on OOLONG and OOLONG-Pairs, over Prime on polyglot and over Hermes on GAIA; statistical ties on TBLite, LongCoT-Mini, GAIA against Prime and polyglot against Hermes. No benchmark is a significant loss.
• OOLONG and OOLONG-Pairs are in-distribution: the improvement loop used held-out data from that benchmark family. The other four are out-of-distribution, with the following disclosure: 0.0.2's changes were motivated in part by aggregate failure analyses (counts and general causes only, no per-item tuning) of the 0.0.1 runs on GAIA, TBLite and LongCoT-Mini, and were validated on synthetic and held-out data, never on the scored items. The 0.0.2 rows on those benchmarks are therefore not a fully blind test of generality.
• Polyglot is nominally lower than 0.0.1 (86.2% vs 89.8%, p = 0.18: not significant); the nominal drop is in Java and JavaScript.
• LongCoT-Mini: 81/507. 283 items were answered in one call with no tools (16 correct); the 224 tool-using items scored 65. The benchmark prompt tells the model not to use tools and Factr-I follows that instruction; Hermes ignores it on some problems, and Prime had a benchmark virtual environment that gave it a chess library. The same prompt was used for every arm and no switch to override the instruction was added.
• TBLite 68 includes one pre-registered infrastructure rerun (first pass 67). 0.0.1's TBLite run had an adapter fairness bug that was fixed for this run.
• One run per benchmark: differences of a few items are within run-to-run noise (measured at about 1 to 4 items out of 60 on development slices).

What changed since 0.0.1 (general capabilities, nothing benchmark-specific)

A label-aware final-answer rule and a more precise answer-template detector, nudges that respect a user's instruction not to use tools (a shared detector gates every nudge, the environment block and the deadline reminder), a verify-after-last-write tracker, a check for answers computed without code, a guard for answers taken from a cut-off page, clearer failure messages and an archive fallback for failed fetches, a background-command timeout fix, actionable missing-module hints with installer routes, a shell resolver, a rule against searching for published answers to the task, and the off-benchmark factr bridge tool registered only when its backend is installed. Each guard has a FACTR_GUARD_<NAME>=0 switch. See docs/CHANGELOG.md and docs/design/v002.md.

Labelled differences and disclosures

• Same as the 0.0.1 release: tool surface of 12 tools (GAIA adds websearch and webfetch), verify-on-stop nudges, environment line, deadline reminders, auto-verify gate off, memory and learning off, engine memory cap of 8192 MB for Factr-I only, GAIA web access pinned for all arms to one shared cached search service by operator configuration, TBLite on the Linux build with REPL network access, up to 5 Factr-I engines run at once.
• Earlier-build disclosures from the 0.0.1 release still apply (cancelled 0.0.0 runs on GAIA, polyglot and TBLite; five improvement rounds on held-out data; the TBLite adapter fairness fix).
• GAIA audit for 0.0.2: 0 queries seeking benchmark or answer material (0.0.1 had one).
• 0.0.2 was frozen before any scored run (commit 807cc65) and run once on the six scored sets (2026-10-05 17:50 to 2026-10-06 05:22). Raw per-item traces are not included.

Downloads and verification

• Engine binaries measured in the benchmarks (built from commit 807cc65 on the v0.0.2 development branch, which is not part of this repository's squashed public history; the release tag's source tree is identical to it except for the version numbers in the backend and desktop manifests and the changelog heading):
  • macOS arm64: factr-0.0.2-macos-arm64, sha256 de38c4aa33dacb7f183b668d21a10f5efa87bb155c7f224def9dae6070b8a1e7
  • Linux amd64 (built on Debian 11, glibc 2.30 or newer): factr-0.0.2-linux-amd64, sha256 825e0185ccadd3283bf7f8b5a37f59baf404e8cd16804ec6a6a4f03ee3e6d6e1
  • factr --version prints Factr-I 0.0.2 (807cc65) for these binaries. A binary you build from the release tag reports the tag commit's hash instead, so its checksum will differ.
• Release tag commit: 7eb66d4; its tree differs from the 807cc65 tree only in the version numbers of the backend and desktop manifests and the changelog heading (tree 2486421f53222296fdfb14f28c95939dafa0aefc vs 7bdca1ce22b9086894e556670ddff0255c7b156e)
• Desktop installers (macOS arm64 .dmg and .zip, Windows x64 .exe, Linux x86_64 .AppImage) are built by the release workflow from the tag. They are unsigned (macOS Gatekeeper and Windows SmartScreen will warn). The Windows and Linux installers have never been launched or tested by the authors; the benchmarks ran the macOS and Linux engine binaries only.
• The full report (tables, intervals, paired tests, memory, incidents) is attached as FINAL-REPORT-v0.0.2.md.

Factr-I 0.0.1

Factr-I 0.0.1

On the same model (gpt-6-luna, medium effort) and the same protocol, Factr-I 0.0.1 is significantly more accurate than Prime Agent 0.9.8 and stock Hermes 2026.9.24 on OOLONG (88.0% vs 60.0% / 34.0%), OOLONG-Pairs (76.9% F1 vs 32.7 / 29.8) and Aider polyglot (89.8% pass@1 vs 73.3 / 81.8; 100% pass@2), statistically tied on GAIA (34.5% vs 37.6 / 28.5) and TBLite (61/100 as run, 63 after an adapter fix, vs 64 / 60), and significantly behind on LongCoT-Mini (11.0% vs 17.6 / 17.8). Peak memory per task is lower on every benchmark (from about 1.1x lower than Hermes on polyglot to about 28x lower than Prime on TBLite; median 28-380 MiB vs 226-826 MiB), at the cost of more model calls and wall time on the long-context sets (about 7x slower than Prime on OOLONG). The OOLONG and OOLONG-Pairs gains are on the benchmark family used for development (held-out validation data); polyglot, GAIA, TBLite and LongCoT-Mini are out-of-distribution.

Factr-I 0.0.1 does not beat Prime Agent and Hermes on all six benchmarks.

Results (Factr-I 0.0.1 / Prime Agent 0.9.8 / Hermes 2026.9.24; one run each, paired exact tests)

Benchmark (n)	Factr-I 0.0.1	Prime	Hermes	Reading
OOLONG-Pairs (20), mean F1	76.9%	32.7%	29.8%	significantly higher
OOLONG (50), score	88.0% (95% CI 76.2-94.4)	60.0%	34.0%	significantly higher (p = 0.0005 vs Prime)
Aider polyglot (225), pass@1	89.8% (85.1-93.1)	73.3%	81.8%	significantly higher (p < 0.0001 / 0.003); pass@2 100% vs 96.9% / 99.1%
GAIA (165)	34.5% (27.7-42.1)	37.6%	28.5%	tied (p = 0.54 / 0.11)
TBLite (100)	61 as run (63 after an adapter fix)	64	60	tied (p = 0.73 / 1.0)
LongCoT-Mini (507)	11.0% (8.6-14.1)	17.6%	17.8%	significantly lower (p = 0.0002 / 0.0003)

Earlier build (0.0.0): OOLONG 28.0%, OOLONG-Pairs 36.3%, LongCoT-Mini 12.0%. Its GAIA, polyglot and TBLite runs were started and then cancelled for compute reasons; they are not reported.

Peak memory per task, median / p95 in MiB (Factr-I / Prime / Hermes): OOLONG-Pairs 86/98, 627/672, 231/249; OOLONG 91/110, 618/628, 221/250; GAIA 50/167, 637/675, 234/434; TBLite 28/58, 778/812, 241/302; LongCoT-Mini 49/90, 609/667, 226/332; polyglot 380/718, 826/1060, 416/665. Idle memory after startup is about 35 MiB.

Cost per item, model calls and wall seconds (Factr-I / Prime / Hermes): OOLONG-Pairs 41 calls and 151 s vs 9.8 and 60 vs 7.6 and 65; OOLONG 56 and 167 s vs 5.2 and 25 vs 3.1 and 19; polyglot 10.0 and 79 s vs 7.9 and 44 vs 8.7 and 68; GAIA 7.1 and 42 s vs 6.9 and 45 vs 6.2 and 40; TBLite 11.9 and 130 s vs 8.2 and 109 vs 9.7 and 122; LongCoT-Mini 2.3 and 39 s vs 3.2 and 55 vs 2.6 and 49. Factr-I ran with other jobs running concurrently, so its wall times are not strictly comparable.

What the results do and do not show

• The OOLONG and OOLONG-Pairs gains are in-distribution: the improvements were developed on held-out data from the same benchmark family (a sealed 120-item slice scored 93/120 vs 72/120 for the earlier build, p = 0.0015). Polyglot, GAIA, TBLite and LongCoT-Mini are the out-of-distribution test: one win, two ties, one loss.
• GAIA and TBLite are ties, not wins: the differences with Prime (3 questions, 3 tasks) are within single-run noise.
• LongCoT-Mini is a real loss. The benchmark prompt tells the model not to use tools; Factr-I follows that instruction and answers in one call on 47% of items (239/507), with 3 of those correct; the 208 items where it used tools scored 24.5%. Hermes ignores the instruction on some problems, and Prime had a benchmark virtual environment that gave it a chess library. The same prompt was used for every arm and no mechanism to override the instruction was added.
• Run-to-run noise on the development slices was about 1 to 4 items out of 60.

Labelled differences between the arms

• Factr-I 0.0.1 tool surface: 12 tools (agentgrep, apply_patch, bash, batch, bg, edit, ls, read, repl, replace, todo, write); GAIA adds websearch and webfetch.
• Defaults left on: verify-on-stop nudges, an environment line on the first message (cwd, tools, file sizes), deadline reminders (turn deadline equal to the task timeout), sub-agent and REPL tools, native compaction, parallel tool calls off. Auto-verify gate, memory and learning off in all headline rows. Reasoning effort medium, default service tier.
• Engine memory cap of 8192 MB for Factr-I only (Prime and Hermes had none). On TBLite the service-type task containers were limited to at most 4096 MB.
• GAIA: web access was pinned for all three arms to the same shared cached search service by operator configuration (searxng backend, allowed fetch host 127.0.0.1, no Wikipedia or Wayback fallback). Factr-I ran days later and saw more live upstream failures.
• TBLite: Linux amd64 build of the same commit; the REPL has network access there and the shell is /bin/sh. Up to 5 Factr-I engines ran at once, Prime and Hermes at most 2.

Disclosures

• The earlier-build runs on GAIA, polyglot and TBLite were cancelled after seeing its OOLONG score, on the advice of two independent advisors.
• The improvement loop used five rounds on held-out data only; rounds 3 and 4 were worse than round 2 and dropped, round 5 failed the pre-registered gate, and round 2 was frozen as pre-registered before the final run.
• TBLite adapter fairness bug: the benchmark driver killed the agent's background services before verification (the other arms' adapters did not). It was fixed adapter-side and four pre-registered tasks were rerun once (61 to 63). Only one is attributable to the fix; rerunning only failures biases the result slightly in Factr-I's favour, so the headline stays 61/100.
• GAIA: one L3 question had a search query seeking benchmark solution material; the answer was wrong, so the score is unchanged. The audit pattern does not catch such queries. A general rule against searching for published answers is planned for a later version.
• A stale search service from a cancelled run briefly blinded the GAIA audit; the lane was restarted and the affected pilot results archived. A runner token-accounting bug and one accidental runner stop were found and fixed during the run (no scores affected).
• Raw per-item traces are not included.

Downloads and verification

• Engine binaries measured in the benchmarks (built from commit d82f0e3, which is a local commit and not part of this repository's squashed public history; this release's source tree is identical to it except for the version numbers in the desktop and backend manifests, see the tag commit message):
  • macOS arm64: factr-0.0.1-macos-arm64, sha256 d9c43f159ef3ed8768995eb03aa300f879bac1be8ad6825b350f9d473c5da9c9
  • Linux amd64 (built on Debian 11, glibc 2.30 or newer): factr-0.0.1-linux-amd64, sha256 96aefd27028858a78b42b5c66d74c90e70008479f57e171afc3284380fe88b3e
  • factr --version prints Factr-I 0.0.1 (d82f0e3) for these binaries. A binary you build from the release tag reports the tag commit's hash instead, so its checksum will differ.
• Release tag commit: 120553c; its tree differs from d82f0e3 only in the version numbers of the backend and desktop manifests (tree ebc08d7b2d215386e41c7553d26bb09f47040432 vs f3aa77f5c0a60fa3719beee6da42ab717881a7d4)
• Desktop installers (macOS arm64 .dmg and .zip, Windows x64 .exe, Linux x86_64 .AppImage) were built by the release workflow on GitHub-hosted runners from the tag. They are unsigned (macOS Gatekeeper and Windows SmartScreen will warn). The Windows and Linux installers have been built but never launched or tested by the authors; the benchmarks ran the macOS and Linux engine binaries only.
• The full report (tables, intervals, paired tests, memory, labelled differences, incidents) is attached as FINAL-REPORT-v0.0.1.md.
• Reproduce the build: docs/BUILDING.md. The benchmark protocol and run configuration: docs/BENCHMARK-RUN-CONFIG.md.