feat(*): tools reliability fixes, coding profile, and main sync - #1
Merged
Conversation
## Summary - Remove the Star Us and Star History section from the English README. - Remove the corresponding Star section from the Chinese README. - Remove both sections from their tables of contents. The embedded Star History chart still does not render reliably, so the promotional section is being removed instead of retaining a broken README surface. ## Type - [ ] Fix - [ ] Feature - [x] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `make check-commits` passed. - `PR_TITLE='docs: remove star history section' make check-pr-title` passed. - `make check-large-files` passed. - Python test target passed: 58 tests. - The isolated ModelPicker test passed. - The full TUI run passed 901 tests and failed 1 unrelated ModelPicker timing assertion. The failure reproduces only in the full suite and is outside this README-only diff. - [ ] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A
## Summary Add first-class MiniMax Token Plan OAuth support for both Global and CN regions. This change adds: - `minimax_global` and `minimax_cn` provider entries with curated MiniMax-M3 and M2.7 models. - OAuth 2.0 device authorization with PKCE S256 and strict state validation. - The shared MiniMax public client ID `coding-plan-cli` with the `openid profile coding_plan` scope. - Region-specific OAuth and Anthropic-compatible API endpoints. - Proactive access-token refresh with a five-minute buffer, cross-process locking, transient retry handling, and atomic refresh-token persistence. - Separate Global and CN credential files, isolated from existing Codex OAuth storage. - Runtime token refresh and MiniMax authentication headers for both normal and streaming requests. - CLI login commands, onboarding, provider status/test/reset support, TUI setup detection, and model-picker entries. - English and Chinese documentation updates. Public model references remain under `minimax-global/` and `minimax-cn/`. The runtime translates them to the Anthropic-compatible LiteLLM route only when sending a request, so provider selection continues to use the correct MiniMax OAuth credential. The implementation accepts MiniMax expiry values expressed as relative seconds, Unix epoch seconds, or Unix epoch milliseconds. OAuth-returned verification and resource URLs are restricted to the expected HTTPS hosts before use. ## Type - [ ] Fix - [x] Feature - [x] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_minimax_oauth.py tests/test_cli_provider_commands.py tests/test_provider_catalog.py tests/test_config_update_providers.py tests/test_tui_rpc_model.py tests/test_tui_rpc_setup.py tests/test_cli_status_commands.py -q -x` - 143 passed. - Focused MiniMax onboarding tests - 3 passed. - `uv run ruff check raven tests` - passed. - `uv run ruff format --check raven tests` - 761 files already formatted. - `make check-large-files` - passed. - `make check-commits` - passed. - Two independent read-only reviews completed; the final review found no blocking issues. - The full test collection was not run because the local environment does not include the optional `dingtalk_stream` channel dependency. All provider, OAuth, onboarding, TUI setup, status, and configuration tests relevant to this change passed. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk - OAuth credentials are stored in region-specific files with restrictive permissions and atomic replacement. - Refresh operations use a cross-process lock and persist rotated refresh tokens before exposing them to the runtime. - OAuth verification and API resource URLs are validated against region-specific host allowlists. - Existing `minimax` API-key configuration remains unchanged; the new OAuth providers use separate slugs and storage. - Rollback consists of reverting this change. Existing API-key provider configuration and credentials are not migrated or modified. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A - Live OAuth smoke test: MiniMax Global MiniMax-M3 returned "MiniMax OAuth works" successfully after normalizing the Anthropic-compatible resource URL. --------- Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
…nd-AI#224) ## Summary CI ran `make test-python` over 4 hardcoded files, so ~268 of 272 top-level test files never executed on any PR. A bare `uv run pytest` could not stand in for it either: collection aborted with 7 errors (channel-adapter tests importing optional SDKs at module level), and `testpaths` pulled `tests/integration/` into every run. This makes the whole unit suite the blocking CI gate. - `Makefile`: `test-python` runs the whole unit tree instead of a hardcoded file list. - `ci.yml`: the `python` job splits into `lint-python` and a `unit` job running the whole suite on ubuntu / py3.12; the matrix block is kept so more versions are one list entry away. - `pyproject.toml`: register `integration` / `e2e` markers, enable `--strict-markers` / `--strict-config`, exclude `tests/integration` via `norecursedirs`, and default to `-m "not integration and not e2e"` so a bare `pytest` is safe. - Guard the 7 channel-adapter tests with `pytest.importorskip`; CI installs `--all-extras`, so they run there rather than skip. - Relocate 9 integration / real-resource files out of the flat unit tree into `tests/integration/`, renaming the token_wise benchmarks to the `real_llm` kind required by AGENTS.md section 5.2. Turning the suite on surfaced 44 pre-existing failures, all fixed here: - 29 from `asyncio.get_event_loop().run_until_complete()` in three producer test files, which picks up a closed loop once other async tests have run; now `asyncio.run()`. - 2 stale `run_turn` stubs missing the `inline_tool_stream` keyword the production callers pass. - 12 from cross-test state leaking through `raven.config.loader._current_config_path`, litellm's global logging handlers, and an ambient dependency on a populated `~/.raven` in the TUI init-error tests. - 1 flaky concurrency test whose timing bound left only 0.05s of margin. The full run also recovered 38 genuine unit tests under `tests/tui/autotest` that a directory-level exclusion would have orphaned; they are selected normally now, with the 24 subprocess-spawning e2e tests deselected by marker. Deliberately out of scope: an integration / nightly CI tier (real-LLM runs are hard to make meaningful in CI and cost money -- to be designed separately), and the naming and assertion debt tracked in EverMind-AI#223. ## Type - [x] CI / tooling ## Verification ``` uv run --all-extras pytest -q # 4611 passed, 24 deselected in 193s bash .claude/scripts/preflight_ci.sh # ALL PREFLIGHT CHECKS PASSED (commitlint, large files, pre-commit, ruff, tests) ``` - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally ## Risk - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes No production code changes -- test files, test configuration, and CI workflow only. The repository ruleset currently declares no required status checks, so renaming the job blocks nothing. If the `unit` job should become a merge gate, add `unit (py3.12 / ubuntu-latest)` to the ruleset separately. ## Related Issues Closes EverMind-AI#222 --------- Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
…Mind-AI#226) ## Summary Two packaging defects, plus the CI simplification the second one unblocks. **`raven[sandbox]` could not be installed on Windows.** boxlite publishes wheels for macOS and Linux only -- it is a microVM runtime built on KVM / Hypervisor.framework, and its PyPI classifiers say so -- but our declaration carried no platform marker, so the install failed outright: ``` error: Distribution `boxlite==0.9.5` can't be installed because it doesn't have a source distribution or wheel for the current platform ``` The code already degrades gracefully when boxlite is absent: `sandbox/interfaces.py` keeps the shared types importable without it, and both call sites probe with `try: import boxlite / except ImportError`. So a Windows user should have gotten a Raven that reports the sandbox as unavailable, not a failed install. `python-socks` and `matrix-nio` already carry the same `sys_platform != 'win32'` marker; boxlite now does too. **The `retrieval` extra had no consumer.** It declared torch and transformers, which nothing imports -- a whole-tree grep finds them only in that declaration, two comments, and a doc note, with no dynamic loading and no doc telling users to install it. It dates back to the initial commit. Keeping it cost a multi-hundred-megabyte download per environment and added a native crash surface of the kind `cli/_exit.py` documents. Removed via `uv remove`; the lockfile drops from 220 to 194 packages. **CI follows.** With torch out of the extras, `--all-extras` now resolves to what the hand-maintained `channels + tools + sandbox` list covered, so the unit job goes back to it and stays correct as extras change. Every job also gains `timeout-minutes`. A `windows-latest` cell was added to measure whether the marker fix was enough to run the suite there. It is not: the install now succeeds, but the suite hit 121 failures by 28 percent, across sixteen files, from systemic platform assumptions (assertions hardcoding forward slashes, fcntl locking, socket and subprocess differences). That is a port, not a fix, so the cell was dropped. This matches comparable projects: hermes-agent runs every job on ubuntu and substitutes a blocking static check for Windows-unsafe primitives, and openclaw's PR-blocking workflow is ubuntu-only with Windows behind workflow_dispatch. The dedicated Windows self-upgrade job still covers the install path. The macos cell is dropped for the same reason, one level milder: it reported the same failures as ubuntu on every run while ubuntu additionally caught two Linux-specific problems it could not, and development happens on macos anyway. The unit job is now a single ubuntu cell, with the matrix block kept so another entry is one list item away. ## Type - [x] Bug fix ## Verification ``` uv run --all-extras pytest -q # 4611 passed, 24 deselected -- same count as before, torch was never used uv run python -c "import importlib.metadata as m; m.version('torch')" # PackageNotFoundError, and boxlite still installs on this macOS host bash .claude/scripts/preflight_ci.sh # ALL PREFLIGHT CHECKS PASSED ``` - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally ## Risk - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes `raven[retrieval]` stops being a valid extra. Nothing in the repo or the docs referenced it, and installing it only pulled torch into the environment without any code path using it, so no behaviour changes. The Windows matrix experiment is described above and left out of the final state; both dependency fixes stand on their own. ## Related Issues Closes EverMind-AI#225 --------- Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
…rMind-AI#220) ## Summary The TUI already read `update_behind` / `update_command` out of the session init bundle, but nothing on the Python side ever filled those fields, so the hint never appeared. This wires up the missing half and moves where it shows. **The check** `raven/cli/update_notice.py` compares the running version against a small cache in `~/.raven/update_check.json`: - `raven tui` calls `maybe_refresh_async()` once per launch. It spawns a daemon thread only when the cache is missing or older than 24h, so a normal launch touches the network at most once a day and never blocks startup. - The gateway only *reads* that cache when it builds the session info bundle (`_default_session_info`). Reading is pure and fast; no network on the session-create path, which is why the check is cached rather than live. - Every failure path is swallowed: no cache, unparseable version, corrupt JSON, network error, rate limit. An update hint must never break startup. The tradeoff is staleness by one launch: the first launch after a release lands refreshes the cache, and the hint shows on the next launch. **Where it shows** The status bar's right slot, in the warn color, replacing the cwd/branch label while an update is available. No extra line, no layout shift. The old banner block in `branding.tsx` is removed. The banner is the first thing on screen at startup and an upgrade nudge does not deserve that weight. `update_behind` is now a flag rather than a commit count, so the text is version-agnostic (`Update available - run raven upgrade`) instead of the old `N commits behind`. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` uv run ruff check raven/ tests/ # All checks passed uv run ruff format --check raven/ tests/ # 760 files already formatted uv run pytest tests/test_cli_update_notice.py \ tests/test_cli_tui_commands.py \ tests/test_tui_rpc_session.py -q # 95 passed cd ui-tui && npm run type-check # clean cd ui-tui && npm test # 904 passed | 3 skipped cd ui-tui && npm run lint # 0 errors cd ui-tui && npx prettier --check "src/**/*.{ts,tsx}" # clean ``` New tests cover the cache read/write and refresh throttle (`tests/test_cli_update_notice.py`), the status-bar slot swap (`ui-tui/src/__tests__/statusRule.test.tsx`), and the banner no longer rendering the block (`branding.test.tsx`). Also exercised the five edge cases by hand: absent cache, newer cached release, cache equal to current, unparseable version string, and a corrupt cache file. Only the second one returns a notice; the rest return `None`. The lint warning reported on `appChrome.tsx` (`react-hooks/exhaustive-deps`, line 275) is pre-existing on `main` and untouched by this change. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed ## Risk - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes The GitHub releases fetch reuses `upgrade_commands._fetch_latest_release`, so no new network surface or credential handling. The cache file holds a version string and a timestamp only. Wire-compatible: `update_behind` / `update_command` were already optional fields in the info bundle, and a client that never receives them behaves as before. Rollback is reverting this commit; the feature degrades to its current state, which is showing nothing. To see it locally: ``` printf '{"latest_version":"9.9.9","checked_at":%s}' "$(date +%s)" > ~/.raven/update_check.json ``` ## Related Issues N/A --------- Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
…I#217) ## Summary Reworks the TUI transcript so a turn reads as a sequence of steps instead of interleaved thinking blocks and tool lines. Every model call becomes one "episode" holding its reasoning, narration and tools. **Rendering** - A step with no narration collapses to one summary line ("reasoning for 8s, read 2 files"); a narrated or running step shows a reasoning fold, the narration as prose, then its tool rows. - Reasoning expands while it streams and folds once the step produces anything visible (narration, a tool, or the answer text). - A run of same-name calls renders as a tree: one header plus a child row per call, so a burst of parallel searches no longer fills the screen. - In-flight tools report elapsed time; finished ones keep their measured span. - Rows are bound to the transcript width and clipped with an ellipsis. Without the bound, long details overflowed the container and the terminal soft-wrapped them, which looked like stray blank lines and edge-cut text. **Tool display contract** Tools now own how they appear. `Tool.display_call` and the new `ToolResult` (`model_text` plus optional `display_text`) separate what the model reads from what the transcript shows: - `ask_user` renders its question and the answer instead of a raw arguments blob, pairing each question with its answer when several are asked at once. - Tools with no explicit verb derive a readable one from their own name, so a newly added tool renders sensibly with no changes to the verb table (it is an overrides list, not a required registry). - The call preview picks the query, question or command out of the arguments instead of the first value, which could be a numeric flag. **Also** - Episodes are the default transcript style; `/transcript legacy` stays as an escape hatch and the legacy path is untouched. - An interrupt commits the collapsed episode view instead of dumping the raw expanded segments. - The virtual-height estimate accounts for episode rows. Without it the transcript reserved too few rows and left stale cells on screen. - Markdown table columns fit the available width with CJK-aware clipping. A cell wider than its column is truncated with an ellipsis and cannot be expanded in the TUI (it used to soft-wrap, which kept every character but looked broken), and inline markup inside a cell is flattened to plain text rather than rendered. - The deep-research offer states that a regular search is the user's chosen path, not a failure, so the model stops narrating it as a broken fallback. **Also in this diff, beyond the stated scope** - The queued-message panel is restyled: the `queued (N)` header is gone and the edit hints moved to a trailing line. The count stays derivable from the window plus `...and N more`, and the edited row is still highlighted. - User messages get their own tier: an accent chevron and bold text, so a prompt reads as a prompt next to the assistant's prose. - `transcriptGutterWidth` takes the tool glyph, so the user and assistant gutters are derived from one place instead of two hardcoded widths. ## Review follow-ups Addressed in the second commit, by the reviewer's IDs: - **R1** an interrupt with no `episode.start` dropped the accrued segments and tool trail; it now falls back to the legacy trail exactly like the completion path, with a regression test. - **R2** the `ToolResult` / `display_call` contract had no Python coverage. Adds `tests/test_ask_user_tool.py` and a loop test asserting `add_tool_result` gets `model_text` while `tool.complete` carries `display_text`. - **R3** the transcript-mode comment was wrong on all three claims. - **R4** `ToolRegistry.execute` now unwraps at the boundary and returns `ToolOutput`, a `str` subclass carrying the display string, so the sentinel action executor, subagent manager, curator and tracing can no longer receive a dataclass. `isinstance(.*ToolResult)` hits the registry only. - **R7** drops `episodeLabel`, `turnSummary`, `toolLine` and `episodeIndex`. - **R8** the height estimate counts the inter-step margin row. - **R9** (mid-turn `/transcript` switch) and **R10** (clarify answer beyond the 200-char preview) are left for a follow-up: both add state or a wire field rather than fixing this diff. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` cd ui-tui && npm run type-check # clean cd ui-tui && npm test # 927 passed | 3 skipped cd ui-tui && npm run lint # 0 errors, no warnings in touched files cd ui-tui && npx prettier --check "src/**/*.{ts,tsx}" # clean uv run ruff check raven/ tests/ # All checks passed uv run ruff format --check raven/ tests/ # 758 files already formatted make test-python # 4621 passed, 2 pre-existing failures cd ui-tui && npx vitest run # 925 passed | 3 skipped ``` New tests cover the episode commit path (including interrupt), the tool summary and verb fallback, the argument preview, and the rendered layout. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed ## Risk - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes The wire gains one optional `tool.start` field (`display`); older payloads without it fall back to the generic preview. `execute` may now return either a string or a `ToolResult`, so existing tools are unaffected. Rollback is `/transcript legacy` for the view, or reverting this commit. Known follow-ups, not addressed here: occasional screen garbling seen when messages are queued (root cause not yet identified), and `web_search` being registered even without an API key. ## Related Issues N/A --------- Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
## Summary A read-only audit of all 298 test files found the suite in good shape -- exactly one genuinely fake test -- with the real debt sitting in file names and a scattering of assertions that passed for the wrong reason. This clears that, and fixes the bugs found while clearing it. **Three dead tests dealt with.** Two removed because neither could ever assert anything: - `test_chat_e2e.py` was a module-level skip over three `NotImplementedError` stubs. The wire-up it waited for has shipped (`tui_commands` instantiates `SubscriptionEmitter`) and the proposal doc it cited is gone. - `test_tui_exit_e2e.py` asserted that building the agent loop leaves lancedb's thread live. It does not: the loop is constructed but never started, and the memory plugin talks to everos over HTTP, so lancedb is not even imported. The assertion could not hold in any configuration. The third, `slashParity.test.ts`, shelled out to python for a `raven_cli.commands.COMMAND_REGISTRY` that no longer exists, and skipped all three of its cases when the import failed -- three skips on every run. The parity it wanted needs no python: `createSlashHandler.ts:45-51` routes a slash locally when `findSlashCommand` resolves it and otherwise hands it to `slash.exec`, a CLI subprocess a mutation cannot reach the live session from. It now asserts that predicate directly. `NATIVE_MUTATING_COMMANDS` and the three-way `classifyRoute` went with it: all six names in that set are also in the local registry, so the native branch could not change any outcome. **19 files renamed** off phase, ticket and bug codes (AGENTS.md section 5.1), including one reversed CLI name and two integration files moved onto a legal `<kind>`. Reference updates ride along: `.pre-commit-config.yaml` names one of these files in a `detect-private-key` exclude, and three docs pages point at them. **Assertions that passed for the wrong reason, tightened**: eight over-broad `pytest.raises(Exception)` narrowed to the type each site actually raises; an `except` clause that swallowed the `AssertionError` its own fakes use as a tripwire; a `hasattr` probe replaced by binding a broker and checking it stuck; and a recall assertion that read as a pass on an empty result set. **Dead weight removed**: ten unused locals, three dead code blocks, six `importorskip` guards on hard dependencies, two hardcoded `/tmp` paths, and ticket codes left in docstrings after the renames. **Two coverage gaps the deletions exposed are now closed**: - `tests/test_cli_exit.py` pins the hard-exit guard, which had no tests at all. - The TUI chat e2e gains a multi-turn case. It reads the persisted session rather than the screen, because a prompt is echoed into the transcript the moment it is typed -- screen-scraping for a planted word passes even when history is broken. **Three timing bugs fixed in the TUI e2e tests.** They keyed off the banner, which paints about seven seconds before the app accepts input, and submitted the moment text was typed. Dropped prompts made the pipeline look dead in one test, and made a negative assertion (`assert not leak_detected`) pass vacuously in another -- no prompt means no streaming means no leak. **Gaps in the two features that landed during review**, small enough to close here: the `session.py` wiring that fills `update_available` / `update_command` (removing it left the suite green), `_upgrade_command_works` (a fixture stubbed it to `True` everywhere), an `episodeSummary` case with an empty test body, and `clipToWidth` / `hasMeaningfulReasoning`, which had no coverage at all. The larger items from that audit are tracked in EverMind-AI#229. **Upstream review attributions dropped from ui-tui comments.** 36 mentions of another project's review thread (#19835 as "Copilot round-N review"), plus #18994, #19194 and #14045 -- none resolve to anything here, and AGENTS.md section 1.1 rules out comments pointing at information only visible elsewhere. The rationale each carried is kept; four `describe()` titles lose a trailing `(#18994)`. Three went further and documented a python counterpart that does not exist: `platform.ts` credited key-spelling normalization to `raven_cli/voice.py` (no such module -- `raven/tui_rpc/methods/_stubs.py` answers `voice.toggle` with "voice not supported in Raven v0.1"), and `useConfigSync.ts` plus its test credited an `interrupt` default to `raven_cli/config.py` and `tui_gateway/server.py::_load_busy_input_mode` (`busy_input_mode` appears nowhere in python; neither module exists). **One flaky test fixed, caught by this PR's own CI run.** `test_stdio_no_executor_does_not_raise` set `command="true"` and let `connect_mcp_servers` really spawn it, so the result depended on how a live child process raced the MCP handshake -- it failed once on CI in the full-suite run while passing 30 out of 30 standalone runs locally. It also asserted only that whatever came back was not a `SandboxInitError`. The transport now raises instead of running (the idiom its sibling test already uses) and the test asserts the stdio branch was reached. Review follow-ups, folded in: the hard-exit test now skips rather than asserting no lancedb thread exists process-wide, `tests/conftest.py` no longer credits the pytest segfault to a lancedb thread the suite never starts (raven imports lancedb nowhere), and the status-bar patterns the two TUI e2e tests wait on live in one module instead of a hand-mirrored copy per file. Every new or changed assertion was checked by mutating the code under test and confirming the assertion fails. ## Type - [x] Others Test files, test configuration, and comments. The only non-test files touched are ui-tui comments and four `describe()` titles -- no behaviour change. ## Verification ``` uv run --all-extras pytest -q # 4654 passed, 25 deselected cd ui-tui && npx vitest run # 936 passed, no skips uv run --all-extras pytest tests/tui/autotest -m e2e -q # passes locally with tui-use installed and ui-tui built bash .claude/scripts/preflight_ci.sh # ALL PREFLIGHT CHECKS PASSED (including the TUI lane) ``` Mutation checks, each confirming the new assertion fails when the code under test is broken: the `session.py` nudge wiring removed; `clipToWidth` replaced with identity; `hasMeaningfulReasoning` forced to `true`; a never-sent prompt added to the multi-turn session assertion. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally ## Risk - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes No behaviour changes: the non-test diff is comments only. The renames touch git history for 19 test files; EverMind-AI#208 is the only open PR affected and has been notified. Two known items are deliberately left alone: `test_dogfood_whitelist.py` is flaky when the whole file runs (tracked in EverMind-AI#228, unchanged here), and the `lancedb_finalization_hazard` gate in `commands.py` never fires today -- its test docstring says so explicitly, so the coverage does not read as an argument for keeping it. ## Related Issues Closes EverMind-AI#223 --------- Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
…erMind-AI#236) ## Summary `test_dogfood_whitelist.py` was reported as flaky: run the file and a few cases fail, a different few each time. It was not flaky. Three defects stacked, and each one hid the next. **Every one of the 14 output assertions was vacuous.** The expected pattern is matched against the whole screen, which includes the composer echoing the command just typed. 13 of the 14 were satisfied by that echo alone -- `skill list` on `skill`, `cron show` on `cron`, `sandbox list` on `sandbox`. The fourteenth, `status` (`OpenRouter|Model:`), was satisfied by the welcome frame's own status line. This is a pure string property, no timing involved. **Readiness keyed off the banner**, which paints before the app accepts input. Typing right after it, the composer was observed still showing its placeholder -- the keystrokes were dropped entirely. Defect 1 is why that was invisible: a dropped command still satisfied the assertion. **Ctrl+C is a ladder, not an exit key.** In `useInputHandlers.ts` a busy UI cancels the turn, a non-empty composer clears the input, and only then does it quit. The test pressed it once, so any command that left the UI busy timed out in `expect_exit`. That is the reported flakiness, and pressing until the process exits takes the file from `3 failed, 4 passed, 4 xfailed, 3 xpassed` to `0 failed`. The three together also explain the experiment recorded in EverMind-AI#228 as inexplicable: fixing readiness alone made one case fail *deterministically*, because the command then really submitted and really ran a turn, so the UI was busy when the single Ctrl+C arrived. Ruled out with instrumentation rather than argument: a pytest plugin logging `tui-use list` at every setup and teardown shows no session ever leaked between cases, and `tui-use info` retains `Status: exited` indefinitely, so `expect_exit` cannot lose a code it already had. ## What else the audit turned up **Three commands in the whitelist do not exist.** `skill refresh` and `skill stats` (the CLI has only `skill list` / `skill get`), and `cron show`, whose rename to `cron get` shipped long ago. All three were answered with "unknown command" and passed anyway. **The same three defects were in every other file in the tier**, so they are fixed there too, and the seven `xfail(strict=False)` markers are gone -- their reason blamed a Raven overlay-exit defect, but with a correct exit sequence all seven pass. **The README was the source.** Its reference example keyed readiness off the banner, asserted `OpenRouter|Model:`, and exited with a single Ctrl+C. Every file that copied it inherited all three. It now documents the correct sequence, states plainly what the tier cannot do, and drops eight references to documents that no longer exist. **The tier is now scoped to what only it can prove.** Dogfood covers four output shapes instead of 12 commands: `createSlashHandler.ts` routes every dispatched slash through the same `slash.exec` call, and the Python side already has 31 dedicated tests, so per-command repetition proved nothing the first case did not. `test_e2e_tui_status_slash.py` is deleted (same spawn, same command, same assertion as the dogfood `status` case) and so is `test_e2e_raven_chat_cli.py`, an xfail-strict placeholder spawning a `raven chat` that is neither implemented nor planned -- its tripwire moves to `tests/test_cli_smoke.py`, where it costs nothing and actually runs. **The Ctrl+C ladder moves into vitest.** `decideCtrlC` is now a pure function, the shape this file already used for `applyVoiceRecordResponse`, so the rungs are pinned without a PTY or a live turn. Hook behaviour is unchanged: `isTurnActive()` is still only consulted on the busy path. A static sweep of all 301 test files for assertions their own input can satisfy found exactly one more, in `test_session_fork_help`, which asserted `"fork"` against `fork --help` output whose usage line echoes the subcommand name. ## Type - [x] Others Tests, test configuration and comments, plus one behaviour-preserving extraction in `ui-tui/src/app/useInputHandlers.ts`. ## Verification ``` uv run --all-extras pytest -q # 4655 passed, 13 deselected cd ui-tui && npm test # 943 passed uv run --all-extras pytest tests/tui/autotest -m e2e # 13 passed, no xfail, no skips, 2m36s (was 22 passed / 4 xfailed / 3 xpassed in 4m21s) bash .claude/scripts/preflight_ci.sh # ALL PREFLIGHT CHECKS PASSED (including the TUI lane) ``` The dogfood file was run three consecutive times: 12 passed each, no flakes. Every new assertion was mutation-checked: restoring `skill refresh` fails on missing output; pointing a pattern at `OpenRouter` trips the idle-screen guard; spawning `sh -c 'cat > /dev/null'` makes the cat-echo assertion fail where the old one passed; registering a `chat` command trips the new tripwire; three separate mutations of the Ctrl+C ladder each turn vitest red. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally ## Risk - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes The only non-test change is extracting `decideCtrlC` out of the `useInputHandlers` Ctrl+C branch; the branch order and the guard on `isTurnActive()` are preserved, and three mutations confirm the new tests fail when it is not. Left alone deliberately: no CI job runs this tier, so these runs rest on local execution -- the repo's existing gap, now stated in the README rather than implied. The tier also still lives at `tests/tui/autotest/tests/`, a third location AGENTS.md section 5 does not sanction; splitting it into `tests/` and `tests/integration/` needs its own PR, since it is renames plus a fixture move with no bearing on this fix. ## Related Issues Closes EverMind-AI#228 --------- Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
…ger (EverMind-AI#238) ## Summary `raven.cli.commands.run` hard-exited past interpreter finalization whenever lancedb's `LanceDBBackgroundEventLoop` thread was live. That thread cannot exist in this process: lancedb is a transitive dependency only, nothing under `raven/` imports it, memory talks to everos over HTTP, and everos runs out-of-process. `torch`, named alongside it in the agent-loop comment, is neither declared nor imported either. The probe returned False in every configuration, so the branch was unreachable. Removed the probe in `_exit.py`, the `if` it fed in `commands.py`, and the comment in `agent_commands.py` claiming the exit chokepoint covers that hazard. `flush_and_hard_exit` stays: `tests/conftest.py` calls it on CI, where a fully green suite was observed exiting 139 on Linux. Its module docstring now describes that caller instead of the lancedb story, and records why the gate went. Deleting the whole `except SystemExit` block is behaviour-preserving. With the probe always False it reduced to `except SystemExit: raise`, and the exit-code normalization inside it only ever fed the hard-exit call -- never the re-raised exception. ## Type - [x] Others Dead-code removal in the CLI exit path, plus the two tests that exercised the removed probe. ## Verification `CliRunner` invokes the Typer app directly and never reaches `run()`, so the console-script entry point was smoked by hand: ``` raven --version -> 0 raven skill get (missing arg) -> 2 raven nosuchcmd -> 2 raven status -> 0 unparseable ~/.raven/config.json -> one clean warning line, no traceback ``` ``` uv run --all-extras pytest -q # 4653 passed, 13 deselected (4655 before: the two probe tests are gone, # the four covering flush_and_hard_exit remain) bash .claude/scripts/preflight_ci.sh # ALL PREFLIGHT CHECKS PASSED ``` Also grepped the whole repo, including `benchmarks/` and `scripts/`, for `lancedb_finalization_hazard` and `LanceDBBackgroundEventLoop`: no references remain. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally ## Risk - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes The change is a deletion in a path no test host reaches, so the console-script smoke above is the real verification rather than the suite. If an embedded local-lancedb mode ever lands, the guard would need reinstating -- deliberately, and against a probe that can actually fire. ## Related Issues Closes EverMind-AI#232 Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
…ng it (EverMind-AI#250) ## Summary AGENTS.md section 3.7 reproduced `.github/pull_request_template.md` inline, and the copy had drifted from it. Every heading differed, and the documented `Type` list had no `Refactor` or `CI / tooling` -- so a refactor PR following this file had to pick `Others`, while the real template offers the right box. The same section tells you the checkbox mirrors the commit type, which a list missing two commit types cannot do. Reference the file rather than restating it: a copy is what drifts. The filling rules now name the template's actual sections and say what each is for -- `Verification` wants the commands and their result rather than a claim that they ran, `Related Issues` distinguishes `Fixes #NNN` from a bare reference, and `Type` is a mirror of the commit type rather than a fresh judgement call. Net effect is 37 lines removed and 7 added: the rules that are not in the template stay, the duplicate of the template goes. ## Type - [ ] Fix - [ ] Feature - [x] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` uv run pre-commit run --files AGENTS.md # passed make check-large-files # passed npx commitlint --from origin/main --to HEAD --config commitlint.config.cjs grep -rn "Change description|Type of change|Checklists" AGENTS.md # no matches left ``` `CLAUDE.md` is a symlink to `AGENTS.md`, so both read the corrected text. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk Documentation only; no code path changes. The advice it replaces was already unusable as written, so following the new text cannot produce a worse PR than following the old one. Rollback is a revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Fixes EverMind-AI#241 Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
…I#249) ## Summary Provider management used to be a whitelist: a registry of 21 vendors that decided who could be configured at all, plus a second execution path that bypassed LiteLLM entirely for knn routing. This makes LiteLLM the single execution layer and reduces the registry to what LiteLLM cannot tell us. **knn routing now runs on LiteLLM.** It needed several endpoints alive in one process, which `CustomProvider` provided by talking to the endpoints directly -- at the cost of native streaming, retries and tool-calling. `PerModelProvider` now builds one `LiteLLMProvider` per endpoint instead, and `custom_provider.py` is deleted. knn is a decision layer again; execution is LiteLLM everywhere. **Any vendor LiteLLM supports is now configurable.** A key under the vendor's name plus a `<vendor>/<model>` id is enough -- 134 vendors instead of 21, with no registry entry required. `provider list`, `raven status`, the startup gate and the TUI model picker all report them, which they previously could not: a working setup read as unconfigured, so Raven kept pushing the user into a wizard that then refused to configure that vendor. Names LiteLLM has never heard of are still rejected as typos, and a typo no longer makes Raven unstartable. **A provider now has one name.** It used to have two -- ours and LiteLLM's -- reconciled by a `litellm_prefix` field read at every site that compared a name or a model-id prefix. That field conflated two opposite facts: LiteLLM spelling the same vendor differently (`vllm` -> `hosted_vllm`) versus a provider reached through another vendor's driver (SiliconFlow over OpenAI's API). "Is this prefix mine?" has opposite answers for the two, and deriving the distinction from whether the prefix happened to be another entry's name was correct only by coincidence. LiteLLM's spelling is now the provider's own name, `litellm_prefix` is gone, and only a borrowed driver states `via_driver` -- which is absent from `route_names` by construction, so an id asking for `openai/` can no longer be answered with SiliconFlow's key. `zhipu` is renamed to `zai` on the same basis. Four facts were each implemented at several call sites and had already drifted apart -- how a model id splits into a prefix, how a name is spelled for comparison, which names refer to a provider, and how a provider's config section is found. Each now has exactly one implementation, in `registry.py` or `ProvidersConfig.get`. ### Credential routing Several paths could hand one vendor's key to another. All are fixed here: - a gateway's model prefix was read from the raw field rather than the derived one, so a gateway whose name is already LiteLLM's declared nothing and the prefix was dropped -- sending the gateway's key to the vendor named in the model id; - an explicit prefix could be overridden by a keyword match, so `deepinfra/deepseek-ai/DeepSeek-V3` put DeepInfra's key in `DEEPSEEK_API_KEY` and rewrote the model id; - a bare id naming an unconfigured vendor fell back to any credentialed direct vendor, e.g. a lone Anthropic key answering for `kimi-k2.5`; - environment variables were derived from LiteLLM's `validate_environment`, so the key was written into every variable it listed -- `AWS_SECRET_ACCESS_KEY` for Bedrock, `CLOUDFLARE_API_BASE` for Cloudflare; - `litellm.api_base` was set process-wide, leaking one provider's endpoint to every other call in the process; - the OpenRouter shortlist offered bare `anthropic/...` ids, which auto-detection reads as a request for that vendor direct, quietly leaving OpenRouter as soon as the user also held that vendor's key; - a `routing.models` entry without an `api_base` fell through to `OPENAI_BASE_URL` or `api.openai.com`, shipping the prompt and that endpoint's key to a third party; it now raises. There is no attacker model here -- these are misrouting bugs, not an authentication bypass -- but each one sends a credential somewhere its owner did not intend, so they are called out rather than folded into the summary. ### Also fixed - an empty declared section no longer answers for a provider, so credentials written under one of its other names are not shadowed by a placeholder; - differently-spelled sections (`azure-openai`, `OpenRouter`, `nanoGpt`) resolve to the same provider on both the read and the write path, so a section written as LiteLLM spells it is no longer invisible to `provider get/set` and a write no longer leaves two sections for one provider; - `_litellm_knows` returns three states instead of raising, because only one of its five call sites caught the exception; - the onboard wizard no longer drops `api_key` / `api_base` for vendors with no registry entry, and no longer overwrites a real key with `None` when a re-configure is abandoned; - `raven status` reads credentials from the loaded config, so ones supplied by environment variable are visible, and an OAuth provider without a token reports "not set" rather than a checkmark. ### New `agents.defaults.modelOverrides` lets a user override request params per model, which previously only the registry could do. knn-routed models inherit it, so a routed model behaves like the default provider. ## Type - [ ] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [x] Refactor - [ ] Other ## Verification ``` uv run --all-extras pytest -q # 4869 passed, 30 skipped, 0 failed uv run ruff check raven/ tests/ benchmarks/ uv run ruff format --check raven/ tests/ benchmarks/ ``` 79 new tests across five files. The provider-resolution suite is sweep-shaped: each invariant runs over every registered provider rather than the case that was last fixed, since the drift this PR removes was invisible to case-shaped tests. Every assertion was checked by breaking the implementation and confirming it goes red. It also carries five source guards against a second implementation reappearing. These are line scans -- tripwires, not proofs. A deliberate rewrite walks past them; what they catch is the shape that actually recurred, the same spelling copied to a new call site. Behaviour was compared against the base branch over a matrix of provider configurations, model-id shapes and forced-provider settings. Every difference falls into one of the intended changes below. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed ## Risk Three user-visible behaviour changes: 1. A user whose only credentials are direct vendors -- no gateway -- and whose default model carries another vendor's prefix now gets "provider not configured" instead of a downstream 401. Anyone holding a gateway key is unaffected: a gateway legitimately routes any id. 2. The canonical section names for vLLM and Ollama become `hosted_vllm` and `ollama_chat`, LiteLLM's own spellings. Existing configs keep loading under the old names; the next `provider set` consolidates onto the new one. 3. A `routing.models` entry without an `api_base` now raises instead of silently defaulting to `api.openai.com`. Rollback is a revert: no data migration runs, and configs written before this change continue to load unchanged. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Fixes EverMind-AI#242 Fixes EverMind-AI#243 Fixes EverMind-AI#244 Fixes EverMind-AI#245 Fixes EverMind-AI#246 --------- Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
Add a detached BackgroundJobRegistry (setsid, log file, survives agent exit) next to PTY exec sessions, and route background=true exec calls to it unconditionally, using a provided session name as the job name. Session-hosted background processes died with the agent and failed verifier-time checks in TB eval (4 tasks lost their servers). Also: - anchor shutdown/reboot/mkfs deny patterns to command position and report the matched pattern in guard errors - clamp over-limit exec timeouts (configurable tools.exec.maxTimeout) instead of rejecting them - teach session errors the creation path; suggest the matching tool on invalid parameters (registry did-you-mean) - register web_search only when an API key is configured - spill truncated exec/session output to ~/.raven/tool-output so it can be recovered with grep/read_file instead of re-running - log the final tool manifest (names + schema hash) for eval attribution Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
Every failure path should carry its own next step (G-1 of the tools improvement plan): - repair unambiguous case/format-mangled tool names (Read_File, execRead) and execute the intended tool with a note, instead of failing the turn - stop appending the generic change-approach suffix to validation errors and tool-authored errors; it buried the targeted guidance (did-you-mean, session creation path). Timeouts and unexpected exceptions keep it, as they have no better text - file tools name the recovery tool in not-found errors (list_dir / find to locate, write_file to create) Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
…kpoints Three long-turn robustness fixes (G-6 of the tools improvement plan): - refuse a byte-identical (tool, arguments) call after it has hard-failed 3 times running, instead of executing a fourth identical failure. Scoped to all-failed streaks only: identical successful repeats are legitimate polling (13 such streaks observed across passing eval tasks) and never trip it - repair dangling tool_call/result pairs in loaded history (synthetic aborted results, orphan results dropped) so a crash cannot poison later provider calls - persist the turn-so-far every 10 iterations. A turn is the unit of session persistence and benchmark tasks are one turn of hundreds of iterations; a crash mid-turn used to lose the whole trajectory Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
Add agents.defaults.profile (assistant | coding). The default keeps the personal-assistant identity byte-identical; "coding" renders a software-engineering identity following opencode's default prompt structure and content (tone, conventions, no-comments code style, verify-with-tests discipline, an explicit completion check, and an opencode-style <env> block), with every tool-specific passage rewritten for raven's tool surface: find/grep/read_file/edit_file/write_file routing over shell equivalents, background jobs for servers and over-ceiling work, sessions + exec_write/exec_read for interactive programs, and the untrusted-content rule. ContextBuilder._get_identity now delegates to the segment renderer so the estimator can no longer drift from the request path. Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
Attribution analysis of the tb21 full run (raven 39.3% vs terminus 44.9%), the P0/G fixes landed on this branch with their evidence, the 6-task re-verification results (pypi-server and configure-git-webserver flipped green; kv-store-grpc / install-windows-3.11 now fail on task logic instead of tool defects), and the deferred items (unified exec, todo/verification discipline) with measured regression exposure. Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
Combines the tools-improvement work (detached background jobs, deny pattern anchoring, timeout clamping, actionable errors, same-call breaker, mid-turn checkpoints, coding profile) with the raven-x completion-gate work. The SE-discipline block is hoisted into a shared constant rendered by BOTH identity profiles, so the eval harness's profile=coding selection keeps the discipline it was written for. Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
Resolves the registry boundary against main's ToolOutput/ToolResult split: the unwrap keeps main's display-text channel while retaining the tool-name-repair note and the no-generic-hint-on-tool-errors policy; the loop's test-evidence gate now reads the unwrapped model text. Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Merges the TB2.1 tools-improvement work into raven-x and syncs with
upstream main (litellm provider refactor).
detached job (4 verifier-time server deaths in tb21 attributed)
configurable tools.exec.maxTimeout; actionable tool errors with
did-you-mean and tool-name repair
mid-turn session checkpoints
the SE-discipline block is hoisted so BOTH profiles render it,
keeping it active under the eval harness's profile=coding
docs/tools-improvement-tb21.md (pypi-server and
configure-git-webserver flip green on re-run)
Verification
uv run pytest tests/test_agent_loop_tool_loop_break.py
tests/test_agent_loop_turn_durability.py tests/test_segments.py
tests/test_tool_registry_suggestions.py tests/test_sandbox_unit.py
(111 passed) and config suites (75 passed); ruff check/format clean.
Known pre-existing failures unrelated to this branch: MCP sandbox guard
test, optional channel-dependency collection errors.