Skip to content

v3.44.0 — the agent picker gets fixed on your machine, and gets measured

Latest
Latest

Choose a tag to compare

View all tags
@ruvnet ruvnet released this 23 Sep 18:56
v3.44.0
0a96fb8

RuFlo 3.44.0: the agent picker gets fixed on your machine, and gets measured

Released 23 September 2026. Upgrade:

npx ruflo@latest --version    # should print: ruflo v3.44.0

The short version

  • The word-matching fix from 3.43.0 now reaches your computer. The file that picks agents for your prompts was never updated after it was first installed. It now updates itself.
  • A new, optional, smarter agent picker uses a real language model instead of comparing spelling. It's off by default. Turn it on with one setting.
  • We built a test to decide which picker should be the default, and ran it. The honest result: none of the new options met the bar yet, so the default stays as it is. The numbers are below.

1. Your installed agent picker now updates itself

Every time you type a prompt in Claude Code, RuFlo's hook suggests which kind of agent should handle it. It does this with a small file, router.js, in ~/.claude/helpers/.

3.43.0 fixed a bug where it matched parts of words: "review the latest issues" went to the tester because "latest" contains "test". But the fix never reached anyone. router.js was written once, when RuFlo was first set up, and upgrades never replaced it. On the maintainer's own machine, it was from April.

Now router.js is one of the signed files RuFlo keeps up to date. The next time you run any ruflo command, it checks the file against a cryptographically signed list and replaces it if it's out of date:

Prompt Before After
"sync and review latest issues" tester reviewer
"are we uing typesafe" (typo) frontend-dev (because of "ui") coder
"write unit tests for the auth module" tester tester

(ADR-389)

2. Optional: a picker that understands meaning

RuFlo also has a second, "semantic" picker (hooks route). Until now it compared spelling patterns, not meaning. That's why it still thought "latest" looked like "test".

RuFlo already ships a real language model (MiniLM) for its memory search. 3.44.0 lets the picker use it too:

export CLAUDE_FLOW_ROUTER_EMBEDDER=minilm
npx ruflo hooks route --task "fix the login crash"
#  default picker: tester  ✗
#  with minilm:    coder   ✓

It costs about 5 extra milliseconds per prompt, plus a one-time model load of about 0.3 seconds. If the model isn't available, it quietly uses the old method and tells you so in its output (embedder=hash). (ADR-390)

3. How we decided what the default should be

Instead of guessing, we built a test (ADR-391):

  • 197 realistic requests, the kind people really type, each labelled with the right agent. A third are deliberately tricky ("the author field is missing in the prefix list" is a coding job, not an auth one).
  • Labelled blind. The person doing the labelling never saw what any picker chose.
  • Frozen. The list is locked with a checksum, so it can't be quietly adjusted later.

Then we ran four pickers on the same held-out 113 requests:

Picker Got it right Time per prompt (95th percentile)
Current default 26% 1.1 ms
With the language model (new option above) 36% 6.2 ms
typesafe, basic 27% 1.2 ms
typesafe, with a model 29% (44% before its safety gate) 8.3 ms

The rule for changing the default was set before running the test: more than 2 points more accurate, no new required download, and no more than 5% slower. The language-model option is clearly more accurate (+10 points) but is 5 ms slower, which breaks the "5% slower" rule because the current picker only takes 1 ms. So the default did not change, and the rule was not bent to make it pass.

What the test taught us (these are the next steps):

  1. The picker can't give the right answer for about 30% of requests. It has no category for "researcher", "reviewer" or "not an engineering task at all". Adding those will probably help more than any model change.
  2. The speed rule should probably be an absolute limit (say, under 10 ms), not "5% slower than 1 ms". That's a decision for the maintainers, recorded in the ADR, and not changed quietly here.
  3. The typesafe picker is the most accurate when it answers (44%), but its "am I sure?" check is too strict and lets it answer only 6% of the time. That needs tuning.

Also fixed

  • A test for the optional typesafe picker had been failing on the main branch since 3.43.0. It expected the old "latest → tester" mistake. It now checks the right thing.

How this release was checked

  • Every required release test passed:
    • codex 241
    • security 583
    • federation 640
    • 121 CLI tests, including all new router tests
  • The five signed helper files were verified against the signed list.
  • The packed release was installed into an empty folder and tested:
    • A deliberately out-of-date router.js was replaced after one ruflo command, and it stopped sending "latest" to the tester.
    • The language-model option picked "coder" for "fix the login crash", where the default picks "tester".

Full details

  • ADRs: 389 (keep the picker updated), 390 (language-model option), 391 (the test and its results): v3/docs/adr/
  • The test set and the script to re-run it: v3/@claude-flow/cli/benchmarks/router/
  • Pull request: #3410

Also published as a gist: https://gist.github.com/ruvnet/6ad4931d63360b0945ab148ad609e5d5

🤖 Generated with RuFlo

Assets 2
Loading