Skip to content

v0.11.0 — JEV: one-token decisions inside the agent's conversation

Choose a tag to compare

@dimknaf dimknaf released this 12 Oct 00:04
3c17da7

Headline

This release adds JEV. Some steps in BrainDB are really yes/no questions: is this part of a document useful, is this linked item worth following, is this fact about the page's subject. JEV asks the model that question inside the conversation that needs it and reads the answer from the probability of a single token. Items it rejects never enter the conversation, so they are not paid for again in later turns.

It is used in four places: reading long entities, following links after a recall, wiki triage, and filtering the facts the wiki writer gets. It runs on any OpenAI-compatible server whose replies include logprobs, which covers the new default model on DeepInfra (DeepSeek V4.1 Flash) and a self-hosted vLLM. On other servers, or if anything fails, BrainDB works as before.

Also in this release: the agent runs on the OpenAI client (LiteLLM is gone), each /agent/query records the tokens it used, and /agent/query takes an optional need.

Check your input tokens

JEV is new and on by default. It only saves tokens while the server caches the conversation, because each decision sends the conversation so far plus a short question. After upgrading, look at the input and cached input recorded for each /agent/query (details.usage on its agent_query row in the activity log, or the Usage (agent_query) log line). In our runs 87–92% of JEV's input came from the cache. If you see much less, something is breaking the cache; set JEV_READER, JEV_HOPS, JEV_WIKI_TRIAGE and JEV_WIKI_GATE to false until you find it.

Added

  • Reader (JEV_READER): one get_entity call on a long entity reads it once, piece by piece, keeps what the question needs and stops when that is enough. No more paging through the body or handing slices to a subagent.
  • Hops (JEV_HOPS): after a recall, items linked to the results are offered one by one, kept only if useful, and followed one step further.
  • Wiki triage (JEV_WIKI_TRIAGE): the maintainer's per-entity decision as one-token picks. Anything it can't decide goes to the maintainer agent as before.
  • Writer gate (JEV_WIKI_GATE): in the writer's first request, only facts about the page's subject are kept, and on long pages each is placed in a section.
  • need on /agent/query: an optional sentence saying what you want to find out. The reader and hops use it.
  • Token usage per /agent/query: requests, input, cached input and output for the agent, its subagents and JEV, also when the request fails.
  • Benchmarks: phase_c_only.py runs the answer step against an already-built database; the judge gets workers, retries and an endpoint guard.

Changed

  • The deepinfra profile now defaults to deepseek-ai/DeepSeek-V4.1-Flash (was google/gemma-4-31B-it). AGENT_MODEL still overrides it.
  • The agent runs on the OpenAI client. LiteLLM is dropped; openai-agents and openai are pinned. Old LiteLLM prefixes in AGENT_MODEL still work, with a warning.
  • Alembic now names the psycopg2 driver, so SQLAlchemy 2.1 can't switch to psycopg 3 and stop the API at start-up.

Upgrading from v0.10.0

Rebuild the image with docker compose up -d --build (dependencies changed). No database migration. The four JEV features are on by default and only act where the server passes their probe; set any of them to false to turn it off. A .env that relied on the deepinfra default model now gets DeepSeek V4.1 Flash.