🧩 Knowledge the model can find — and chats that can't outgrow it

Released 2026-10-09

A support chat on our own site answered its first question, then replied "I apologize, but I encountered an error" to every message after the third. The real error was in the row we stored: the prompt was 145,155 tokens against the model's 128,000. Nothing the visitor typed was long. Three of our own knowledge searches were — and the agent was sending them back to the model on every turn. That conversation could not recover; every later message would have failed the same way. That is the main thing this release fixes.

Alongside it: a www site whose only problem was its www record, a support agent that told a prospective customer we can't hand chats to a human, two copies of the rule that keeps a business's own users out of its leads, and lead capture asking visitors for an email they had just given us.

Highlights

Knowledge stored in pieces the model can actually find

A crawled page was stored as one row, whatever its length — 12,000 to 31,000 characters for a typical page, and 1.9 million for one. The search embedder reads 512 tokens, roughly the first screen, so every page was searchable by its banner and navigation and nothing below them. Five differently worded questions about the free plan all returned the same three pages, 75,000 characters each time, and none of it contained the answer, which sat 6,900 characters into the pricing page.

Pages are now split into chunks of about 1,100 characters, cut on paragraph, line, sentence and word boundaries. Nothing is trimmed and nothing is rewritten: a page's chunks join back to the exact text that was crawled, and the dashboard and FAQ generator join them that way. Each chunk is embedded together with the page's title and the end of the chunk before it, so a section halfway down a pricing page still matches a question that only names the plan.

A search returns its best matches together with the chunk on either side of each, so a table or a numbered procedure that crosses a boundary arrives whole, and the first chunk of every page it matched, where a product's name and price or a plan's headline terms usually sit. Results are grouped by page, in page order, and held under 15,000 characters; a page that matched always takes priority over another page's heading. Asking again in the same turn returns what has not been shown yet instead of the same text.

Measured on ten production knowledge bases copied to a scratch database: the old search answered 7 of 13 questions while returning 2.7 million characters; the new one answers every question the data can answer, 12 of 12, in 167,000. Replaying the failed chat with a live model, the old prompt grew 42k, 62k, 82k, 102k, then 122.6k tokens, and seven of eight answers were some form of "the knowledge base does not specify". The new prompt never passed 7,300 tokens, and every answer was correct.

On our own hosting, the one-off re-chunk turned 649 whole-page rows into 9,566 chunks across 106 knowledge bases in 21 minutes. Every one of the 933 pages was compared with a backup taken just before, and every one is byte-identical. The three questions from the failed chat now each come back with "1 agent, 1 knowledge source, 300 messages a month".

A conversation can no longer outgrow the model

Search results from earlier turns are no longer replayed in full. The answer built from them is already in the history; the result itself is replaced with a one-line placeholder, and the oldest turns drop out past a size budget. Stored history is untouched — this changes only what is sent to the model. If a provider still rejects a prompt as too long, the turn is retried once without history instead of failing, so a long conversation can lose its early context but can't get stuck.

The embedding model loads once, not on every call

The embedding library rebuilt its model inside every call: about 1.5 seconds of loading around a 7-millisecond embedding. With one row per page that was hidden. With a row per chunk it would not have been, so the model is now loaded once per process and shared.

A www site whose www record is broken

A customer added https://www.utsabzone.store and was told the domain does not resolve. The site was fine; only the www record was broken, and the bare domain served it. A www address with no address of its own is now retried as its bare domain — 7,651 characters instead of a failure in half a second. Only the starting address is rewritten, only after www is confirmed dead, and the fallback still goes through the same safety checks before anything is fetched.

"Can you hand me over to a human?" — "No"

An agent with Transfer to Human switched off was told "Transfer to human is disabled for this agent", and models read that as a fact about the business. Asked whether ChatterMate can hand a chat over to a person, our own support agent said no — about a feature we sell, to someone deciding whether to buy it. Any agent could do this to its own customers. The instruction now applies only to the current conversation, and questions about what the product can do go to the knowledge base.

Lead capture stops asking for an email we already hold

A visitor typed their address into the "enter your email to continue" gate, and a minute later the agent asked for it again, then recorded it as "Email: Not provided". Visitors identified by an embedding app's token got the same treatment. The agent is now told which details it already has, not to ask for them, and to count them as collected. A visitor who volunteers something different still wins.

One rule for identifying an app's signed-in user

The logic that creates or updates a customer for a user an embedding app has already signed in now lives in one place instead of two. That rule is what keeps a business's own users out of its People and leads views, so it should not have two copies. Two bugs went with the duplicate: two tabs identifying the same person at once could fail with a 500, and every token request rewrote the customer row with what it already held.

Notes for self-hosters

• Re-chunk your existing knowledge once after upgrading. New and edited pages are chunked automatically; rows stored before this release stay whole until you run:

  python -m scripts.rechunk_knowledge --dry-run
  python -m scripts.rechunk_knowledge
  

  Run it in the backend or knowledge processor container. It works from the content already in the database — nothing is re-crawled — and replaces each row in its own transaction, so a failure leaves that row as it was. The dry run reports what would change and stops if any new chunk id already exists. It is safe to run again. Until it has run, an oversized row returns its first few chunks with a note that the page continues, so it can't overflow a conversation in the meantime.
• No migrations.
• New optional settings, all with defaults: KNOWLEDGE_CHUNK_SIZE (1100), KNOWLEDGE_CHUNK_OVERLAP (120), KNOWLEDGE_SEARCH_RESULTS (6), KNOWLEDGE_SEARCH_MAX_CHARS (15000), AGENT_HISTORY_MAX_CHARS (40000).
• Restart the knowledge processor along with the backend; it runs the crawler.

Full changelog: v1.0.25...v1.0.26