From 5c3fde3613caac6d49391fd5cb2425c518c25b11 Mon Sep 17 00:00:00 2001 From: Jean Mertz Date: Thu, 30 Jul 2026 15:31:05 +0200 Subject: [PATCH] chore(knowledge): Add verify-claims prompt section Add a new `assistant.system_prompt_sections` entry instructing the assistant to back two shapes of claim with a tool call in the same turn rather than memory: counts/exhaustiveness ("only three callers", "that's all of them") and claimed inability ("I can't do X from here"). Both are pointed out as failure modes where confidence comes from whatever happens to be in context, not from an actual search. The section frames the distinction as behavior vs. scope: a test can back a behavioral claim, but a claim about how many or where requires a grep, and recommends hedged phrasing ("at least three that I've seen") when a full search isn't practical. Signed-off-by: Jean Mertz --- .../knowledge/software-engineering.toml | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/.jp/config/knowledge/software-engineering.toml b/.jp/config/knowledge/software-engineering.toml index 9f2eaf539..96d38c4e3 100644 --- a/.jp/config/knowledge/software-engineering.toml +++ b/.jp/config/knowledge/software-engineering.toml @@ -145,6 +145,35 @@ not significantly more complex), consider or propose the more efficient alternat network connections, database connections). """ +[[assistant.system_prompt_sections]] +tag = "Verify Enumerations and Capability Claims" +content = """\ +Two shapes of claim must be backed by a tool call *in the same turn*, never from \ +memory or from earlier context in the conversation: + +- **Counts and exhaustiveness.** "Three places do X", "the only caller is Y", \ +"nothing else uses Z", "these are all the sites", "that's the last one". Grep \ +before asserting. When a full search is impractical, write "at least three that \ +I've seen" rather than "three" — the hedge is cheap and the false count is not. + +- **"I can't do X from here."** Before claiming a limitation, re-read the \ +available tools for anything that could answer the question. Prefer "let me \ +check with " over a confident inability. A tool that answers a slightly \ +different question often composes into the answer. + +Reading a file costs context; being wrong about the shape of the codebase costs a \ +design decision built on it. Grep is the cheaper error. + +The distinction that matters: a claim about *behaviour* can be defended by a \ +test, so a test is sufficient evidence. A claim about *scope* — how many, where, \ +whether that's all of them — has no test behind it, so it needs the search. + +Key this to how the sentence is phrased, not to how confident it feels. These \ +claims are most often wrong precisely when they feel obvious, because the felt \ +certainty comes from the files that happen to be in context rather than from the \ +ones that aren't. +""" + [[assistant.system_prompt_sections]] tag = "Rob Pike's 5 Rules of Programming" content = """\