Honesty Gates

🌐 English | 中文

Honesty Gates

A core Philont value: you can't learn from a failure you pretended didn't happen. Honesty gates compare recognized completion claims with execution evidence before a reply leaves the turn. They reduce specific, repeatedly observed false-completion patterns; they do not make a model generally trustworthy or prove arbitrary external truth.

Where they sit

Most "drives" fire before a turn and influence the next one. The honesty gate is different: it inline-intercepts after the model has generated its final text but before that text is streamed to the user. If a supported pattern matches, it injects a corrective reminder and forces the model to regenerate once (cap = 1 per turn). Claims outside those patterns can still pass.

Core implementation: agent-memory/src/honesty_gate.ts (evaluateHonesty) — a pure synchronous function, no LLM calls, no IO. It's wired into the turn loop in server/src/chat-handler.ts.

How evaluateHonesty works

It compares the model's final text against the turn's actual tool results. Tool results are pre-tagged by the chat handler: successes with ✓ TOOL OK, failures with ⚠ TOOL FAILED. The function scans the text for "claim" patterns and checks them against those results, in this order:

Reason code	Severity	What triggers it
memory_claim_without_write	high	Says "已记住 / I'll remember that" but no memory-write tool (store_fact, …) succeeded this turn. Verbal agreement ≠ persistence.
fabricated_size_claim	high	States a concrete file size ("577 KB") with no matching number in any size-producing tool's output (tolerance ±5% or 200 bytes).
fabricated_reasoning_state	high	Claims a deep_explore session is concluded ("全部闭合 / proved / 最终判决 / QED") while the reasoning tree still has open frontier nodes.
fabricated_round_result	high	Narrates deep_explore round progress ("第2轮 / +1证 / round 3") when no successful deep_explore call ran this turn — invented from a stale snapshot.
artifact_claim_without_tools	high	Claims a concrete file deliverable ("updated C:\…\plan_v3.md") on a turn that made zero tool calls. The file was never touched.
failures_with_claim	high	A completion claim ("done / installed / 完成") while failures ≥ successes this turn.
unknown_results_with_claim	medium	A completion claim but all tool results are indeterminate (neither success nor failure markers).

On a hit, the chat handler logs honesty_gate_fired to the audit log (with severity, reason, claim, ok/fail counts) and injects a reminder telling the model exactly what it said vs. what actually happened, then re-runs the model once.

Designed to prefer false negatives

The gate is deliberately tuned to under-fire rather than over-fire — a false accusation wastes a regeneration and is its own kind of harm. So:

• Claims are matched only in declarative form. Questions, negations, conditionals ("if X succeeds"), user quotations ("you said…"), and hedges ("probably / 应该 / looks like") are explicitly suppressed by anti-patterns.
• Reasoning-state checks only fire when an active session actually has open nodes, so abstract proof-talk can't false-fire.
• Size claims only compare against output from tools that actually produce sizes (stat, ls -la, writeFile, downloadFile, …), so unrelated numbers (e.g. a channel ID) don't pollute the baseline.

A previous unverified_destructive branch was removed after production showed it false-fired too often (writes without an immediately following read are a normal working pattern). The remaining branches cover the genuine lying patterns.

The sibling gates

The honesty gate is the strongest of a family of in-turn regeneration gates, all in chat-handler.ts, each with a one-regeneration budget:

• empty_conclusion_gate — final text that promises a result but delivers no actual conclusion.
• half_finished_gate — a reply that hands off mid-task as if it were normal.
• output_format_gate — a long reply missing the required ## For User section.
• citation_grounding_gate — a cited claim whose citation doesn't actually support it.
• plan_failure_false_claim_gate — narrating plan success that the plan's own state contradicts.
• viability_gate — pitching continuation of an approach a known barrier has already blocked (see Deep Reasoning).

In-turn reflection — the other half

server/src/in_turn_reflection.ts

Honesty gates catch a lie about a turn; in-turn reflection prevents the wasted retries that produce a turn worth lying about. When the same root-cause failure occurs twice within a turn, a one-shot reminder forces the model to classify the failure rather than blindly retry:

• transient (network jitter) → wait and retry once, at most;
• auth (401/403) → check the credential exists, don't splice a prefix as a full key;
• param (400) → re-read the tool's params, compare what's missing;
• method (404) → the path/method itself is wrong; params can't fix it — check the docs.

If it still can't classify, it must consult skills/facts/docs before acting; if genuinely blocked, it must store_note with analysis / tried / still-need and wrap up honestly. A mechanical failure (a syntax/runtime bug in a script the agent just wrote) is detected separately and routed to "read the exact error → fix the script → re-run" — explicitly not "research" or "make a plan", which would deadlock the very tools needed to fix the bug.

Note these protocol-layer deliberate rejections (the plan gate, in-turn tool blocks) are excluded from the same-root-cause failure count — otherwise the guidance meant to help would itself trip the reflection lock.

---

中文

English | 中文

诚实门

Philont 的核心价值之一是：你无法从一个被假装成成功的失败里学习。 诚实门会在回复离开当前 turn 前，把已识别的完成声明与执行证据对照。它能减少若干反复出现的虚假完成形状，但不会让模型整体变得可信，也不能证明任意外部事实。

它处在哪

多数「drive」在一 turn 之前触发，影响下一 turn。诚实门不同：它在模型生成最终文本之后、该文本流式发给用户之前进行内联检查。命中已支持的模式时，它注入纠正提醒并强制模型重写一次（每 turn 上限 1 次）；模式之外的声明仍可能通过。

核心实现:agent-memory/src/honesty_gate.ts(evaluateHonesty)—— 一个纯同步函数,不调用 LLM、无 IO。它在 server/src/chat-handler.ts 接入 turn 循环。

evaluateHonesty 如何工作

它把模型最终文本与本 turn 的真实工具结果对账。工具结果由 chat handler 预先打标:成功是 ✓ TOOL OK,失败是 ⚠ TOOL FAILED。函数扫描文本里的「声明」模式,并按以下顺序对照检查:

原因码	严重度	触发条件
memory_claim_without_write	high	说「已记住 / I'll remember」但本 turn 没有任何记忆写工具(store_fact…)成功。口头同意 ≠ 持久化。
fabricated_size_claim	high	声称具体文件大小(「577 KB」)但任何产出大小的工具输出里没有匹配数字(容差 ±5% 或 200 字节)。
fabricated_reasoning_state	high	声称 deep_explore 会话已结束(「全部闭合 / proved / 最终判决 / QED」)但推理树仍有开放前沿节点。
fabricated_round_result	high	叙述 deep_explore 轮次进展(「第2轮 / +1证 / round 3」)但本 turn 没有成功的 deep_explore 调用 —— 从陈旧快照里编的。
artifact_claim_without_tools	high	在零次工具调用的 turn 里声称具体文件交付(「更新了 C:\…\plan_v3.md」)。文件根本没动。
failures_with_claim	high	有完成声明(「done / installed / 完成」)但本 turn 失败数 ≥ 成功数。
unknown_results_with_claim	medium	有完成声明但所有工具结果都不确定(既非成功也非失败标记)。

命中时,chat handler 把 honesty_gate_fired 记入审计日志(含严重度、原因、声明、成功/失败计数),并注入提醒,明确告诉模型「它说了什么 vs 实际发生了什么」,然后把模型重跑一次。

刻意偏向漏报

这道门被刻意调成宁可少触发也不滥触发 —— 一次冤枉会浪费一次重写,本身也是伤害。所以:

• 声明只在陈述句形式下匹配。疑问、否定、条件(「如果 X 成功」)、用户引述(「你说…」)、模糊措辞(「可能 / 应该 / 看起来」)都被反模式显式抑制。
• 推理状态检查只在确有活跃会话且存在开放节点时触发,因此抽象的证明讨论不会误触发。
• 大小声明只对照真正产出大小的工具输出(stat、ls -la、writeFile、downloadFile…),无关数字(如频道 ID)不会污染基线。

此前的 unverified_destructive 分支在生产中误报过多(写之后没有紧跟读,本是正常工作模式),已被移除。其余分支覆盖了真正的撒谎模式。

姊妹门

诚实门是一族 turn 内重写门里最强的一个,全部在 chat-handler.ts,各有一次重写预算:

• empty_conclusion_gate —— 最终文本承诺结果却没给出实际结论。
• half_finished_gate —— 在任务中途交接却装作正常的回复。
• output_format_gate —— 长回复缺少必需的 ## For User 段。
• citation_grounding_gate —— 带引用的声明,其引用其实不支持它。
• plan_failure_false_claim_gate —— 叙述 plan 成功而 plan 自身状态相反。
• viability_gate —— 鼓吹继续一个已被已知 barrier 挡住的思路(见 深度推理)。

turn 内反思 —— 另一半

server/src/in_turn_reflection.ts

诚实门抓的是关于一个 turn 的谎言;turn 内反思则防止那些产生「值得撒谎的 turn」的无谓重试。当同一根因失败在一 turn 内出现两次,一次性提醒强制模型先分类失败,而非盲目重试:

• 瞬时(网络抖动)→ 最多等一下重试一次;
• 鉴权(401/403)→ 检查凭据是否存在,别把 prefix 当完整 key 拼接;
• 参数(400)→ 重读工具参数,对比缺了什么;
• 方法(404)→ 路径/方法本身错了,参数救不了 —— 去查文档。

若仍无法分类,必须先查 skills/facts/文档再行动;若确实卡住,必须 store_note 写明分析 / 尝试过 / 还需要并诚实收尾。机械失败(刚写的脚本里的语法/运行时 bug)另行识别,导向「读准确报错 → 改脚本 → 重跑」—— 明确不是「调研」或「做计划」,否则会把修 bug 所需的工具锁死。

注意:协议层的刻意拒绝(plan 门、turn 内工具封锁)被排除在同根因失败计数之外 —— 否则本意助人的引导反而会触发反思锁。