Autonomy and Self Learning

🌐 English | 中文

Autonomy & Self-Learning

This is what makes Philont a being rather than a task runner: it has goals of its own and it learns from its own failures. Both run inside a budgeted loop with hard ceilings, so autonomy never becomes a runaway.

Intrinsic drives

agent-memory/src/autonomous/drivers/ and agent-memory/src/kernel_drives.ts

Most agents are purely extrinsic — they wait for a task, do it, stop. Philont has three drives of its own:

Drive	Implementation	What it does
Curiosity	CuriosityDriver (drivers/curiosity_driver.ts)	At idle time, researches knowledge gaps — from token-gap discovery and dormant-pursuit recovery. Self-caps at 3 proposals per tick.
Pursuit	PursuitDriver (drivers/pursuit_driver.ts)	Advances stalled long-term goals unasked, via advance-question (research open questions) and check-resolution (test whether a goal's resolution criteria are now met). Default stalled-after = 7 days.
Task commitment	TsTaskCommitmentDrive (kernel_drives.ts)	A turn-time quality check: detects when the model is about to hand off a still-reachable problem, and injects a decision tree that pushes back before it gives up.

Architecturally, K7 is the turn-time drive (task commitment) and K8 is the idle-time initiative layer (curiosity, pursuit).

Where those drives point is yours to declare. The focus areas in your compass are seeded as pursuit rows, which is what both self-drive channels select from — the in-turn drives during your work and the background loop between turns. And the trait tuning below runs inside the leash your compass sets, so a drive can adapt but never escape your bounds.

The autonomous loop and its budget

loop: agent-memory/src/autonomous/loop.ts · budget: agent-memory/src/autonomous/budget.ts · env parsing: server/src/autonomous_budget_env.ts

The loop ticks roughly every 5 minutes. Each tick checks the budget before running, and again before each initiative is dispatched; spend is committed (per-day, into an autonomous_budget table) after the executor returns. The budget has five tiers, all overridable by env:

Cap	Default	Env
Daily tokens	(set via README's PHILONT_AUTONOMOUS_DAILY_TOKENS, default 20000)	PHILONT_AUTONOMOUS_DAILY_TOKENS
Daily tool calls	50	PHILONT_AUTONOMOUS_DAILY_TOOL_CALLS
Per-tick tokens	16 000	PHILONT_AUTONOMOUS_PER_TICK_TOKENS
Per-tick initiatives	8	PHILONT_AUTONOMOUS_PER_TICK_INITIATIVES
Per-initiative tokens	2 000	PHILONT_AUTONOMOUS_PER_INITIATIVE_TOKENS

Global kill switch: PHILONT_AUTONOMOUS=0. Autonomous actions are currently read/research-oriented; write-capable autonomous actions are on the roadmap behind stricter budget + audit.

Self-learning: the reflection pipeline

turn-close: agent-memory/src/reflection.ts + server/src/reflection_runner.ts · in-turn: server/src/in_turn_reflection.ts

Learning happens at two cadences:

• In-turn (immediate) — covered in Honesty Gates: two same-root-cause failures inside a turn force the model to reflect now rather than retry.
• Turn-close (background) — after a task completes (or fails, or hits ≥3 same-root-cause failures, or trips the honesty gate, or runs long), a reflection pass distills durable lessons into memory.

The turn-close pass can produce six kinds of learning:

Type	Effect
routing_rule	"Next time scenario X, do Y" — a trigger + carveout + evidence.
skill_refine	Append a learned condition to an existing skill's description.
playbook	An abstract lesson: when it applies + next-time action + why it isn't a routing rule.
new_skill	A complete reusable step template (name, triggers, action template).
plan_revision	Revise the steps of an active plan.
plan_knowledge	Distil successful endpoint calls into a project's plan.md.

Application is fail-soft: one learning failing to apply does not block the others. The crucial dependency: honest failure closes feed honest playbooks. A pretended success teaches nothing — which is why the honesty gates and plan_close spec-coverage matter to learning, not just to truthfulness.

A new_skill starts as draft. What promotes it is the subject of the next section — and the honest answer is that the mechanism this wiki used to describe ("promoted by successful reuse") was measuring the wrong thing.

Self-learning, redesigned: verify-then-condense

design doc · server/src/learning_judge.ts · server/src/controller_registry.ts

The audit that started it

Traced through the actual call graphs rather than the comments, none of Philont's three learning sub-loops closed in normal operation:

Loop	What the code actually did
Skill maturity ladder	The feedback edge fired on use_skill — which merely fetches a skill's body. "Confirmed" meant retrieved twice, not worked twice. Meanwhile a FIFO cap deleted never-tried drafts.
Routing-rule confidence	recordRuleOutcome always recorded success, never failure — a one-way ratchet. Production: 1022 rules stored, 0 validated.
Playbooks	Write-only. No outcome recorder existed anywhere; created once, injected forever, never validated, never retired.

The root cause was a paradigm, not wiring: all three attempted online per-artifact attribution — "did THIS artifact help THIS turn?" — which is the sparsest, noisiest signal available in a chat agent. Every fix that added another attribution edge was fighting the same headwind.

The organizing principle

Knowledge flows up a layered hierarchy, and condenses to the next level only once it is verified at the current one. Verification gives robustness; condensation gives learning. They are the same pipeline, not opposing goals.

This is also the answer to "reliability or learning": reliability is the valve that keeps the moat clean. Philont's own constitution already said it — learning depends on honesty; a pretended failure corrupts your memory. The five memory layers (episodic → semantic → procedural → guards → meta) stop being just a recall order and become a promotion hierarchy, each upward step gated by verification.

The keystone: the learning judge

server/src/learning_judge.ts · flag PHILONT_LEARNING_JUDGE (default on — shadow only)

Every downstream engine needs one thing that did not exist: something that can say whether a turn actually succeeded. The judge scores each turn success / failure / could_not_verify, and it is built to fail toward silence:

• Deterministic rails decide the clear cases — an execution claim with zero execution tools is a failure; a fired honesty gate is a failure. The aux LLM is consulted only for the ambiguous middle, and an unconfigured / erroring / garbage-returning aux yields could_not_verify — never success.
• success requires a successful grounding tool — one that actually did or verified the thing (shell / execute / http / PARI-GP / z3 / Lean / magnitude), not a bystander readFile. This single rule is what survives adversarial phrasing: a red-team pass found that "some tool succeeded" let a fabricated result plus an irrelevant successful read produce a sycophantic success — memory poisoning. Because the rule does not parse the claim, rephrasing cannot defeat it.
• Pure-reasoning turns land at could_not_verify, never at a false failure. The asymmetry is intended: a missed learning opportunity is cheap; a fabricated success poisons memory.

It was red-teamed before being wired, and every finding is now a regression test. It runs in shadow: it logs a verdict and drives nothing. Phase 2 — crystallizing skills from judge-verified successes, born confirmed rather than climbing a broken ladder — is deliberately gated on the shadow distribution proving trustworthy in real logs. Do not wire learning to a judge that cannot see.

Phase 0: the cheap honest fixes (shipped)

• Retrieval decoupled from efficacy. recordUsage bumps use_count/last_used_at for ranking and touches neither success_count nor the maturity state machine. Efficacy is credited only by the reflector's attribution, which observes whether the actions after a retrieval actually succeeded.
• Superseded playbooks retire. When a reflection produces an executable artifact with the same task signature, the lesson-playbook it replaces is moved to deprecated and excluded from both injection paths. Note what was rejected: a naive count/age cap, which would recreate the FIFO-conveyor pathology that deletes still-relevant lessons. Supersession is a real signal; a cap is not.

Phase 3a: one guard system instead of ten hand-written gates (shipped)

The ~10 answer-time gates Philont wrote one-per-bug (honesty, empty-conclusion, half-finished, output-format, citation-grounding, numeric-grounding, viability, conscience, plan-protocol, phase) are now enumerable as a single controller registry with a data-only catalog and fire counters forwarded to metrics. A new failure mode registers a controller row instead of growing an eleventh bespoke gate. The change was purely additive — every counter sits beside an existing audit append, on no control-flow path.

Three mechanisms that were cancelling the loop (fixed)

Production, 2026-07-21: 14 consecutive scheduled runs, judge success = 0, routing rules 1094 stored / 3 validated. Three independent defects, each alone sufficient to keep the loop from ever closing:

1. A gate fired on an ordinal. The fabrication detector matched 第N轮 — but the branch it guards catches progress invented from an in-progress reasoning snapshot, and with no active session there is no snapshot. A scheduled check-in numbering its own runs tripped it in 5 of 13 runs (第25轮 fired; 第25次 passed), and each hit forced the judge to a deterministic failure. Now gated on an active session — and the sibling rewrite guard, which had been replacing whole replies (five real reports, 1820b → 274b), carries the same premise.
2. The learning layer's output was trimmed away before every prompt. "Lessons I have learned" was shaved first by the prefix trimmer — cut to its floor on thirteen consecutive turns. It is not a historical accumulation; it is the only channel reflection has into the prompt. It is now trimmed last. What had actually grown was a fact series: a recurring task storing one record per run owned a fact slot per run and took the whole top-N within two hours, so collapseFactSeries now keeps the freshest member per key stem (older members stay in the DB).
3. A 24-hour global window was pinned into a per-turn signal. sameRootCauseFailures clusters failures across a global 24h window, so it describes the last day of system state, not this turn — once any failure recurred it sat ≥2 forever and no turn could close clean (success=12 vs ambiguous_skipped=106). Removed from the strong-failure set; the survivors are same-turn events only. (This is the second time this exact defect appeared — the previous occupant of the slot was removed on 2026-07-13, and this one simply moved into the vacancy.)

The general lesson is worth stating separately: a learning loop is a product of AND-gates. Any one silent veto anywhere along it zeroes the whole thing, and without instrumentation the symptom is not an error — it is nothing happening.

Skill self-repair

agent-memory/src/skill_repair.ts · agent-memory/src/autonomous/drivers/skill_repair_driver.ts · agent-memory/src/autonomous/skill_revision_writer.ts · design doc

The reflection pipeline above demotes a callable recipe that stops working — and for a long time stopped there: demote-and-forget. Skill self-repair closes that last hop.

A callable recipe (a skill carrying a machine-checkable verification and a tool policy) is re-verified on every reuse. When it fails its own check it is demoted to advisory — and then, at idle time, the SkillRepairDriver picks it up:

1. Diagnose from the ledger, not from imagination. The recipe's real failed runs are read straight from the execution ledger (ActionLog.getBySkill(..., {onlyFailed:true})) and handed to a single-turn LLM call — one call, no tools. The task is bounded ("here are the failed runs and the recipe body — what is the root cause and the fix"), not open research, so it is not routed through deep_explore.
2. Rewrite, keeping the old version. SkillStore.reviseRecipe snapshots the outgoing steps/verification into a revision_history and writes the fix, re-entering the recipe at draft so it has to re-earn trust through real reuse.
3. Re-verify on next reuse — the same reuse-verification that caught the break now judges the fix. No special path.

Bounds that keep repair from doing harm (a production run on 2026-07-11 first proposed apt-get install to "fix" a missing binary — non-portable, privileged, and it silently escalates what the recipe does to the host on every reuse):

• The diagnosis prompt forbids assuming an OS / shell / package manager and forbids install / privileged / system-mutating fixes. A missing external dependency must be detected and reported, never installed.
• An inconclusive diagnosis rewrites nothing — the recipe simply stays advisory. A guessed rewrite that re-arms a broken recipe is worse than no rewrite.
• A recipe that resists three repairs is retired from further proposals (the revision_history marker counts repair attempts).
• It is the only autonomous driver whose outcome mutates a reusable artifact, so it keeps its own kill switch: PHILONT_SKILL_REPAIR=0 (default on).

Honest scope: this documents the mechanism. Whether a repaired recipe's later reuse actually beats its pre-repair version is a production question, not a proven claim — revision_history is what finally makes it measurable per recipe. If repairs prove net-harmful in the field, the default flips back; that is a data decision.

---

中文

English | 中文

自主与自学习

这是 Philont 之所以是存在而非任务执行器的关键:它有自己的目标,并从自己的失败中学习。二者都运行在带硬上限的预算化回路内,因此自主永不失控。

内在驱动

agent-memory/src/autonomous/drivers/ 与 agent-memory/src/kernel_drives.ts

多数 agent 纯外驱 —— 等任务、做完、停下。Philont 有三个自己的驱动:

驱动	实现	它做什么
好奇(Curiosity)	CuriosityDriver(drivers/curiosity_driver.ts)	空闲时研究知识缺口 —— 来自 token-gap 发现与休眠 pursuit 恢复。每 tick 自限 3 个提案。
追求(Pursuit)	PursuitDriver(drivers/pursuit_driver.ts)	不待吩咐推进停滞的长期目标,通过 advance-question(研究开放问题)与 check-resolution(检验目标的达成标准是否已满足)。默认停滞判定 = 7 天。
任务承诺(Task commitment)	TsTaskCommitmentDrive(kernel_drives.ts)	turn 时质检:检测模型正要交接一个仍可达的问题,并注入决策树在它放弃前顶回去。

架构上,K7 是 turn 时驱动(任务承诺),K8 是空闲时主动层(好奇、追求)。

这些驱动朝哪儿使劲,由你声明。 你指南针里的焦点会被播种成 pursuit 行,而那正是两条自驱通道共同挑活的地方 —— 你在场时的 turn 内驱动,以及两轮之间的后台回路。下面的特质调参也跑在你 compass 设定的 leash 内:驱动可以自适应,但越不出你的边界。

自主回路及其预算

回路:agent-memory/src/autonomous/loop.ts · 预算:agent-memory/src/autonomous/budget.ts · env 解析:server/src/autonomous_budget_env.ts

回路大约每 5 分钟一 tick。每 tick 运行前查预算,每个 initiative 派发前再查;执行器返回后提交花费(按天记入 autonomous_budget 表)。预算有五档,均可由 env 覆盖:

上限	默认	Env
每日 token	(经 README 的 PHILONT_AUTONOMOUS_DAILY_TOKENS,默认 20000)	PHILONT_AUTONOMOUS_DAILY_TOKENS
每日工具调用	50	PHILONT_AUTONOMOUS_DAILY_TOOL_CALLS
每 tick token	16 000	PHILONT_AUTONOMOUS_PER_TICK_TOKENS
每 tick initiative	8	PHILONT_AUTONOMOUS_PER_TICK_INITIATIVES
每 initiative token	2 000	PHILONT_AUTONOMOUS_PER_INITIATIVE_TOKENS

全局急停:PHILONT_AUTONOMOUS=0。自主动作目前以读/研究为主;可写的自主动作在路线图上,需更严的预算 + 审计。

自学习:反思流水线

turn 关闭:agent-memory/src/reflection.ts + server/src/reflection_runner.ts · turn 内:server/src/in_turn_reflection.ts

学习以两种节奏发生:

• turn 内(即时)—— 见 诚实门:一 turn 内两次同根因失败,强制模型当下反思而非重试。
• turn 关闭(后台)—— 任务完成(或失败、或触及 ≥3 次同根因失败、或触发诚实门、或运行过长)后,一次反思过程把可持久的教训蒸馏进记忆。

turn 关闭过程可产出六类学习:

类型	效果
routing_rule	「下次场景 X 就做 Y」—— 触发条件 + 例外 + 证据。
skill_refine	给已有 skill 的描述追加一个学到的条件。
playbook	抽象教训:何时适用 + 下次动作 + 为何它不是 routing rule。
new_skill	一个完整可复用的步骤模板(名称、触发词、动作模板)。
plan_revision	修订一个活跃 plan 的步骤。
plan_knowledge	把成功的端点调用蒸馏进项目的 plan.md。

应用是fail-soft:一条学习应用失败不阻塞其他。关键依赖:诚实的失败关闭喂出诚实的 playbook。 假装的成功什么也教不了 —— 这正是 诚实门 与 plan_close 规格覆盖对学习(而不只是诚实)同样重要的原因。

一个 new_skill 从 draft 起步。真正让它晋升的是什么,是下一节的主题 —— 而诚实的答案是:本 wiki 过去写的那套机制(「经成功复用晋升」)量错了东西。

自学习的重设计:先验证,再凝结(verify-then-condense)

设计文档 · server/src/learning_judge.ts · server/src/controller_registry.ts

引发这一切的那次审计

不看注释、只顺着真实调用图追,结论是:Philont 的三条自学习子回路,在正常运行下没有一条闭合。

回路	代码实际在做什么
技能成熟度阶梯	反馈边挂在 use_skill 上 —— 而那只是把技能正文取出来。「confirmed」的真实含义是被取过两次,不是管用过两次。与此同时一个 FIFO 上限在删除从没被试过的 draft。
路由规则置信度	recordRuleOutcome 只记成功、从不记失败 —— 一台单向棘轮。生产:1022 条规则,validated = 0。
Playbook	只写不读账。代码里根本不存在结果记录器;造一次、永远注入、从不验证、从不退休。

根因是范式而非接线:三者都在做在线逐产物归因 —— 「这个产物对这一轮有没有帮助?」—— 那是聊天 agent 里最稀疏、最嘈杂的信号。每一次「再加一条归因边」的修复,都在顶同一股逆风。

组织原则

知识沿分层结构向上流动,只有在当前层被验证之后才凝结到下一层。验证给出鲁棒性,凝结给出学习。它们是同一条流水线,不是对立目标。

这也回答了「可靠性 还是 学习」:可靠性是保持护城河干净的那道阀门。Philont 自己的宪法早就写了 —— 学习依赖诚实;一次假装的失败会污染你的记忆。 五层记忆(情景 → 语义 → 程序 → 守卫 → 元)不再只是召回顺序,而成为一条晋升阶梯,每一次向上都由验证把关。

拱心石:学习裁判(learning judge)

server/src/learning_judge.ts · flag PHILONT_LEARNING_JUDGE(默认开 —— 仅 shadow)

所有下游引擎都需要一样此前并不存在的东西:一个能说出这一轮到底成没成的裁判。它把每轮打成 success / failure / could_not_verify,并且被设计成「失败时倒向沉默」:

• 确定性护栏先判清楚的情形 —— 有执行声明却零执行工具 = 失败;诚实门开过火 = 失败。aux LLM 只被用于中间那段模糊地带,而 aux 未配置 / 出错 / 返回垃圾时一律得到 could_not_verify,永远不会是 success。
• success 必须有一个成功的「接地工具」 —— 一个真的把事做了或验了的工具(shell / execute / http / PARI-GP / z3 / Lean / magnitude),不是旁观性的 readFile。这条规则是唯一能扛住换措辞攻击的东西:红队发现「有某个工具成功了」这种判据,会让「编造的结果 + 一个无关的成功读文件」在谄媚的 aux 手里变成 success —— 那是记忆投毒。而这条规则不解析声明本身,所以改写话术打不穿它。
• 纯推理的轮次落在 could_not_verify,绝不落成假失败。 这个不对称是刻意的:错过一次学习机会很便宜;一次编造的成功会污染记忆。

它在接线之前先被红队打了一遍,每条发现如今都是一个回归测试。它跑在 shadow:只记录判决,不驱动任何东西。Phase 2 —— 从裁判确认的成功里结晶技能、让技能生下来就是 confirmed 而不是去爬那架坏掉的梯子 —— 被刻意卡在「shadow 分布先在真实日志里被证明可信」这道门后。绝不把学习接在一个看不见东西的裁判上。

Phase 0:便宜且诚实的修复(已上线)

• 把「取用」与「有效」解耦。 recordUsage 只更新 use_count/last_used_at 供排序用,既不碰 success_count 也不碰成熟度状态机。有效性只由 reflector 的归因来记账 —— 它看的是「取用之后的动作到底成没成」。
• 被取代的 playbook 会退休。 当一次反思产出了同任务签名的可执行产物,它所取代的那条教训 playbook 会被置为 deprecated 并从两条注入路径中排除。注意被否决的方案:一个朴素的数量/年龄上限 —— 那会重演「FIFO 传送带」的病理,把仍然有用的教训删掉。「被取代」是真信号,「超额」不是。

Phase 3a:一套守卫系统,取代十道手写门(已上线)

Philont 过去一个 bug 写一道的 ~10 道答案时守卫(诚实、空结论、半成品、输出格式、引用接地、数值接地、可行性、良知、plan 协议、相位),现在被收编为一个可枚举的控制器注册表:纯数据目录 + 转发到 metrics 的开火计数。新的失败类型注册一行控制器,而不是长出第十一道定制门。改动纯增量 —— 每个计数点都紧挨着一条已有的审计写入,不在任何控制流路径上。

三个正在取消这条回路的机制(已修)

生产,2026-07-21:连续 14 次定时运行,裁判 success = 0,路由规则 1094 条 / validated 3 条。三个互相独立的缺陷,每一个单独都足以让回路永远关不上:

1. 一道门对着序数开火。 编造检测器匹配了 第N轮 —— 但它守的那条分支抓的是「从提示词里的进行中推理快照编出来的进展」,而没有活跃会话时根本没有快照。一个给自己运行编号的定时签到,在 13 次运行中触发了 5 次(写 第25轮 开火,写 第25次 通过),每次命中都把裁判逼成确定性的失败判决。现在这条分支以「存在活跃会话」为前提 —— 而它的兄弟(那道会整段替换回复的改写守卫,曾让五份真实签到报告从 1820b 变成 274b)现在也带同样的前提。
2. 学习层的产出在每次拼提示词前被裁掉。 「我学到的教训」在前缀裁剪里排第一个被削 —— 连续十三轮都被削到地板。它不是历史累积物,它是反思通向提示词的唯一通道。现在它最后一个才被裁。真正在膨胀的是事实序列:一个复发任务每跑一次存一条记录,就占一个 fact 槽,两小时内吃满整个 top-N;于是 collapseFactSeries 按 key 词干只保留最新一条(更旧的仍留在库里)。
3. 一个 24 小时全局窗口被钉进了「本轮」信号里。 sameRootCauseFailures 在全局 24h 窗口上聚类失败,所以它描述的是系统最近一天的状态,而不是这一轮 —— 只要任何失败复发过一次,它就永远 ≥2,没有任何一轮能干净收尾(success=12 vs ambiguous_skipped=106)。已从强失败集合里移除;留下的都是同轮事件。(这个缺陷是第二次出现 —— 上一个占这个位置的信号在 2026-07-13 被移除,而这个只是搬进了空出来的位子。)

有一条更一般的教训值得单独写下来:一条学习回路是一堆 AND 门的乘积。 链路上任何一处的静默否决都会把整体归零,而在没有仪表的情况下,症状不是报错 —— 症状是什么都没发生。

技能自修复

agent-memory/src/skill_repair.ts · agent-memory/src/autonomous/drivers/skill_repair_driver.ts · agent-memory/src/autonomous/skill_revision_writer.ts · 设计文档

上面的反思流水线会把一个失效的 callable recipe 降级——但很长一段时间就停在这:降级后无人管。技能自修复补上了这最后一跳。

一个 callable recipe(带机器可校验的 verification + 工具策略的技能)每次复用都会被重新校验。一旦它没通过自己的校验,就被降级为纯建议——然后在空闲时,SkillRepairDriver 会接手:

1. 从账本诊断,而不是凭空想象。 这个 recipe 的真实失败轨迹直接从执行账本读出(ActionLog.getBySkill(..., {onlyFailed:true})),交给一次单轮 LLM 调用——一次调用,零工具。任务是有界的("这些是失败轨迹和 recipe 内容——根因是什么、怎么修"),不是开放调研,所以不走 deep_explore。
2. 改写,同时保留旧版本。 SkillStore.reviseRecipe 把旧的 steps/verification 存进 revision_history 再写入修复,并让 recipe 回落到 draft,逼它通过真实复用重新赚回信任。
3. 下次复用时再校验——当初抓到失效的那套复用校验,现在来审判这个修复。没有特殊通道。

防止修复帮倒忙的边界(2026-07-11 的一次生产运行里,模型第一次给"缺 pandoc"开出了 apt-get install 自愈方案——不可移植、越权,而且每次复用都会悄悄升级 recipe 对宿主做的事):

• 诊断提示词禁止假设 OS / shell / 包管理器,禁止安装/特权/改系统的修复。缺外部依赖时,正解是检测并报告,绝不安装。
• 诊断不足则什么都不改——recipe 保持降级即可。乱猜一个把失效 recipe 重新武装起来的改写,比不改更糟。
• 连续三次修复失败的 recipe 会被退休,不再被提案(revision_history 里的标记用于计数修复次数)。
• 它是唯一一个输出会改写可复用产物的自主 driver,所以保留自己的开关:PHILONT_SKILL_REPAIR=0(默认开)。

诚实的边界:这里记录的是机制。被修复的 recipe 后续复用是否真的比修复前更好,是一个生产问题,不是已证明的结论——revision_history 正是让它第一次可以按 recipe 度量的东西。若实地证明修复净损害,默认会翻回关;那是数据决策。