Evaluation Status

🌐 English | 中文

Evaluation Status

Philont currently has strong implementation evidence—tests, execution ledgers, dogfood logs, and regression cases—but limited product-level and longitudinal evidence.

What the repository can demonstrate today

• policy, grant, validator, and audit components have automated tests;
• execution and plan outcomes are recorded rather than inferred only from final prose;
• historical failures are reintroduced to verify that regression guards actually fire;
• the learning judge runs in shadow and records verdicts;
• skill recipes carry validation, demotion, revision, and repair machinery.

What is not yet established

• that the same lower-cost model has a higher verified task-success rate with Philont than with a minimal agent loop;
• that total cost per verified success is lower than a frontier-model baseline;
• that repeated use measurably reduces human correction;
• that learned rules and repaired skills improve unseen future tasks;
• that application-layer policy constitutes a hardened containment boundary.

The next evidence milestone

The immediate goal is not a large synthetic leaderboard. It is a transparent dogfood report built from real daily work:

• tasks with explicit success criteria;
• verified success, partial, failed, and not-applicable outcomes;
• false-completion catches;
• authorization and recovery events;
• human interventions;
• total tokens, latency, and estimated cost;
• representative failures, not only successful demos.

Once repeated task types exist, selected episodes can become deterministic fixtures for lower-cost baseline, Philont, frontier baseline, and mechanism-ablation comparisons.

中文

English | 中文

评测状态

Philont 当前拥有较强的实现证据——测试、执行账本、dogfood 日志和历史回归案例——但产品级和长期纵向证据仍然有限。

仓库今天能证明什么

• 权限、grant、validator 与审计组件拥有自动测试；
• execution 与 plan 结果会被记录，而不只从最终文字推断；
• 历史缺陷会被重新植入，以验证回归护栏真的会触发；
• learning judge 以 shadow 模式运行并记录判定；
• 技能 recipe 具备验证、降级、修订和修复机制。

尚未证明什么

• 同一个低成本模型使用 Philont 后，验证任务成功率高于最小 Agent loop；
• 每次验证成功的总成本低于前沿模型基线；
• 重复使用会可测量地减少人工纠正；
• 学到的规则和修复后的技能能改善未见过的未来任务；
• 应用层权限策略构成强化隔离边界。

下一项证据里程碑

眼下的目标不是制作大型合成排行榜，而是根据真实日常工作发布透明的 dogfood report：

• 为任务记录明确成功条件；
• 记录验证成功、部分成功、失败与不适用；
• 记录虚假完成拦截；
• 记录授权与恢复事件；
• 记录人工干预；
• 记录总 token、延迟与估算成本；
• 公开代表性失败，而不只展示成功 demo。

当出现稳定的重复任务类型后，可以把其中一部分固化为确定性 fixture，用来比较低成本基线、Philont、前沿模型基线以及机制 ablation。