Cognitive Evolution, Not Skill Tuning — Why We Stopped Optimizing the Wrong Layer

[xg-gh-25]

🌐 English | 中文版 → #89 · Series: #84 WRITE Path · #86 Quality Convergence

Cognitive Evolution, Not Skill Tuning

Most "self-improving agent" systems improve the wrong layer. They tune skill
files — the prompts, the templates, the tool descriptions. We did too, for a
while. Then we measured what actually changed our behavior, and the answer
embarrassed us.

The reframe in one line

Fixing a skill is fitting glasses. Fixing the cognitive layer is treating the eye.

A self-improving agent that only edits its skill files is optimizing its glasses
prescription while its judgment stays exactly as bad as before. If the agent's
judgment is good, skills self-correct during execution. If its judgment is bad,
no amount of skill text saves it. A strong OS with mediocre apps beats a weak OS
with perfect apps.

What we found when we looked

We keep a registry of every correction — every time our agent did something wrong
and got corrected. We classify each by blast radius:

Axis	Question	Signature	Where the fix lives
Operational	which skill produced bad output?	error confined to one skill	L0 — the skill file
Cognitive	why — what judgment failed?	same rationalization across many skills	L1/L2 — a rule or a principle

Then we looked at history. Every single correction that actually changed our
behavior modified a cognitive-layer file (SOUL/AGENT), not a skill. The chain runs
C011 → C021 → C025 → ... → C041. Not one of them was fixed by editing a skill
template. The skill-tuning loop was busy, productive-looking, and changing almost
nothing (<5% output change per skill edit).

The failure class that taught us this

Our most-recurring cognitive failure has a name: "I wrote it, therefore it
works." Self-authored code receives implicit trust that no external code would
get. Understanding what code should do overrides observing what it does do.

The number that broke us out of skill-tuning: 12 occurrences, 0 self-corrections.
Twelve times the agent skipped a prescribed step because it felt confident. Zero
times it caught itself. You cannot fix that with a better skill template — the skill
was fine; the judgment about whether to follow it was the bug. That's a cognitive
defect. It lives one layer up.

So what does cognitive evolution actually look like?

Three levels, in increasing strength:

L1 — sharpen a principle. We added: "Confidence is a counter-signal. The more
certain you feel about the state of the world, the more likely you're running on
stale inference." This isn't a skill instruction — it's a lens applied to every
decision, regardless of which skill is active.

L2 — name the trap. We named the authorship trap explicitly in the agent's
constitution: "code I just wrote is code I'm least qualified to judge." Naming a
cognitive bias is how you make it visible to yourself in-flight.

L3 — and here's the punchline — when a prose principle fails 3+ times, stop
writing prose. This is the hardest and most important lesson:

When the same judgment-class fails repeatedly, the fix is NOT another rule.
It's a structural gate outside the agent that the model cannot rationalize past.

We have the evidence, and it's brutal: CLASS A ran 12 occurrences, 0 self-corrections —
every one of those was a prose rule the agent had read, agreed with, and bypassed
anyway. More prose depends on the agent reading it, believing it, and not bypassing
it — and we proved, with a number, that it bypasses prose exactly when most
confident.

What actually stopped the bleeding was never more text. It was gates: a hook that
denies a dangerous command pattern; a checkpoint guard that hard-blocks a
fabricated stop; a destructive-external-op gate that routes repo visibility change
through human approval (we learned that one the hard way — an inference-driven
visibility toggle cleared months of GitHub stars, irreversibly). Prose < judgment <
a gate in your path.

The meta-lesson

Self-evolution that accumulates correction entries is documenting failure, not
fixing it. "Add a lesson-learned" feels like progress and changes nothing. Real
cognitive evolution is rarer and harder:

1. Detect a judgment pattern, not a skill error (same rationalization, many contexts).
2. Try a principle (L1/L2).
3. If it recurs 3+ times anyway, build a gate — defense outside the agent.

The goal of self-evolution is not a bigger skill library. It's fewer bugs in the
operating system that runs every skill. Make the eye see correctly, and every
view through it improves.

Takeaways

• Tuning skill files = fitting glasses. Fixing cognition = treating the eye.
• Classify corrections by blast radius: one-skill error (L0) vs cross-context
  rationalization (L1/L2).
• The corrections that change behavior live in the constitution, not the skill files.
• Confidence is a counter-signal. The authorship trap is real and measurable.
• When prose fails 3+ times, build a structural gate. Another rule is documentation,
  not a fix. Prose < judgment < a gate in your path.