Closing the Learning Loop: An Incremental Proposal for Soul Evolution

[virtualian]

Following up on #531 ("Is the Learning System Actually Learning?"), where @danielmiessler confirmed that the final piece to close the loop is needed and coming. This is a concrete, incremental proposal for how to get there.

Also related: #527 (PAI Memory System) explores a more comprehensive memory architecture. This proposal is complementary—focused on getting captured learnings back into the session context with minimal changes.

---

Context

PAI's capture infrastructure is impressive. Between ExplicitRatingCapture, ImplicitSentimentCapture, WorkCompletionLearning, and FailureCapture, the system writes rich learning data to MEMORY/LEARNING/ and MEMORY/LEARNING/SIGNALS/. The hooks are reliable, the data is structured, and the statusline surfaces it beautifully.

The missing piece, as Daniel noted in #531, is the read-back. LoadContext.hook.ts loads SKILL.md, AI Steering Rules, and progress files — but never touches LEARNING/, SIGNALS/, or SYNTHESIS/. The assistant starts each session without awareness of past mistakes or patterns.

SoulEvolution.hook.ts in v2.5 takes a step toward closing this loop by reading recent learnings and proposing updates to a persistent identity file. The concept is exactly right. In practice, it currently targets a path (KAI.md) that isn't shipped in releases, so it exits silently on most installations. Fixing the target path would bring it to life.

---

Proposal: Three Incremental Levels

Each level builds on the last. Each is valuable in its own right and can be shipped separately.

Level 1: Passive Recall (smallest change, biggest immediate impact)

At session start, LoadContext.hook.ts reads recent learning files and injects a compact summary (~200 tokens) into context.

LoadContext currently loads:
  ✅ SKILL.md
  ✅ AI Steering Rules
  ✅ Progress files
  ❌ Recent learnings ← add this

What to inject: The last 7 days of low-rating learnings, summarised into something like:

"Recent patterns: user frustrated by incomplete solutions (×3), overstepping boundaries (×1). Prioritise completeness and ask before making assumptions."

Why this works: The assistant now starts each session aware of its recent failure patterns. It stops repeating the same mistakes. ~50 lines added to an existing hook.

Level 2: Active Synthesis

LearningPatternSynthesis.ts already exists but has never been run (the SYNTHESIS/ directory is empty on most installations). Schedule it to run periodically — either via cron, or triggered at session start if the last synthesis is stale.

The output (SYNTHESIS/YYYY-MM/weekly-patterns.md) replaces raw learning injection from Level 1. Instead of listing individual incidents, the assistant gets compressed behavioural guidance: top 3-5 patterns, what triggers them, and what to do differently.

This scales over months without growing context usage.

Level 3: Identity Evolution

The existing SoulEvolution.hook.ts concept—updating a persistent identity/soul file based on accumulated evidence—is the right long-term architecture. Two changes would bring it to life:

1. Make the target path configurable (from settings.json) instead of hardcoded. This way, each installation's DA identity file evolves based on its own interaction history.

2. Feed synthesis (Level 2) into the evolution logic, not raw learning files. This gives the evolution hook compressed, high-signal input rather than noisy individual incidents.

The tiered update authority already designed into SoulEvolution is exactly right:

• Auto-update: behavioural learnings, known pitfalls
• Require approval: core identity, values, voice

---

Signal Quality (Optional but Helpful)

The implicit sentiment hook currently defaults neutral messages to rating 5, which floods ratings.jsonl with noise (~86% neutral in my installation). Two suggestions:

• Skip logging neutral sentiment entirely — only write when the signal is genuinely non-neutral
• Weight explicit ratings higher in synthesis — a user typing 2 carries far more signal than Haiku inferring a 5

These aren't blockers for the proposal above, but they'd make the data cleaner for all downstream consumers.

---

Adoption Path

Phase	What	Effort	Impact
1	Passive Recall in LoadContext	Small (~50 lines)	Immediate: assistant aware of recent failures
2	Schedule LearningPatternSynthesis	Small (cron or hook trigger)	Compressed patterns replace raw incidents
3	Fix SoulEvolution targeting	Small (configurable path)	Hook works on all installations
4	Connect synthesis → evolution	Medium	Persistent self-improving identity

Phase 1 alone would be a meaningful improvement.

---

Related discussions:

• Is the Learning System Actually Learning? (USER/ files + ratings) #531 — Is the Learning System Actually Learning?
• PAI Memory System [v0.2] #527 — PAI Memory System [v0.2]
• 🚨 🚀 Big Improvements Coming to PAI in 3.x and Beyond #500 — Big Improvements Coming to PAI in 3.x and Beyond

---

Written with help from Sonny (PAI), Claude Code, and Opus 4.6

[mschroettle]

Thanks for this summary. It addresses what I have experienced and got me frustrated to the point of killing my DA altogether. It would just stubbornly make the same mistakes over and over.

I have found DAIDENTITY.md in my skills/PAI/USER/ directory. Isn't this the one resembling a generic KAI.md. If so, the fix for Level 3 would be easy.

Someting like the PAI memory system sounds very nice. Have you tried it?

I think I will start small by implementing Level 1 and Level 2 and testing the Level 3 fix. At least there is some hope again :-)

[virtualian]

Great!

I've not tried it yet but will asap.

Let me know how you get on an let's compare notes.

[danielmiessler]

This is now addressed — see the reply in #531. v3.0 ships the MineReflections and AlgorithmUpgrade workflows in the PAIUpgrade skill, which close the loop from signal capture → pattern extraction → algorithm spec improvement. The Soul Evolution framing here is good and consistent with that direction.

[virtualian]

@danielmiessler, sorry to bring this up again, but as of v4.0.1, I think the Learning Pipeline dead ends.

See #850

And what are your plans to make the loop automatic?

[danielmiessler]

Fixed and added to 4.0.3 release.

…

On Sun, Mar 01, 2026 at 5:07 PM, virtualian < ***@***.*** > wrote: @ danielmiessler ( https://github.com/danielmiessler ) , sorry to bring this up again, but as of v4.0.1, I think the Learning Pipeline dead ends. See #850 ( #850 ) — Reply to this email directly, view it on GitHub ( #632 (reply in thread) ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AAAMLXRJBNVS3CHOF4WO4BT4OTNGNAVCNFSM6AAAAACUVNMJAGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKOJWGU4DANY ). You are receiving this because you were mentioned. Message ID: <danielmiessler/Personal_AI_Infrastructure/repo-discussions/632/comments/15965807 @ github. com>