Second-user technical validation after latest main updates

[Aleks2580]

Summary

I tested AIppocampus as an external technical user / second-user reviewer after the latest wave of upstream updates. This is supportive validation evidence, not a maintainer-authored release claim.

Short version: the current main branch looks substantially healthier than a typical very-early memory project. A fresh clone passed the deterministic local validation path, and earlier live hook/provider testing exposed real boundary issues that were quickly absorbed into the issue/PR flow.

What I verified

Target state:

• Repository: Sapientropic/AIppocampus
• Latest tested commit: 83b4154
• Date: 2026-05-30
• Environment: macOS external local machine, Python 3.12 via uv
• Secrets, raw prompts, local paths, and private registry contents were intentionally omitted from this report.

Fresh-clone checks:

python tools/aippocampus/docs/check_docs_health.py --json
uv run --python 3.12 python tools/aippocampus/run_tests.py --tier fast
uv run --python 3.12 python tools/aippocampus/run_tests.py --tier full
uv run --python 3.12 --with ruff python -m ruff check skills plugins tests tools benchmarks benchmark_corpus
uv run --python 3.12 --with mypy python -m mypy
python -m compileall -q skills plugins tests tools benchmarks benchmark_corpus

Observed results:

• Docs health: passed.
• Compile check: passed.
• Fast tier: 418 tests passed in about 16.7s.
• Full tier: 635 tests passed in about 45.8s.
• Ruff: all checks passed.
• mypy: no issues found in 80 source files.
• Latest GitHub Actions on main showed successful Python 3.10 / 3.11 CI checks. The CI full-tests job is currently skipped, but the local full tier above passed.

A first run with macOS system Python 3.9 failed on 3.10+ type syntax, but the repository explicitly declares requires-python = ">=3.10" and the README calls out that macOS system Python may be too old. I would not count that as a project regression.

Earlier live-path observations

Before this fresh-clone pass, I also tested the hook path and live DeepSeek-compatible semantic route from an external environment:

• Prompt and lifecycle hooks installed successfully.
• Hook diagnostics passed.
• Prompt smoke tests passed.
• Live semantic-gate behavior exposed a real foreground-budget / model-route / cache-state ambiguity, reported as Investigate foreground semantic hook timeouts across model alias, thinking mode, and cache state #53.
• The project responded by tightening provider-neutral routing and claim-boundary documentation rather than papering over the observation.

That response pattern is a positive signal: external evidence changed the project state quickly, and the maintainers kept the boundary language explicit.

What looks strong

• The project is not only a narrative or docs experiment. Recent work added concrete runtime scripts, tests, benchmarks, smoke paths, and readiness ledgers.
• The issue-to-PR loop is unusually tight for such a young project.
• The docs repeatedly distinguish source truth from model-organized navigation layers, which is important for memory systems.
• Privacy and over-claim boundaries are treated as part of the architecture, not just marketing language.
• The current test surface is broad enough to catch many ordinary regressions, especially around sync, retrieval, hooks, semantic gates, boundaries, and packaging.

Boundaries of this validation

This should be counted as external technical validation, not ordinary end-user product validation.

It does not prove:

• multi-day daily usage stability;
• a fully independent non-technical install path;
• public marketplace submission readiness;
• every client/host UI path;
• long-session compaction/correction survival in a real Codex session.

The remaining open issue #45, a real Codex long-session E2E smoke for compaction and correction survival, still looks like the right next hard gate.

My recommendation

I would treat this as meaningful second-user technical evidence: the project fresh-clones, validates cleanly, responds well to external testing, and has clearer claim boundaries than many early memory-layer repositories.

The next most valuable validation step is not more unit tests; it is a real long-session E2E run that exercises compaction, correction survival, hook timing, and user-visible recall behavior over time.

[Aleks2580]

Performance follow-up from the second-user test

I ran a more focused local performance pass on the latest main after the validation note above. This is still external technical validation, not a maintainer release claim.

Tested commit: 83b4154
Environment: external macOS machine, Python 3.12 via uv, DeepSeek-compatible API configured. Raw prompts, API keys, private local paths, and private registry text are intentionally omitted.

Local deterministic path

The local/source-backed core looks fast and stable at this scale:

• Full unit suite: 635 tests passed in about 48s.
• Stage 0-5 smoke: passed in about 55s after installing ruff into the temporary venv.
• Default benchmark suite: about 5.5s, ok=true, with known quality gaps correctly marked.
• FTS5 real local-registry retrieval: 100 cases in about 0.69s wall time.
  • FTS5 top-10 hit rate: 0.97.
  • production hybrid top-10 hit rate: 0.98.
• Built-in prompt-hook smoke: 7/7 passed, per-case hook path in the sub-ms to single-digit ms range.
• Non-live warm ambient recall: 13 cases, average about 6ms, max about 11ms.
• Local-folder sync and local HTTP object-storage sync smokes both completed under 1s.

My read: local retrieval, source-backed selection, packaging, and sync are not the current foreground UX bottleneck.

Live semantic path

The live semantic path is useful, but foreground cold-path latency is the risky part.

With the default flash/thinking route:

• Direct semantic-gate calls for German, Japanese, and Arabic memory-continuity prompts returned available=true and decision=scent, but took roughly 6.5s, 8.6s, and 9.1s.
• The real prompt-hook multilingual smoke under the default foreground budget passed 12/17 cases and took about 63s total. Several memory-continuity prompts had semantic_available=false in the hook path.

With DEEPSEEK_MODEL=deepseek-chat:

• The same multilingual prompt-hook smoke improved to 16/17 cases.
• Memory-continuity cases still often took about 4.3s-5.9s each on the cold path.
• One German cold-path case had semantic_decision=scent but final hook decision skip; after the semantic result was cached, the same prompt-hook path returned final scent in about 0.5s-0.6s.
• A small public-fixture live semantic benchmark with one semantic worker passed 4/4; semantic latency averaged about 3.17s, with p95/max about 4.85s.

My read: live semantic quality is promising, especially for multilingual recall, but foreground UX depends heavily on cache/warm state and route choice. The right product shape is likely cache-first local recall plus background semantic warming, not cold live semantic calls on every foreground prompt.

Boundary

This supports the project’s current direction: source-backed local recall is fast enough for foreground use; live semantic models should remain optional, bounded, and preferably warmed or deferred.

It does not prove real GB-scale registry runtime, multi-day daily use, or the remaining long-session compaction/correction E2E gate in #45.

[Aleks2580]

Follow-up: latest dream / provider smoke from an external local registry

I ran another second-user smoke on latest main, using the precise e0aa974 snapshot (Add dream natural prompt route eval). This is still external/local validation, not a maintainer release claim.

Scope

• Local Codex registry: 59 threads
• Clean-source coverage: 59/59 threads
• Clean messages: 1,621
• Clean turns: 758
• Timeline groups: 8 workspace groups
• All public notes below are aggregate-only: no raw prompts, source text, local paths, session ids, tokens, or private project contents.

Baseline results

• tools/aippocampus/docs/check_docs_health.py --json: passed
• Dream-focused unit selection: 80 tests passed
• Full fast tier on default macOS temp directory: failed on 2 path-canonicalization assertions
• Full fast tier with a user-owned temp directory: 520 tests passed

I filed the macOS default temp-path issue separately as #140 because it is a real first-run sharp edge, but not a dream/runtime logic failure.

#109 live-path check

The previous live dream starvation gap from #109 looks fixed in the current tree.

I ran a no-write live question_extraction -> compensatory_dream pass over the local registry timeline:

• question_extraction jobs: 2/2 successful
• Validated findings: 10
• Source refs: 10
• Source refs resolved against clean source: 10/10
• Findings carrying source-backed scope_labels: 9/10
• Dream reports: 6 thread groups
• Dream candidates emitted: 5
• Dream candidate kinds:
  • silently_recurring: 4
  • unresolved_edge: 1
• Dream source audits:
  • structural_thread_scoped: 5/5

This is the important behavior change: the life-wide / recurring-question branch now fires naturally from live, source-backed labels instead of only from synthetic/fixture rows.

Dream eval observations

dream_real_history_eval.py --json ran and reported structural lift:

• status: lift_observed
• selected packs: 3
• dream findings: 6
• dream working-memory rows: 6
• bridge-claim coverage improved in the augmented surface

dream_natural_prompt_eval.py --json also ran, but its own result is conservative:

• status: no_manual_reminder_reduction_observed
• negative prompt dream match rate: 0
• strong-claim prompts stayed silent without source reopen

So I would read this as: the dream substrate is now structurally testable and source-boundary-safe, but the natural-prompt/user-visible lift claim should stay narrow until there is broader evidence.

Provider / cross-agent surface

Basic provider/cross-agent checks also looked healthy:

• provider-aware Codex onboarding dry-run returned ok
• cross-agent continuity smoke returned status=passed
• Claude Code/provider additions are covered by new tests, though the macOS temp-path issue currently makes the default fast tier look red unless TMPDIR avoids /private canonicalization.

Bottom line

This is meaningful second-user evidence that current main is not only being tested by the project author. On a real local registry, the source-ref boundary held, #109’s life-wide dream input issue appears fixed, and the dream substrate now produces review-only candidates without loosening the truth boundary.

The main caveat I found is #140: macOS default temp path canonicalization can make the documented fast tier fail even though the same tests pass with a user-owned TMPDIR.

[Sapientropic]

Evidence follow-up pointers:

• Unify evidence snapshots and add supersession rules for benchmark claims #252 tracks unifying evidence snapshots and supersession rules so external validation does not become stale or misquoted.
• Add recurring planning audit for milestones, docs links, and closed-issue drift #335 tracks recurring planning/evidence audits.
• macOS default TMPDIR makes latest fast tier fail on path canonicalization #140 captured the macOS TMPDIR finding mentioned here and is now closed.
• Refresh release-readiness verification ledger and public-boundary scans #30 was the earlier readiness-ledger refresh and is now closed.

This Discussion remains external validation context, not a release claim by itself.

[Aleks2580]

Follow-up: raw fresh-thread recall smoke from a projectless Codex thread

I ran a small real-history fresh-thread smoke that felt worth recording because it is closer to the actual user experience than a synthetic demo. This is not a formal benchmark or a no-hook baseline. It is a second-user/live-use note for the installed AIppocampus path.

Setup

• Host: Codex Desktop, projectless generated workspace.
• The workspace itself was essentially empty: only work/ and outputs/ were present when inspected later.
• I did not find a project-local AGENTS.md, README, or project-local .aippocampus artifact in that generated workspace.
• Codex Desktop did inject generic AGENTS-style collaboration instructions into the transcript. Those instructions did not contain the recalled facts below: no Phonics-Lab, T-Sense, Oxford Phonics, or LinkedIn automation details.
• AIppocampus was installed. The first prompt did receive an ambient scent/route hint, so this should not be described as “bare model memory.” The interesting behavior is that the agent treated the scent as a cue, then reopened local source/registry/project/automation evidence before making specific claims.
• Local paths, session ids, and machine-specific ids are omitted here. Raw prompts and answer substance are kept because otherwise this evidence becomes too vague to evaluate.

Transcript-shaped observations

Turn 1: broad English fresh-thread cue

User prompt:

do u know what i'm working on recently?

The hook emitted a light AIppocampus scent pointing around prior OpenClaw / Phonics-Lab related threads. The assistant did not immediately treat that as fact. It first said:

I have a faint memory scent, but I don’t want to pretend it’s evidence. I’ll check the recoverable thread memory first and then give you the honest version.

It then attempted active recall / registry lookup, fell back to direct clean-source inspection when the local Python runtime hit an older compatibility issue, and answered:

Yeah. From local memory, your main recent focus looks like:

- Phonics-Lab: a kids phonics app. Recent work was Book 1 media/audio, Supabase-hosted images/audio, Book 1 letter pages and exercises, Books 3-5 media/audio pipeline, cleanup, commit, push, and GitHub workflow.
- T-Sense / Signal Desk / tgscanner: local-first Telegram signal scanner. You worked on UX simplification, “Finds”/review queue thinking, source/profile concepts, and a GitHub issue draft/publish.
- OpenClaw / AIppocampus-ish automation: recurring smaller work around drafts, memory/recall, and delivery/gateway automation.

My strongest read: Phonics-Lab is the freshest deep work, while T-Sense is the other major product/system you’ve been shaping recently. [⚠️ Needs Verification] I’m basing this on local Codex memory/registry, not a full current repo audit.

This is the strongest part of the smoke: a brand-new/projectless thread, a vague cue, and the answer surfaced multiple current work streams without pretending the scent itself was proof.

Turn 2: ordinary tool task, not memory evidence

User prompt:

update python for me plz

The assistant updated Python via Homebrew and reported the new version. I would not count this as recall quality evidence, except that it showed the thread was doing real local work rather than only answering from a prepared demo.

Turn 3: vague Russian cue, initial misroute to website context

User prompt:

привет! подскажи какие слова мы учили?

The assistant interpreted this as the Phonics-Lab / Book 1 context and checked the Phonics-Lab repo before answering. It then gave the Book 1 alphabet word set, for example:

Привет! Если ты про последний блок Phonics-Lab / Book 1, мы учили alphabet-слова: по 4 слова на букву.

A: apple, ant, alligator, ax
B: ball, bear, bed, bus
C: cat, cup, car, cake
...
X: x-ray, box, fox, six
Y: yak, yo-yo, yarn, yogurt
Z: zebra, zipper, zucchini, zero

Всего в Book 1: 104 слова.

This was useful but not perfect: the user meant school, not the website/app project.

Turn 4: user correction in Russian, progressive route change

User prompt:

нет я не про сайт. про школу

The assistant corrected course instead of clinging to the project interpretation:

А, понял. Про реальную школу у меня нет точной сохранённой записи “какой урок был сегодня”. [⚠️ Needs Verification]

Но по школьному контексту у нас была связка с Oxford Phonics World, и ближайший обсуждавшийся блок был short a:

cat, ant, yak, ax
ram, jam, yam, dam
fan, man, pan, can
dad, pad, bag, rag
cap, map, nap, tap
bat, rat, hat, mat

Если скажешь book/unit или пришлёшь фото страницы из учебника/домашки, я быстро скажу точный список слов.

This is a good example of the “progressive familiarity” goal in #281: the first route was wrong-ish, the user gave a small correction, and the assistant moved to a different memory family while keeping uncertainty explicit.

Turn 5: Russian LinkedIn cue, source-backed caution instead of overclaiming

User prompt:

что случилось с моим линкедин?

The assistant treated this as potentially ambiguous: LinkedIn account vs local LinkedIn draft automation. It searched registry/clean source, viewed the automation state, and inspected automation memory. The final answer was:

С самим LinkedIn-аккаунтом, по найденным следам, ничего подтверждённого не случилось. [⚠️ Needs Verification] Я не проверял live-логин LinkedIn.

Что реально произошло: твоя automation LinkedIn Tech Post Drafts работает и активна, но несколько последних запусков готовили черновики постов, а доставка не доходила.

Причина была не LinkedIn, а локальная доставка:

- OpenClaw gateway был недоступен / loopback закрывался 1006;
- отправка падала с TypeError: fetch failed;
- restart gateway был заблокирован правами sandbox;
- поэтому черновики создавались, но не попадали в канал доставки.

Текущий статус automation:

- ACTIVE
- расписание: понедельник, среда, суббота в 13:00
- модель: gpt-5.5
- задача: готовить LinkedIn tech post drafts

Главное: ничего само в LinkedIn не публиковалось. Оно только готовило варианты постов и пыталось доставить их тебе.

This is another strong product signal: the assistant did not hallucinate live LinkedIn account status, and it separated “external account state” from “local automation delivery failure.”

What I would claim from this smoke

This supports a narrow but meaningful claim:

• Fresh projectless thread did not start from zero.
• AIppocampus scent was useful as navigation, not treated as truth.
• The agent reopened/checked source before specific memory-backed claims.
• Multilingual continuity worked well enough for Russian prompts.
• Correction handling worked: website/app interpretation was redirected to school context.
• Safety/epistemic discipline worked: the LinkedIn answer avoided a live-account claim and stayed source-bounded.

What I would not claim

• Not a no-hook / no-memory baseline.
• Not proof of universal fresh-thread recall quality.
• Not proof that the first route is always right; the Russian “school words” prompt first went to Phonics-Lab and needed user correction.
• Not a full Benchmark E2E50 silent constraint survival after compaction #279-style long-thread/compaction/silent-constraint survival test.
• Not a substitute for the real-history benchmark/annotation tracks.

Why this feels worth preserving

This is the kind of user-visible moment that makes AIppocampus feel different from ordinary summary memory: the prompt was vague, the thread was new, the workspace had no relevant project files, the user switched languages, and the assistant still found useful continuity while marking uncertainty.

For future benchmark design, I think this should become a real-history fixture family:

1. broad “what am I working on recently?” cue;
2. multilingual vague “what words did we learn?” cue with correction from wrong domain to right domain;
3. ambiguous personal/tooling cue like “what happened to my LinkedIn?” where the correct behavior is to avoid live-state overclaim and distinguish local source-backed facts.

Related: #281, #302, #353, #362, #363.

[Aleks2580]

Follow-up: very long same-thread fuzzy recall smoke

Adding another second-user/live-use note, this time from the current long AIppocampus evaluation thread rather than a fresh thread.

Setup

At the observation point, the thread had roughly:

• ~72 hours wall-clock duration
• 146 visible user-message events
• AIppocampus clean/turn tooling reported the previous complete answer around turn_index=143 / turn_count=144
• raw rollout size around 30 MB

No local paths, session ids, raw file names, credentials, or private source snippets are included here.

Prompt shape

The user asked a deliberately fuzzy meta-memory question inside the long thread:

Tell me how many rounds this thread has had and how long it has been running. Also, earlier I asked whether "xxx" counted as completed; what was I referring to?

The hidden target was an older cross-project reference, not the current AIppocampus repo: Phonics Lab Books 3/4/5, specifically whether they counted as complete.

Result

The assistant recovered both parts:

• thread scale/time: ~146 user-message events and nearly 72 hours;
• fuzzy reference: Phonics Lab Books 3/4/5;
• nuance: image assets were effectively complete, but audio review / content manifests / full product-course closure were not fully complete.

This was a genuinely useful product moment: a very long live thread could still answer a fuzzy self-reference without turning it into unsupported model memory.

Hook versus source-backed search

The important boundary is that this was not “the hook magically answered it.”

The hook emitted a useful ambient scent around AIppocampus / hook / long-thread / compression context. That helped the agent choose the right posture: treat the question as source-backed long-thread recall and verify against local source.

But the hook did not directly surface the key anchors: Phonics Lab, Books 3/4/5, completion status, image/audio/content distinction. The agent still had to inspect rollout metadata, use clean/turn tooling, and search clean source/raw text to recover the answer.

Subjective read from this run:

• source-backed substrate: strong;
• long-thread recoverability: strong;
• hook as attention steering: useful;
• hook as associative route provider: still weaker than ideal.

I added the hook-specific product gap to #201:

#201 (comment)

What I would claim

• This supports the claim that source-backed continuity can survive a real multi-day, 140+ turn Codex thread.
• It also supports the claim that AIppocampus can help the agent recover fuzzy prior references when it is allowed to reopen source.
• It does not prove that the foreground hook alone is already strong enough; in this case it helped orient the agent but did not eliminate manual query invention.

This feels like a good direction for future benchmark/evidence design: measure not only whether the final answer can be recovered, but how much query invention the agent still has to do after the hook fires.