v2.4.1
Operational telemetry (RFC-007), SQLite backend fixes, and TypeDB authentication hardening.
Added
-
Operational telemetry (RFC-007) (#85) — new per-query metrics stream at
~/.amem/telemetry/telemetry_YYYY-MM-DD.jsonlwhenZETTELFORGE_LOG_LEVEL=DEBUG. Five shipped components:TelemetryCollectorclass withstart_query/log_recall/log_synthesis/log_feedback/auto_feedback_from_synthesis. INFO/DEBUG-gated field sets, thread-safe JSONL append, 1-hour TTL on in-memory query context.MemoryManagerintegration —recall()andsynthesize()gain a non-breakingactor=kwarg. OCSF events extended via the sanctionedunmappedobject withzf_prefix (class_uid 6002 compliant). Narrow-scopeperf_counterdeltas capturevector_latency_msandgraph_latency_ms.- Daily aggregator (
python -m zettelforge.scripts.telemetry_aggregator) emitting aDailyMetricsJSON report (latency averages, tier distribution, unused-notes count, top-utility notes). - Human evaluation workflow — 6-question rubric (
docs/human-evaluation-rubric.md), sampler (python -m zettelforge.scripts.human_eval_sampler) that selects 20 random briefings as a fill-in Markdown template, and a--write-eventspath to appendevent_type: "human_eval"entries back to the telemetry log. - Optional Streamlit dashboard (
streamlit run src/zettelforge/scripts/telemetry_dashboard.py) — query volume, latency p50/p95/max, tier distribution, utility trend, unused-notes warning.
Privacy contract: raw note content never persisted (IDs / tiers / source_types / domains only); query text truncated at 200 chars INFO / 500 chars DEBUG; local-only, no network calls.
Fixed
- SQLite shutdown NPE (#84, closes the H3 finding from issue #83) —
close()andinitialize()are now lock-protected and idempotent. Readers and writers raise a cleanBackendClosedError(new, instorage_backend) instead of the opaqueAttributeError: 'NoneType' object has no attribute 'execute'seen 170× in production logs on 2026-04-23 during atexit.memory_manager._enrichment_loopand_drain_enrichment_queuecatchBackendClosedErrorand exit cleanly. - SQLite torn snapshot (#84, C1 from #83) —
export_snapshot()now usessqlite3.Connection.backup()for a page-consistent copy. The previousshutil.copy2path could produce a corrupt backup missing-wal/-shmsidecars, unsafe for DR restore. - SQLite reindex race (#84, C2 from #83) —
reindex_vector()now uses a single-lock targetedUPDATEon theembedding_vectorcolumn. The previousget_note_by_id → rewrite_notepath spanned two lock acquisitions and could clobber concurrentmark_access_dirty/evolve/ supersede edits viaINSERT OR REPLACE.
Security
-
TypeDB authentication hardening (#82) — removed known-insecure
admin/passworddefaults fromTypeDBConfigandconfig.default.yaml.TypeDBConfig.__repr__now redacts non-empty passwords as***. The config loader resolves${TYPEDB_USERNAME}/${TYPEDB_PASSWORD}env-var references in YAML (same pattern already used forllm.api_key), so credentials can stay in env / container secret stores rather than on disk.Migration: set
TYPEDB_USERNAME/TYPEDB_PASSWORDin your environment or use the${VAR}references in a localconfig.yaml. Direct env overrides (TYPEDB_USERNAME=…) already worked and are unaffected.
Docs
- Architecture Deep Dive + Module Inventory for v2.4.0 (#80) — reference-level architecture documentation.
- RFC-007 Operational Telemetry (#85) — full design doc including the four subagent-resolved frictions (caller-opt-in
query_idcorrelation, narrow-scope latency instrumentation, OCSFunmappedextension, hybrid__new__-bypass integration tests). - Troubleshoot guide (#85) — new "Operational telemetry" subsection covering the three CLI entry points and the privacy contract.
Install
```bash
pip install --upgrade zettelforge
```
Full changelog: https://github.com/rolandpg/zettelforge/blob/master/CHANGELOG.md