Repository navigation
enigma-memory v0.1.5
Official standard benchmark harness release.
Install:
npm install -g enigma-memory
Benchmark datasets:
npm run benchmark:datasets -- --dry-run
npm run benchmark:datasets -- --execute --dataset all --out-dir .enigma/benchmarks/datasets --manifest .enigma/benchmarks/dataset-manifest.json
npm run benchmark:standard -- --locomo .enigma/benchmarks/datasets/locomo10.json --longmemeval .enigma/benchmarks/datasets/longmemeval_s_cleaned.json --max-locomo-qa 25 --max-longmemeval-items 25 --top-k 5 --out .enigma/benchmarks/standard-memory-benchmark-sample.json
Highlights:
- Adds official LoCoMo downloader support.
- Adds LongMemEval oracle/S/M downloader support with streaming large-file downloads.
- Adds public-safe retrieval/evidence proxy benchmark runner.
- Adds enhanced deterministic Enigma relevance over simple keyword filtering.
- Adds reproducibility docs and optional CI example.
Observed sample result:
- LoCoMo sample hit@5: keyword 0.28, Enigma relevance 0.52.
- LoCoMo sample exact coverage: keyword 0.28, Enigma relevance 0.36.
- LongMemEval-S sample estimated tokens: keyword 60,969, Enigma relevance 55,689.
- LongMemEval Oracle sample estimated tokens: keyword 54,066, Enigma relevance 46,939.
Verification:
- standard benchmark downloader/runner tests pass
- datasets downloaded with SHA-256 manifest
- standard sample reports generated
- npm run check
- npm test
- npm pack --dry-run
- npm publish --access public --dry-run
- npm publish --access public
- npm run registry:verify -- --execute
- registry install smoke
- GitHub CI passed on Ubuntu and Windows
Boundaries:
Retrieval/evidence proxy only. No LLM answer accuracy, provider deletion proof, model forgetting proof, ROI/savings guarantee, compliance certification, third-party competitor score, or benchmark-leadership claim.