AMIS

AMIS — Ankh-Morpork Intelligence System — is a local-first retrieval and evaluation system for literary EPUB corpora. It implements deterministic ingestion, citable chunking, semantic and lexical retrieval, rank fusion, optional guarded reranking, passage-free evaluation, and a local Ollama-backed grounded-answer CLI with schema-constrained, citation-validating output.

At A Glance

Stack	Python 3.13, NumPy exact cosine, Sentence Transformers/PyTorch, BM25, Reciprocal Rank Fusion, Transformers cross-encoders, optional local MLflow
Selected models	google/embeddinggemma-300m embeddings; Alibaba-NLP/gte-reranker-modernbert-base guarded reranking family
Public evidence	Crime and Punishment benchmark v2: 90 intents, 450 query variants, 400 default-scored variants
Stage	M2 retrieval quality complete; local grounded answering is available with an explicitly installed Ollama model

Explore: BM25 demo · semantic/full route · local grounded answers · architecture · retrieval evidence · benchmark artifacts

Model Choices

Model	Position	Evidence-based rationale
google/embeddinggemma-300m	Selected embedding	Revision-pinned gated model; Sentence Transformers/PyTorch; 768-dimensional normalized vectors; 2,048-token contract; supports the accepted 3,000/4,000/400-character chunk policy
BAAI/bge-small-en-v1.5	Evaluated lightweight fallback	Valid FastEmbed/Qdrant ONNX path with 384-dimensional vectors; its 512-token window required a smaller experimental 1,200/1,650/150-character policy; not the runtime default
Alibaba-NLP/gte-reranker-modernbert-base	Promoted practical reranker family	Guarded source/reranker rank fusion is promoted; direct replacement is not
BAAI/bge-reranker-v2-m3	Measured comparator	Slower and weaker than GTE for the final public decision; direct BGE reranking was negative
cross-encoder/ettin-reranker-68m-v1	Repaired diagnostic	The correct Sentence Transformers adapter made scoring viable, but a limited smoke did not justify promotion

These are bounded engineering and benchmark decisions, not universal model superiority or multilingual claims. The public reranker runtime accepts an explicit compatible local model and does not hash-verify it as the measured GTE snapshot.

Retrieval Quality

On 400 default benchmark-v2 query variants:

• vector retrieval leads source-only methods at 375/400 lenient Hit@20, 0.642479546 MRR@20, and 0.5115838 graded nDCG@20;
• BM25 reaches 330/400 lenient Hit@20 and provides asymmetric lexical complementarity;
• hybrid RRF is competitive at 374/400 lenient Hit@20 and leads strict Hit@20/50 plus lenient Hit@50;
• selected guarded GTE fusion reaches 379/400 lenient Hit@20 and 0.693721535 MRR@20 with zero top-rank regressions on this benchmark.

Direct reranking is not promoted. Direct GTE produced seven top-rank regressions; direct BGE lost 13 Hit@20 cases and produced 31.

Implemented Pipeline

public or user-supplied EPUB
  -> deterministic sections and chunks
  -> exact semantic index and/or BM25
  -> optional hybrid RRF or guarded reranking
  -> traceable citations with bounded excerpts

The checked-in benchmark artifacts contain questions, summaries, identifiers, coordinates, hashes, and metrics—but no source passages, generated chunks, retrieved excerpts, full rankings, vectors, model files, or local paths.

Quick Start

AMIS requires Python 3.13 (>=3.13,<3.14).

python3.13 -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev]"
python -m amis

The lightweight BM25 demo requires no embedding model. The semantic and full route adds explicit EmbeddingGemma acquisition, vector indexing, hybrid search, and optional guarded reranking.

amis ask is a local-only, vector-backed grounded-answer path. It requires an explicit installed Ollama model and local vector artifacts; it never pulls a model, permits a remote-backed Ollama model, sends evidence to a cloud endpoint, or treats citation membership as a semantic-faithfulness guarantee. Its current numeric defaults are operational baseline values, not quality-selected settings. See Local Grounded Answers.

Documentation

• Architecture
• Getting started and public demo
• Retrieval quality
• Reranking
• Evaluation methodology
• Local MLflow tracking
• Semantic index and model lifecycle
• Public-domain source and legal caveats

Development

python -m pytest
ruff check .
ruff format --check .
python tools/render_public_benchmark_charts.py --check

Keep copyrighted source files, local datasets, generated indexes, model files, and experiment outputs outside the tracked repository. See CONTRIBUTING.md.