Design Decisions

Design Decisions

This page records the implemented architectural choices and the evidence gates for future changes.

Separate immutable search artifact

Decision: project and enrich in a Slackquery-owned state database, then copy a complete serving subset into a new immutable DuckDB artifact.

Why: DuckDB is not used here as a shared concurrent writer/reader serving database. Immutable artifacts isolate readers from pipeline mutation, permit fresh-process validation, make checksums meaningful, and allow atomic rollback.

Rejected: serving directly from canonical Slackpipe or mutable enrichment state.

Canonical ownership remains with Slackpipe

Decision: Slackquery attaches the canonical Slack DuckDB read-only and writes only its configured state and artifact directories.

Why: this preserves a single canonical owner and prevents search enrichment from changing ingestion truth.

One generalist embedding model

Decision: use one nomic-embed-text:v1.5 vector space for messages, file chunks, and queries. Thread-context documents contribute contextual candidates without requiring duplicate vectors.

Why: one space permits direct semantic comparison. A separate message model and file model would create incompatible spaces even if dimensions happened to match, requiring separate retrievers and rank fusion.

Pinned vector generation identity

Decision: production deployments should pin the exact server digest, native dimension, first-512 truncation, L2 normalization, distinct query/document prefixes, and recipe versioning.

Why: model names and endpoint URLs do not guarantee vector compatibility. The full transformation contract does.

Consequence: PyTorch CUDA and Ollama can share existing vectors when every generation-defining property matches. Transport itself is not identity.

Exact cosine before ANN

Decision: use exact cosine scan over FLOAT[512] vectors.

Why: at representative message corpus active documents, measured semantic p95 is 259.97 ms and hybrid p95 is 309.69 ms, meeting the current interactive LAN target. Exact retrieval is simple, deterministic, and the reference for any future ANN recall evaluation.

Deferred: persisted DuckDB VSS/HNSW or another vector engine. Introduce ANN only after measured latency/cost violates an accepted SLO and Recall@100 against exact retrieval meets the agreed target.

FTS/BM25 plus semantic retrieval

Decision: keep independent lexical and semantic retrievers.

Why: Slack contains exact incident strings, IDs, URLs, filenames, code, and acronyms where lexical matching excels, plus paraphrases and conceptual questions where embeddings help.

Reciprocal rank fusion instead of raw-score addition

Decision: route exact, conceptual, and mixed queries, then fuse lexical, semantic, and contextual candidate lists with weighted RRF, k=60.

Why: BM25 and cosine scores have incompatible, query-dependent scales. RRF uses rank, avoids brittle score calibration, and handles candidates missing from one list naturally.

Caveat: this is a baseline to evaluate. The human-judged relevance benchmark has not been completed, so RRF superiority is not claimed.

Atomic messages as retrieval units

Decision: embed and rank individual messages, then retrieve bounded channel or thread context on demand.

Why: atomic messages preserve precise citations, avoid repeating entire threads in every vector, reduce edit cost, and reduce truncation of the actual matching reply.

Silver candidate: evaluate root+reply or thread-level representations as separate retrievers without discarding atomic retrieval.

Structured MCP instead of SQL access

Decision: expose four bounded, read-only tools over MCP Streamable HTTP.

Why: structured tools enforce query limits, parameterized filters, stable records, provenance, and a smaller security surface than arbitrary SQL or paths.

Durable state in DuckDB, not Dagster events

Decision: store per-item embedding state, retries, leases, vectors, and build history in /srv/slackquery/state/slackquery.duckdb.

Why: work survives process/run restarts and can be resumed without treating orchestration metadata as the data store.

Bronze, Silver, and Gold

These levels are promotion gates, not marketing labels.

Bronze: current implementation, incomplete promotion

Implemented scope:

• messages only;
• one verified 512-dimensional generation;
• immutable artifact;
• FTS + exact cosine + RRF (k=60, window 100);
• four MCP tools;
• manual and hourly reconciliation;
• checks, checksum, atomic publication, and rollback retention.

Evidence status:

• corpus/vector coverage is representative message corpus/representative message corpus in the validated artifact;
• fresh read-only/checksum validation passed;
• latency and build time were measured;
• human relevance evaluation remains outstanding, so Bronze promotion is not complete.

Silver: partial foundations only

Existing foundations include content-addressed incremental embeddings, durable retry state, and scheduled repair. Still required are judged contextual thread retrieval, file chunks when source extraction exists, dual-generation migration operations, quality regression gates, and stronger freshness automation.

Gold: future and evidence-driven

Gold candidates include ANN, reranking, calibrated routing, multiple learned or specialized retrievers, blue/green MCP serving, retained quality history, alerting/SLO ownership, and rehearsed disaster recovery. None should be promoted without judged gain or operational need.

Decision triggers

• Add ANN only when exact scan misses an accepted latency/cost SLO.
• Add a reranker only when judged quality gain justifies added latency and complexity.
• Add file chunks only when attachment text extraction is actually available and independently evaluated.
• Change RRF parameters only from recorded judged evidence.
• Change embedding generation only with explicit migration and complete artifact separation.

See Testing-and-Benchmarks and Architecture.