1.6.0

This release adds HTML support and fixes several bugs that were silently losing text from indexed documents.

• HTML files (.html, .htm, .xhtml) can now be indexed. Add semanticsearch.NewHTMLStrategy() to your strategy list
• Pages are split by their headings, so results carry the headings they sit under. Tags, scripts, styles, and navigation menus never reach the index
• Fixed a bug where a heading with no paragraph text under it was dropped completely, which left documents like lab reports and spec sheets almost unsearchable. This affected Markdown, Word, and PDF too
• Fixed a bug where a section's heading was left out when its text was turned into a vector. A section titled "Refunds" whose text reads "Within five business days" is now findable by searching for refunds
• Fixed garbled PDF extraction, where letters came out spaced apart or a line was broken into pieces. Across a 60 file corpus this recovered 686 more words and removed 3289 stray single letters
• Fixed PDFs that fake bold by drawing text twice, which produced doubled letters

1.5.0

• Chunks are now buffered across documents and sent to the embedding server in batches instead of one request per document, cutting HTTP round trips on remote embedding endpoints.
• Added IndexOptions.EmbedBatchSize to control the batch size. Leave it nil for the default of 50; set a positive value to override (1 sends one chunk per request). Zero or negative is rejected.
• Batches contain whole documents and succeed or fail as a unit. A failed batch leaves its documents in the chunked state for the next run to retry.
• Added IndexOptions.OnProgress to report indexing progress as (phase, done, total) across the scanning, indexing, and cleanup phases.
• Added the examples/progress program showing progress reporting during an index run.
• Documented batching, change detection, and file identity behavior in docs/architecture.md.
• Raised the fingerprint and process document page sizes from 1 to 100, reducing per document database round trips during indexing.
• Expanded test coverage of storage, search, and fingerprint error paths.

1.4.1

• Raised test coverage across strategies, pipeline, embedder client, and the facade
• Added Codecov coverage reporting
• Fixed a Windows CI failure by making the cleanup stat test cross-platform

1.4.0

• Re-indexing now removes documents whose files were deleted from disk, along with their chunks and vectors, so the index stays in sync with the directory.
• This pruning runs by default at the end of Engine.Index; set IndexOptions.KeepMissingFiles to true to retain documents for missing files.
• Added a Cleanup pipeline that pages through stored documents and removes only the ones whose file is confirmed missing, leaving present files and ambiguous stat errors untouched.
• Deletes a document's vectors before its metadata, so an interrupted cleanup cannot strand a document whose vectors are already gone, and a re-run finishes it.
• Added DocumentsFromID and DeleteDocument store methods for both the SQLite and PostgreSQL backends.

1.3.0

• Search now returns documents instead of chunks: Engine.Search takes a SearchConfig and returns DocumentResults, each carrying the matched chunks ranked best first.
• Added SearchConfig with Query, TaskType, MinRelevance, MaxDocuments, and MaxChunks, using plain values with defaults of keep all, 20 documents, and 3 chunks per document.
• Result scores are now a 0 to 1 relevance (higher is closer), consistent with MinRelevance.
• Made the search algorithm replaceable through a public search.Searcher interface, injectable via Config.Searcher.
• Added a public core/search package holding SearchConfig, SearchResult, and DocumentResult.
• Capped the internal chunk fetch at 4096 (sqlite-vec's KNN limit) so search stays bounded on large indexes.
• Added three runnable examples (basic, searchconfig, postgres) with a shared sample file set.
• Refreshed the README, architecture, and chunking docs to describe the search flow, and trimmed the todo to the format-support roadmap.

1.2.0

• Added five predefined embedding models: Nomic v1.5, Multilingual E5 large, BGE large en v1.5, Qwen3 Embedding 0.6B, and mxbai embed large v1, each selectable through NewModel.
• Added NewGeneralModel for template-free models, so any OpenAI-compatible model or vector size works without a dedicated type.
• Added caller-controlled query task type. Search accepts an optional task, and each task-aware model exposes its own tasks (for example TaskGemma, TaskNomic); retrieval-only models reject a task instead of silently ignoring it.
• Split the embedder into two injected parts: a transport client (AiClient) that owns the wire protocol and a model (EmbeddingModel) that owns the prompt templates.
• Reorganized the embedder into client and model subpackages.
• Moved the search composition into the internal pipeline package, matching how indexing is structured.
• Renamed the Embedder interface to AiClient (breaking).
• Renamed OpenAIEmbedder to OpenAIClient (breaking).
• Renamed Model to EmbeddingModel (breaking).
• Removed the default embedding server base URL, which must now be set explicitly (breaking).
• Added a README architecture diagram, a "What semantic search is" intro section, and a table of contents.

1.1.0

• Added a PostgreSQL metadata store (NewPostgresStorage) using the pure-Go pgx driver, with no cgo required.
• Added a pgvector vector store (NewPostgresVectorStorage) for server-side, meaning-based search.
• Added an HNSW option for approximate search at scale, selectable with PostgresKNN (exact) or PostgresHNSW (approximate).
• Enabled mixing backends, so metadata and vectors can live in the same database or on separate servers.
• Added an optional API key (sent as a Bearer token) and a per-request timeout to the AI embedder.
• Documented in-memory SQLite usage for single-process, RAM-only setups.
• Added a docker-compose file (pgvector) and a CI job that tests against a live pgvector service.
• Added a golangci-lint job to CI so the pipeline covers build, tests, and lint.
• Reworked the README with use cases, badges, clearer requirements, and shorter usage examples.

1.0.0

• Semantic search over your files: meaning-based retrieval powered by AI embeddings and a vector database, not keyword matching.
• Go library, injectable by design: compose an Engine from an embedder, a metadata store, a vector store, and strategies; bring your own implementation of any interface.
• Multi-format indexing: built-in strategies for PDF, Markdown, source code, DOCX, and plain text, each parsed and chunked to fit its structure.
• Structure-aware chunking: headings, functions/classes, and document sections become titled chunks, sharpening retrieval relevance.
• Source code understanding: per-definition chunks for Go, JavaScript, TypeScript, Python, PHP, Java, Rust, C/C++, C#, and more, titled with their nesting path.
• Pluggable embedders: configurable OpenAI-compatible embedding API that works with local servers like LM Studio and Ollama, or any hosted model.
• Swappable storage: SQLite metadata store and sqlite-vec vector store included; point them at separate databases, or implement your own backend.
• Exact vector search: k-nearest-neighbor (kNN) search over embeddings for precise, full-recall results.
• Incremental delta indexing: re-running only re-embeds changed files, detected by content hash.
• Pure-Go format parsing: no external binaries; PDF via WebAssembly, code via a pure-Go lexer, DOCX via the standard library.
• A self-hostable backend for meaning-based document search: a building block for retrieval-augmented generation (RAG) and NotebookLM-style search over your own content.