v0.5.1

Fixed

Documentation now points at ragmill setup-chat everywhere.

0.5.0 added the command and corrected the three files that printed the broken pip invocation, but left pip install "ragmill[chat]" — a source build, and the command that fails on Windows — as the headline route in nine other places:

• the README install list, the chat-backend tables in both the README and the chat guide, the README chat snippet, and the extras table in the installation guide
• the quickstarts in docs/index.md, docs/quickstart.md and the README CLI list, which went straight from pip install to ragmill chat and so walked a new reader into the missing-model error
• the CLI reference table, which did not list setup-chat at all

Two tests now enforce this: no document may print the wheel-index command without --only-binary, and any quickstart showing ragmill chat must also mention setup-chat. Checking by hand caught three files last release and missed nine.

Getting started

pip install "ragmill[all]"
ragmill setup-chat
ragmill ingest /path/to/your/documents
ragmill chat

Docs and version only — no source changes.

v0.5.0

Added

ragmill setup-chat installs the local chat model runtime for you:

$ ragmill setup-chat
This will install the local chat model runtime:

  package:  llama-cpp-python (prebuilt wheel, no compiler needed)
  from:     https://abetlen.github.io/llama-cpp-python/whl/cpu
  command:  pip install llama-cpp-python --extra-index-url … --only-binary llama-cpp-python

That index is maintained by the llama-cpp-python author and is not PyPI.
It is needed because the package publishes no wheels to PyPI itself.

Continue? [y/N]

--yes skips the prompt, and is required when stdin is not a terminal, so it cannot install unattended by accident. It installs into the interpreter running RAGMill rather than whichever pip is first on PATH, and confirms the module imports afterwards instead of trusting pip's exit code. If it fails, it names the likely cause — a Python version with no wheel on that index, which covers 3.8 to 3.13 — and points at the hosted backends.

This deliberately is not a post-install hook. A wheel is unpacked, never executed, so Python packaging has no such hook — by design, after npm's postinstall became a supply-chain attack vector. Making it an explicit command also keeps the non-PyPI index visible and consented to, the way playwright install and python -m spacy download do.

Fixed

The documented command for installing the local model did not work.

pip install llama-cpp-python --extra-index-url https://abetlen.github.io/…
  → Downloading llama_cpp_python-0.3.34.tar.gz (71.6 MB)

--extra-index-url merges both indexes and pip picks the highest version across them — and PyPI carries a newer sdist-only release than the wheel index carries wheels. So pip chose the source archive, and the documented workaround for the Windows build failure triggered that very failure.

Adding --only-binary llama-cpp-python makes pip skip versions that have no wheel: it now fetches a 3.4 MB wheel with no compilation. Corrected everywhere the command appears — the CLI error, README, installation guide and chat guide.

If you were given the old command, use ragmill setup-chat or add --only-binary llama-cpp-python.

Upgrading

pip install --upgrade "ragmill[all]"

No breaking changes. Minor version bump because this adds a CLI command.

v0.4.3

Changed

ragmill chat now fails cleanly when the local model is not installed.

Since llama-cpp-python left [all] in 0.4.1, this is the first thing a new user meets — and it was handled badly in two ways.

It raised from inside the REPL loop, so the failure only surfaced after you had typed a question. The question was discarded, and an ordinary "optional package not installed yet" state was reported as a Python traceback. The backend is now checked before the prompt opens, and the command exits with status 1 and a message.

The message also pointed elsewhere. It offered the Gemini and OpenAI backends — but someone running the local backend asked for local chat, and a hosted service that needs an API key answers a different question. It also suggested pip install "ragmill[chat]", which is a source build and precisely what fails on Windows. Both are gone. What you get now:

The local chat model is not installed yet.

Install it with this command:

  pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu

Then run `ragmill chat` again. The first question downloads the model
(about 1.1GB); after that it runs fully offline.

The check is skipped for the hosted backends, so a Gemini or OpenAI user is never told to install a local model.

Upgrading

pip install --upgrade "ragmill[all]"

No API changes. This affects the guidance you get when the local model is missing — the install command itself is unchanged and works on 0.4.2 too.

v0.4.2

Fixed

Text files are decoded by their actual encoding instead of being read as UTF-8.

Every text reader opened files with errors="ignore", which never fails on a non-UTF-8 file — it silently corrupts it. This mattered most on Windows, because it is what Notepad writes:

Notepad "Save as"	Encoding	What RAGMill stored
Unicode	UTF-16 LE	T\x00h\x00e\x00 \x00c\x00o\x00m\x00p…
Unicode big endian	UTF-16 BE	same
ANSI	cp1252	Café — naïve “quoted” résumé costs £50 → Caf nave quoted rsum costs 50
UTF-8 with BOM	UTF-8 + BOM	stray  leading the first chunk

The UTF-16 cases ingested without any error, but the text embedded as noise and never matched a query — a .txt file that reported success and then returned nothing. The ANSI case dropped characters outright: costs £50 became costs 50, a change of meaning with no warning.

Encoding is now detected: byte-order marks are honoured (UTF-8/16/32, with the BOM stripped rather than left in the text), BOM-less UTF-16 is caught via interior NULs, and anything else falls back to cp1252 then latin-1 with a warning naming the encoding used. This covers every text reader — plain text (.txt, .md, .log, .rst), .csv/.tsv, .html, and .rtf — each of which carried its own copy of the flaw.

If a .txt file appeared to ingest but never turned up in search results, this was why.

Upgrading

pip install --upgrade "ragmill[all]"

Documents ingested before this release were stored with corrupted text if they were not UTF-8. Re-ingest them to pick up the correct content.

No API changes.

v0.4.1

Fixed

pip install ragmill[all] now works on a clean machine.

The all (and dev) extras pulled llama-cpp-python, which ships no PyPI wheels for recent versions. pip fell back to a 70 MB+ sdist that vendors llama.cpp — that needs a C++ toolchain, and on Windows the vendored tree exceeds the 260-character MAX_PATH limit, aborting the entire install before anything was written:

ERROR: Could not install packages due to an OSError: [Errno 2] No such file or
directory: 'C:\...\vendor\llama.cpp\tools\ui\...'

[all] and [dev] now resolve to wheels on every platform. The local LLM is opt-in — install a prebuilt wheel, which needs no compiler:

pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu

Or skip the local model entirely with a hosted backend (ragmill[chat-gemini], ragmill[chat-openai]) — both are included in [all].

Missing-dependency errors now name a command for your own OS.

tesseract and poppler's pdftoppm cannot come from pip, so these messages are the only guidance available — and they previously suggested brew install on every platform, useless on Windows and Linux. They now select per platform (UB-Mannheim installer / poppler-windows on Windows, brew on macOS, apt-get on Linux). The scanned-PDF error also mentions enable_ocr=False, so opting out of OCR is as discoverable as opting in.

The "local chat backend unavailable" error no longer points solely at pip install "ragmill[chat]" — the command that fails on Windows. It now leads with the prebuilt-wheel index, explains why the package is not bundled, and offers the hosted backends.

Source distributions are built from an allowlist.

The sdist previously included every file that .gitignore did not exclude, so a maintainer's local scratch directories could be swept into a locally built tarball. It now ships an explicit list of paths.

Upgrading

pip install --upgrade "ragmill[all]"

No API or configuration changes — 0.4.1 is a packaging and diagnostics fix only.

v0.4.0

[0.4.0] - 2026-07-24

Added

• Extended file-format support: .csv/.tsv (stdlib, no extra), plus .html/.htm, .rtf, .xlsx, and .pptx via the new office extra (pip install ragmill[office]).
• OCR support via the new ocr extra (pip install ragmill[ocr]): text extraction from images (.png, .jpg, .jpeg, .tiff, .bmp, .gif) and automatic fallback to OCR for scanned/image-only PDFs (requires the system tesseract binary, and pdftoppm/poppler for PDFs).
• DOCX extraction now also captures table cell text, not just paragraphs.

Changed

• Chat answers now lead with a direct answer followed by a brief 2-4 sentence explanation, and no longer embed bracketed citation markers.
• The ingestion engine now skips files that yield no extractable text with a warning instead of storing an empty document.

v0.3.2

Fixed

• CLI no longer requires numpy just to start: ragmill --version and ragmill --help now work on a core-only install (heavy imports are deferred into the commands that need them).
• Data commands that need numpy (ingest, sync, search, count, export, import) now fail with a clear "install ragmill[embeddings]" message instead of a raw ModuleNotFoundError: numpy.

v0.3.1

Added

• ragmill --version prints the installed version, and ragmill.__version__ exposes it programmatically.

v0.3.0

Added

• CLI entry point (ragmill command) with subcommands: ingest, sync, search, chat, count, serve, export, import, configure.
• REST API server (ragmill serve) with FastAPI — endpoints for ingest, sync, search, chat, export, import, count, health.
• Retrieval-augmented chat via three backends: local GGUF model (ragmill[chat]), Gemini (ragmill[chat-gemini]), OpenAI (ragmill[chat-openai]).
• Standalone config UI (ragmill configure) for setting up cloud backends and chat keys without editing .env by hand.
• Pinecone cloud vector store backend (ragmill[pinecone]).
• Qdrant cloud vector store backend (ragmill[qdrant]).
• JSONL export/import for backup and cross-backend migration.
• RAGMillConfig centralizes all settings in a single dataclass, loaded from env vars / .env.
• Configurable embedding dimension (RAGMILL_EMBEDDING_DIM).
• Batched embedding and upsert operations for better throughput.
• Docker support with docker-compose.yml (SQLite and Qdrant profiles).

Changed

• Default SQLite path is now ./ragmill.db (was :memory:) — data persists across CLI invocations.
• Server binds to 127.0.0.1 by default (was 0.0.0.0) — not exposed to the network unless explicitly configured.
• config-ui extra now includes FastAPI, uvicorn, pydantic, and numpy so ragmill configure works out of the box.
• server extra now includes python-dotenv so .env files are loaded automatically.
• Pinecone RAGMILL_PINECONE_ENVIRONMENT is now honored (parses region from formats like us-west-2, us-west1-gcp).

v0.2.0

Changelog

All notable changes to this project are documented here.
Format follows Keep a Changelog.

[0.2.0]

Added

• EmbeddingModel (ragmill.embeddings) — local ONNX-based sentence embeddings via a quantized MiniLM model, downloaded once and cached offline.
• VectorStore (ragmill.vector_store) — SQLite-backed storage with brute-force cosine similarity search, plus filtering by filename, source_file, modified_after, modified_before.
• sync_directory (ragmill.sync) — incremental sync between a folder and a VectorStore: skips unchanged files (content-hash based), replaces chunks for changed files, removes chunks for deleted files.
• modified_at is now captured during ingestion and threaded through chunk metadata.

Changed

• Project renamed to RAGMill. The original name, nexus-flow, was already taken on PyPI by an unrelated package; the next candidate, nexusflow, was rejected by PyPI for being confusingly similar to it. Package name is now ragmill, and the import path changed accordingly: import ragmill (previously import nexus_flow). The main class was renamed NexusEngine → RAGEngine to match.

[0.1.0]

Added

• RAGEngine — directory ingestion (.txt, .md, .log, .rst, .pdf via pypdf, .docx via python-docx) and semantic chunking with paragraph/sentence-boundary splitting and configurable overlap.
• Optional extras (pdf, docx, all, dev) so the core package has zero hard dependencies.