Skip to content

2.3.0

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 17 Oct 12:19
2.3.0
1b60c6f

Mini-RAG 2.3.0

Mini-RAG now works with llama.cpp as well as Ollama, and every request is sized to fit your model's context window, so large notes and folders are no longer silently cut off. The settings have been reorganised into three simple steps, and Summarize now covers folders of any size.

Highlights

  • llama.cpp support: chat with, and embed through, llama-server.
  • Settings in three steps: choose a provider, connect to it, tune your chats.
  • Requests that fit the context window: retrieved excerpts, chat history and whole-note fallbacks are sized to fit.
  • Summarize any folder: large notes and folders are summarized in stages, with progress and a Stop button.
  • Clearer help when a model is missing, with the exact command to run.

llama.cpp support

  • Choose llama.cpp under Provider. Mini-RAG talks to llama-server on http://localhost:8080 by default.
  • llama-server runs one model at a time, so Mini-RAG uses the model your server is actually running. The Chat Model follows the chat server, and the settings show the command to restart it with another model.
  • Dedicated embedding model: run a second llama-server with --embeddings and point Embedding server URL at it (http://localhost:8081 by default). Mini-RAG suggests the same well-known embedding models as for Ollama, as Hugging Face GGUF builds, split into Running and Not running. For one that isn't running, it shows the llama-server -hf … --embeddings --port … command to copy, using the port from your Embedding server URL.
  • If the embedding model you pick isn't the one the embedding server is running, the settings say which one is in use and how to restart the server with yours.
  • GPU troubleshooting tips speak llama.cpp's language: -ngl and -c rather than Ollama's settings.

Settings

  • The settings are laid out as three steps:
    1. Choose a provider
    2. Connect to {provider}, with a ✓ Connected / Not connected status
    3. Tune your chats
  • Each provider remembers its own URL and models, so you can switch between them without losing anything.
  • The embedding model shows its status at a glance: Installed or Running, Not running, Nothing is running at …, or Checking… while it looks. It checks again each time you choose a model.
  • Setup commands sit in full-width hints with a copy button, and a "Command copied" notice confirms the copy.
  • Model pickers say Loading… or No models instead of sitting empty.

Chats and the context window

  • Every request is sized to the chat model's context window. llama.cpp reports the size it was started with (-c). Ollama doesn't report it, so Mini-RAG assumes Ollama's default of 4,096 tokens.
  • Retrieved excerpts and chat history share the window, so long chats keep working: the oldest turns are dropped first.
  • When notes have to be left out of a request, a short line in the chat says so, and says when the window was assumed.
  • An open chat stays in step with the settings: changing the chat model or provider updates its header and starts afresh.

Summarize

  • A note or folder that fits the context window is summarized in one go, as before.
  • A larger one is summarized in stages: part by part, then combined into one summary, so every note is covered. A line in the chat counts the parts as it goes ("Summarizing part 3 of 16…"), and the Summarize button becomes Stop while it runs.
  • The conversation remembers the summary, not the whole folder's text, so follow-up questions stay quick.

Clearer help

  • If the embedding model isn't available, the chat panel says which model it expected and where, gives the command to run (with a copy button), and offers Refresh. If problems persist, it points to the settings and to support.
  • If no model is chosen at all, it asks you to choose one, rather than suggesting a command for a model with no name.

Support Mini-RAG

Mini-RAG is a one-man project. If it's useful to you, you can now support it on GitHub Sponsors. The link appears only within Mini-RAG's own interface: a banner at the top of its settings, a heart in the chat panel's header, and the Support development command. Nothing is shown unprompted.

Fixes

  • Buttons in the chat panel show one tooltip instead of two.
  • Refresh in the chat's help panel now behaves like opening the chat again: if the chat model can't create embeddings, Mini-RAG includes the notes whole instead of asking you to install something.
  • If your vault's configuration folder isn't called .obsidian, the embedding cache is now stored in it, instead of in a separate .obsidian folder.
  • The embedding timing message in the developer console now appears only at the console's Verbose level, so the console stays quiet by default.

Upgrading from 2.2

  • Your settings are migrated automatically: your existing Ollama URL and models carry over, and Ollama stays selected.
  • The embedding cache uses a new format that also records the provider and the embedding server's address, so it is rebuilt once, the first time you open each note or folder.
  • On Ollama: requests are now sized for an assumed 4,096-token context window. If you have raised Ollama's context size (for example with OLLAMA_CONTEXT_LENGTH), Mini-RAG still sizes requests for 4,096 tokens.

Under the hood

  • A new provider layer makes room for more local servers.
  • An automated test suite of more than 1,100 tests, with coverage checks, runs before every build.

Full Changelog: 2.2.0...2.3.0

Assets 5
Loading