Repository navigation
Releases: Pooria82/ParsRAG
Release list
ParsRAG v1.1.2
Added
-
Update check: once a day the server asks GitHub (anonymously) for newer
releases (GET /updates,UPDATE_CHECK_ENABLED, on by default and in
Settings). The sidebar offers the latest one, and its window lists the
changes of every newer release in Persian or English, asks for a backup of
the conversations, and gives one command:
scripts/update.ps1(Windows) orscripts/update.sh. -
The update scripts back up
.env, the Qdrant data, and the model settings,
go straight to the latest release tag, rebuild with the same Compose files
and profile, wait until the service is ready, and start the previous
version again if the rebuild fails. -
After an update the app shows what changed in every version since the last
one seen (GET /release-notes, fromrelease-notes/<version>.<fa|en>.md),
and an open page loads the new interface by itself: the page and the service
worker are served withCache-Control: no-cache, hashed assets are cached
for a year, and a waiting interface is activated when nothing is running. -
One version for the project in
VERSION, read by the backend and the
frontend build;scripts/set_version.pycopies it topyproject.toml,
package.json,package-lock.json, andCITATION.cff, and a test keeps
them in step. -
OCR_DOCUMENT_TIMEOUT_SECONDS(default 900, changeable in Settings) bounds
the OCR time of one document. Remaining pages and images are skipped with a
notice and the rest is indexed. -
PowerPoint speaker notes are indexed with their slide.
-
CSV and TSV files are read as tables, with row records and whole-table
answers. -
Text files are split at Markdown headings or about 300 words, so sources
name a section instead of the whole file. -
Uploaded images keep a preview shown next to their source.
-
Streamed answers send a
pingevent every 15 seconds while the server
works, so the browser waits as long as the model needs (up to the server's
own timeouts) and gives up only on a connection that went quiet. -
An unanswered question (stopped before any text, or interrupted) has a
Try again button. -
Open tabs of the workspace share their changes instead of overwriting each
other's conversations.
Improved
- The service status card shows the model clearly for any name: the model
in monospace with its variant as a tag (gemma34b), its publisher and
the provider below (google · API service), an icon for local or API, and
the full id on hover. - Scanned PDF pages are recognized several at a time, like images in Word and
PowerPoint, and near-certain OCR readings skip the skew search. On the
96-image final report the text is unchanged.
Fixed
- After a reload the conversation that was open opens again; once
conversations lived in IndexedDB the app always opened the first one.
Conversations are written before the app reloads itself (for an update) and
when the page is hidden, so a just-finished answer is not lost. - Two open tabs no longer undo each other's unsaved edits, new conversations,
or deletions; a save that had to fall back tolocalStoragealways wins over
the older IndexedDB copy. - "Clear all data" no longer stays on "Saving…"; retrying or stopping an
earlier question streams and shows its note in place; the keyboard stays in
the message box after the first question; a document cannot be deleted twice
at once; the message box says when an upload is running. - Setting
PARSRAG_ALLOWED_HOSTSkeeps localhost allowed, so the container
health check and the update script keep working. - A saved API connection starts while offline, and an API host that does not
resolve shows as unreachable instead of an error. - Saving an API without a key now shows its own message; malformed CSV files,
empty generated titles, oversized requests and bad session ids return clear
codes instead of internal errors; Word files saved online
(word/document2.xml) are accepted; reusing a document with the same
content as one already in the conversation is refused. - The update scripts work when the stack is stopped, do not stop on Docker
warnings in Windows PowerShell, and always install the release tag. - After the Settings window (or any dialog) closes with the mouse, the button
that opened it no longer shows the keyboard focus ring. - An external model API can no longer be connected without an API key.
OpenRouter lists its models without a key, so saving the connection passed
and every question then failed with "internal error". Saving is now refused
with a message, the sidebar says "No API key is set" instead of "Service
ready", and a question names the missing key. The key form shows which host
the service holds a key for (a key is only sent to its own host; a key in
.envbelongs toMODEL_API_BASE_URL). - "Service ready" no longer appears while the model cannot answer. A new
GET /models/statuschecks the active model service (Ollama running with
the selected model installed; an API reachable and accepting the key, at
most every 10 seconds), and the sidebar says "Ollama is not reachable",
"This model is not installed in Ollama", "The model API is not reachable"
or "The API key was rejected" instead, with a Check again button whose
icon spins while the check runs. - Answers are written in the language of the question, also when the
documents are in the other language. A follow-up such as «چطور تمدید
میشوند؟» about English documents was rewritten as an English question, and
the answer language was taken from that rewrite; with gemma3:4b all five
such test questions were answered in the wrong language, and now all five
are right. The prompt asks to translate what it uses and starts the answer
in the question's language. A question without words (a formula) is
answered in the interface language, which the browser now sends. - Your questions sit on the side their language starts from: Persian on the
right and English on the left, in either interface language. They were on
the opposite side. - Text and pictures inside grouped PowerPoint shapes are indexed, and a
linked picture adds a notice instead of rejecting the whole deck. - Content inside Word content controls (forms, templates) is indexed; a
generated table of contents is skipped. - An image that declares more pixels than allowed is reported as an OCR
notice instead of a server error. - HTML table cells are separated instead of running together.
- A UTF-8 file with a few stray bytes is read as UTF-8 instead of turning
into Windows-1256 mojibake. - A question asked while a document was being deleted could keep that
document in keyword search until the next change. /query/streamanswers a full query capacity with 429 and an unknown
@{file}with 400 before streaming, instead of 200 plus an error event. A
client that disconnects marks the query failed instead of leaving it at
"generating" for five minutes.- Documents-only mode no longer skips its evidence check for ordinary
questions that contain words such as «خلاصه», «موضوع» or «چه نوع»; only
questions about the document itself, or a bare request to summarize, do. - The documents-only refusal is the same sentence in the evidence check and
in the prompt, in the question's language. An invalid
STRICT_RAG_THRESHOLDfalls back to the default instead of failing every
documents-only question. - Brackets inside math and code, such as the interval
[1, 3], are no
longer counted as citations. - Hybrid mode trims the context in its ranked order; mixing reranker and
cosine scores could drop the best reranked passage. - A stored API key is only sent to the server it was entered for; changing
the API address without a new key no longer sends the old one. - API requests have one retry layer (
MODEL_API_MAX_RETRIES); the OpenAI SDK
no longer retries on top of it. - Conversation titles and suggested questions are capped on Ollama too.
- Resetting CPU threads to automatic restores the startup value.
- A failed reranker download waits five minutes before the next attempt
instead of being retried on every Hybrid question, and the download has a
timeout. - Listing, previewing, reusing and deleting documents wait until the
embedding model is ready, like questions and uploads. - Runtime settings are validated, saved, and applied as one step; a failed
save changes nothing. - The message box accepts 12,000 characters, the API's limit, and a question
that is too long says so. A failed or unanswered turn is no longer sent as
history, and long turns are shortened, so one failure does not make the
next questions fail too. - Answers keep links,
snake_casenames and dollar amounts such as "$5"
instead of typesetting them as math. - Unsaved server-setting edits survive the 30-second health check and other
changes, and clearing a number field no longer switches it to Automatic. - Editing a question that never got an answer no longer attaches the new
answer to a later question; the answer is placed right after it. - A save that fell back to
localStorage(IndexedDB failed) is loaded on the
next start instead of the older IndexedDB copy. - Keyboard shortcuts work with the Persian keyboard layout.
- The unused browser setting
strictThresholdis removed; the Strict
threshold is a server setting. npm run devproxies/settings, so server settings load in development.- Deleting a conversation or clearing all data works while the service is
offline; documents on the server are deleted when it is reachable again.
Conversations without documents never need the server. - Importing a backup applies its preferences, and its documents show as not
on this server until the server lists them. - A timeout or dropped connection keeps the text that already streamed, with
the reason below it, as Stop does. - Upload limits follow a server-settings change at once; answer-version
arrows point the rea...
ParsRAG v1.1.1
Changed
- The default local model is
gemma3:4beverywhere (code, Compose,
.env.example, the interface, and the README). It fits fully on a 6 GB GPU
and was the most accurate of the small models tested. SetLLM_MODEL_NAME
to keep another model.
Fixed
- Request and operation logs are written to stdout at
LOG_LEVEL(one JSON
object per line, or plain text withENVIRONMENT=development). Nothing
configured them before, so they were dropped. - After the startup retries run out, the service reports
failedbut keeps
retrying every minute and becomes ready once Qdrant or the model cache is
back, without a restart. - The README sections that run Ollama inside Docker set
OLLAMA_DOCKER_BASE_URL=http://ollama:11434; without it the app kept using
the Ollama installed on the computer. EMBED_QUERY_PREFIXandEMBED_PASSAGE_PREFIXreach the app in Docker.- The README settings tables mark every setting that can be changed in the
app and documentMODEL_API_MAX_RETRIES. - PowerPoint slides no longer add their slide number, date, or footer to the
text, so the slide-number field (shown as‹#›) no longer appears in
answer sources or search. Upload existing presentations again to remove it
from them.
Security
- Pillow is upgraded to 13.0.0 to fix denial-of-service and information
disclosure vulnerabilities in 12.3.0, one of them critical.
Upgrade note
- Nothing to migrate. Upload PowerPoint files again to remove the slide-number field from their sources. If your
.envhas noLLM_MODEL_NAME, the default is nowgemma3:4b; pull it withollama pull gemma3:4bor set the model you use.
Full changelog: v1.1.0...v1.1.1
ParsRAG v1.1.0
Added
-
Settings in the app can change the options that used to need
.envand a
restart (27 in all), in four groups:- model: context sizes, thinking, retries, timeouts
- retrieval: threshold, depths, keyword search
- documents: upload limits, OCR, image previews
- indexing speed: OCR workers, CPU threads, batches
Each shows its
.envvalue and can be reset to it. Overrides survive
restarts. Ports, allowed addresses, Qdrant, memory limits, and the embedding
model stay in.env. -
Answer sources show the image an excerpt was read from, next to the excerpt.
-
Installed Ollama models are offered in a menu, with their sizes.
-
.env.examplenow has only the model choice active. Every other setting
shows its default, grouped by whether Settings can also change it. -
The README starts with one complete setup using every capability; other ways
to run follow it. -
The app can use the Ollama installed on the computer (Windows, macOS,
Linux) with its existing models and GPU. Set
OLLAMA_DOCKER_BASE_URL=http://host.docker.internal:11434in.envand
start without thelocal-modelprofile. No override file is needed, and the
app container needs no GPU. The README's "Complete run" section covers
each system, GPU memory, and troubleshooting. -
OLLAMA_ALLOWED_HOSTSadds hosts that Ollama URLs may use.
OLLAMA_INTERNAL_HOSTis still read. -
Settings explains Ollama failures ("address not allowed", "did not answer")
in Persian and English. When an API is active, it offers the server's own
Ollama URL. -
Images inside Word and PowerPoint files are cited by number: text read
from an image is its own excerpt with the image's caption, shown as
"paragraph 120 · image 7" or "slide 25 · image 9", and answers say when a
fact comes from an image. Each image excerpt also carries the text that
introduces it (the end of the paragraph before it, or the slide's text). -
Word tables are cited by number and the paragraph they follow
("paragraph 120 · table 3") instead of an internal element index. -
When part of a table is retrieved, the rest of the table comes with it, so
questions about the largest or smallest value see every row.
Improved
-
Questions that ask several things ("… with which key, and which fields …")
are also searched part by part. Each part keeps its best excerpts, and the
model is asked to answer every part. In the acceptance set, 5 of the 6
answers that missed a part became complete, with no regressions among the
questions that were already right. -
Indexing is faster with the same results:
- Pictures in Word and PowerPoint are read several at a time
(OCR_WORKERS). The 96-image final report parsed in 33 s instead of
121 s, with identical text. - Each document is embedded in length-sorted batches.
Indexing that whole report went from about 4.5 minutes to about 3.
- Pictures in Word and PowerPoint are read several at a time
-
OCR reads screenshots better: a second, sparse-text pass adds text that
page layout analysis skipped (status panels, dark UI themes). -
Up to 200 images per Word or PowerPoint file are read (was 30).
-
With several documents, specific questions search all files together, so
the file that holds the answer can provide most excerpts; each other file
still contributes its best match, and passages repeated across files are
kept once. -
Commands and code in one-cell boxes in Word stay with the sentence that
introduces them, so questions about them find them. -
OCR reads text on tinted panels (for example a light-blue status box).
Fixed
[extract_itex]...[/extract_itex]markup that Gemini models leak into answers
is shown as math, and table rows inside a source excerpt render as a table.- Thinking models on Ollama (such as
qwen3.5) no longer return empty
answers. They could spend the whole answer budget on reasoning, so Ollama
is now asked not to think unlessOLLAMA_THINK=1. On a 6 GB GPU with
qwen3.5:4b, an empty reply after 59 s became a cited answer in 7 s. - Reasoning that some APIs put inside answers (
<thought>...</thought>from
Google AI Studio's Gemma 4,<think>...</think>elsewhere) is removed from
answers, streams, conversation titles, and suggested questions. - Transient API failures (HTTP 5xx and 429) are retried up to
MODEL_API_MAX_RETRIEStimes (default 4) with growing delays; Google AI
Studio's free tier fails about one Gemma request in three. - Google AI Studio model ids get the
models/prefix its endpoint requires. - English questions get English answers in Hybrid and open-chat modes; they
were often answered in Persian. Every prompt now names the answer language,
detected from the question. - The service retries startup when Qdrant is briefly unavailable instead of
staying not ready until restarted. - A question is no longer limited to one file just because it shares an
ordinary word with the file name (such as «اول» or «تست»). - An API provider that is out of credit (HTTP 402) is reported as such, with
a hint to top up the account or choose another model.
Security
- Questions with long runs of spaces or many unclosed
@{mentions can no
longer make question splitting or answer-language detection slow (CodeQL
regular-expression alerts).
Changed
- Every backend test runs without skips and without Docker:
PARSRAG_E2E=1 pytestruns the end-to-end tests against an in-memory
Qdrant unlessQDRANT_HOSTnames a server. - Dependencies: FastAPI 0.143, Uvicorn 0.54, python-dotenv 1.2.4, Ruff 0.17,
mypy 2.4, KaTeX 0.19, Vite 8.3.4, Playwright 1.64, lucide-react 1.56,
msgpack 1.2.3 or later.
Upgrade note
- Documents indexed by 1.0.0 keep working. Upload Word and PowerPoint files
again to gain image and table citations, image previews next to sources,
images past the 30th, and the improved OCR of screenshots. - Settings changed in the app are saved in the
app_configvolume and take
precedence over.env; each can be reset to its.envvalue.
Acceptance test
52 detailed Persian questions over 10 real documents (one PowerPoint and nine Word reports, 244 images), each asked in both modes with gemini-flash-lite-latest:
| 1.0.0 | 1.1.0 | |
|---|---|---|
| Documents only: correct answers | 33/52 | 44/52 |
| Hybrid: correct answers | 30/52 | 42/52 |
| Image questions correct (documents only / hybrid) | 5 / 4 of 19 | 12 / 11 of 19 |
| Table questions correct (documents only / hybrid) | 10 / 9 of 11 | 11 / 11 of 11 |
| Answer cites the right document (documents only / hybrid) | 45 / 48 | 51 / 52 |
The 1.0.0 column used google/gemma-4-26b-a4b-it, so part of the difference comes from the model. Questions that ask several things were measured afterwards: 5 of the 6 incomplete answers became complete, with no regressions.
Full changelog: v1.0.0...v1.1.0
ParsRAG v1.0.0
Added
- Answers stream token by token (
POST /query/stream, Server-Sent Events):
stages, the numbered sources, text, and the final result. Stopping keeps
what was already written. - Numbered citations: the model cites excerpts as
[n], the response lists
which ones it used (cited), and the interface turns them into chips that
open the cited passage. Uncited passages fold under "more passages
consulted". - Questions your documents can answer: an empty conversation suggests three
specific questions written from its documents (POST /sessions/{id}/suggestions). - Every answer shows where its content came from (documents only, documents
and model knowledge, model knowledge, or not found in the documents). - Conversation backup and restore (versioned JSON) and per-conversation
Markdown export. - Keyword (BM25) search fused with vector search; on keyword-style queries it
raised mean reciprocal rank from 0.748 to 0.823. - OCR re-reads sideways, upside-down, and skewed scans (character error for
pages rotated 90° fell from 0.85 to 0.09). - Windows-1256 (Persian), UTF-16, and Windows-1252 text files.
- Duplicate uploads are recognized by content, not only by name.
- Opt-in E5 query/passage prefixes (
EMBED_E5_PREFIXES=1). - End-to-end CI job: upload to cited answer through real parsing, embeddings,
Qdrant, and reranking, with a retrieval quality gate.
Improved
- Documents-only (Strict) mode answers by meaning: paraphrased questions,
whole-document questions, and partial answers no longer get refused, while
facts must still come from the documents. - Persian PDF text is rebuilt in reading order from glyph positions; the ezafe
mark stays on its letter; text layers without usable Unicode go to OCR. - OCR: grayscale at 200 dpi, no contrast stretching, a confidence-based second
pass, and no invisible bidi marks (mean character error 0.225 → 0.133). - Mathematical answers: Persian words are never typeset as math, bare LaTeX
commands are rendered, and unclosed delimiters are removed. - Hybrid reranks with an Arabic-script cross-encoder that is cached on disk
(MRR 0.354 → 0.710 against the previous English model). - Persian spelling is normalized in indexed text and questions (Arabic Yeh and
Kaf, presentation forms, diacritics, digits). - Word documents are chunked under their headings (median chunk 13 → 91 words)
and each chunk carries its heading path. - Prompts fit the model's context window, and Ollama receives
num_ctx. - Conversations are stored in IndexedDB (migrated automatically from
localStorage).
Fixed
- A failed upload no longer leaves a partly indexed document behind.
- Word and PowerPoint files with more than 30 images, and scans longer than
OCR_MAX_PAGES, are indexed in part with a notice instead of rejected. - A PDF with a few image pages is indexed when Tesseract is not installed;
the skipped pages are named in a notice. - Uploads up to the advertised 500 MiB fit the container's temporary storage.
- Server timeouts no longer outlive the browser's wait.
- A rendering error shows a recovery screen instead of a blank page.
Security
- Requests addressed to host names other than localhost, 127.0.0.1, and ::1
are rejected (DNS rebinding);PARSRAG_ALLOWED_HOSTSadds names for LAN use.
Changed
- Error responses carry a stable
codenext todetail, and the interface
explains each one (for example an encrypted PDF or a rejected API key). POST /ingestreturns per-file chunk counts and notices;POST /query
returnscitedandoutcome.- The project website moved to its own repository,
ParsRAG-Landing, published at
https://pooria82.github.io/ParsRAG-Landing/; the old GitHub Pages address
redirects there.
Upgrade note
- Documents indexed by 0.2.0 keep working. Upload them again to gain Persian
normalization, heading-based Word chunks, content-based duplicate detection,
and the improved PDF text and OCR. - The first start downloads the reranker (about 100 MB) into the model cache.
Live evaluation
google/gemma-4-26b-a4b-it, 7 private PDFs, through the 1.0 Docker image's HTTP API:
| 0.2.0 | 1.0.0 | |
|---|---|---|
| Strict answers passing (facts ≥ 2/3 and the right file and page) | 11/14 | 12/14 |
| Strict wrongly refusing a document question | 0/14 | 0/14 |
| Hybrid answers passing | 6/7 | 13/14 |
| Off-topic question refused in Strict | 7/7 | 7/7 |
| Fact coverage, Strict / Hybrid | 0.78 / 0.73 | 0.86 / 0.85 |
| Cited sources include the labeled page | — | 26/28 |
Full changelog: v0.2.0...v1.0.0
ParsRAG v0.2.0 — Offline Persian (Farsi) RAG
ParsRAG v0.2.0 — offline Persian (Farsi) RAG workspace
A pre-1.0 release that hardens the architecture, dependencies, and runtime image ahead of 1.0.0.
Highlights
- Layered backend: ports and adapters, one use case per operation, input/output DTOs, and a single composition root. The HTTP API is unchanged. (ARCHITECTURE.md, ADR 08)
- New documentation: DATA.md covers ingestion, vector storage, RAG modes, and why Qdrant. Blueprints cover data flow, use cases, and DTOs.
- Runtime image fixes:
- PyTorch is now 2.13 on Debian Trixie for the CPU and NVIDIA images.
- transformers and sentence-transformers are pinned. A floating transformers 5.19 broke the import on PyTorch 2.6 (the AMD image).
- CI now imports the app inside the built image.
- Retrieval settings:
STRICT_RAG_TOP_K/RAG_TOP_KandHYBRID_RERANK_TOP_Know take effect. They're empty by default, which keeps adaptive depth.- The Strict threshold default is 0.80 everywhere.
- Dependencies: React 19, TypeScript 7, Python runtime and tooling updates,
actions/upload-artifactv7. Dependabot now targetsdevelop. - Discovery: a bilingual project website and improved README, citation, and package metadata.
Also new since v0.1.0
- Installable PWA with an offline interface shell.
- Document mentions with
@, reusable indexed files, quick/commands, platform-aware shortcuts, color palettes, and a guided tour. - Searchable page-boundary excerpts for PDFs, with page-range citations.
- Strict answers can use borderline matches only when the retrieved text contains the question's distinctive terms.
- OCR also covers image-heavy PDF pages with short text layers.
Upgrade notes
- Existing Qdrant collections remain readable. To gain the page-boundary chunks and the improved OCR, remove previously indexed PDFs and upload them again.
- If you set
RAG_TOP_K,STRICT_RAG_TOP_K, orHYBRID_RERANK_TOP_Kin your own.env, those values now take effect as a fixed depth. Clear them to keep adaptive depth.
Quick start
git clone https://github.com/Pooria82/ParsRAG.git && cd ParsRAG
cp .env.example .env
docker compose --profile local-model up -d qdrant ollama
docker compose --profile local-model run --rm ollama-init
docker compose --profile local-model up -d --build appپارسرگ نسخه 0.2.0 — RAG آفلاین برای اسناد فارسی
نسخهای پیش از 1.0 که معماری، وابستگیها و ایمیج اجرایی را برای انتشار 1.0.0 آماده میکند.
مهمترین تغییرات
- بکاند لایهای: پورت و آداپتور، یک یوزکیس برای هر عملیات، DTOهای ورودی و خروجی، و یک نقطهٔ واحد برای سرهمکردن وابستگیها. API بدون تغییر مانده است.
- مستندات جدید: ARCHITECTURE.md و DATA.md (ورود اسناد، ذخیرهٔ برداری، حالتهای RAG و دلیل انتخاب Qdrant)، بههمراه نقشههای جریان داده، یوزکیسها و DTOها.
- اصلاح ایمیج اجرایی:
- PyTorch 2.13 روی Debian Trixie برای ایمیجهای CPU و NVIDIA.
- نسخهٔ transformers و sentence-transformers ثابت شد؛ نسخهٔ 5.19 بارگذاری برنامه را روی PyTorch 2.6 (ایمیج AMD) خراب میکرد.
- CI حالا برنامه را داخل ایمیج ساختهشده بارگذاری و بررسی میکند.
- تنظیمات بازیابی:
- متغیرهای عمق بازیابی حالا اثر دارند و بهطور پیشفرض خالیاند، یعنی عمق تطبیقی حفظ میشود.
- آستانهٔ حالت سختگیرانه همهجا 0.80 است.
- وابستگیها: React 19، TypeScript 7، بهروزرسانی بستههای پایتون و ابزارها؛ Dependabot حالا روی شاخهٔ develop کار میکند.
- دیدهشدن پروژه: وبسایت دوزبانه و اطلاعات بهتر برای README، استناد و بستهها.
موارد دیگر از نسخهٔ 0.1.0 به بعد
- نسخهٔ قابل نصب (PWA) با پوستهٔ آفلاین رابط کاربری.
- اشاره به اسناد با
@، استفادهٔ دوباره از فایلهای ایندکسشده، دستورهای سریع با/، میانبرهای صفحهکلید، پالتهای رنگی و تور راهنما. - بخشهای قابل جستوجو در مرز صفحههای PDF با ارجاع به بازهٔ صفحه.
نکات ارتقا
- کالکشنهای Qdrant قبلی قابل خواندن میمانند. برای بهرهمندی از بخشهای مرز صفحه و OCR بهتر، PDFهای قبلی را حذف و دوباره بارگذاری کنید.
- اگر در
.envخود متغیرهایRAG_TOP_K،STRICT_RAG_TOP_KیاHYBRID_RERANK_TOP_Kرا مقدار دادهاید، حالا بهعنوان عمق ثابت اعمال میشوند. برای حفظ عمق تطبیقی آنها را خالی کنید.
Full changelog: v0.1.0...v0.2.0 · CHANGELOG.md · Website: https://pooria82.github.io/ParsRAG/