RayRAG 0.4.49 — PDF streams decode with a ceiling (parity v0.3.9a)

RayRAG 0.4.49 — PDFs can no longer be used to fill your memory

The last unbounded expansion path

PDF streams are compressed, so a few kilobytes can describe gigabytes of content. Every
place RayRAG decoded a PDF stream went through a library call that buffers exactly as
much as the stream expands to — a 256 KiB PDF declaring a 256 MiB page was simply
allocated.

Decoding now has a ceiling. The inflater stops as soon as it passes the limit, so a
"bomb" never becomes an allocation; the ASCII encodings and the PNG/TIFF predictors
that accompany them are handled the same way. A filter that cannot be decoded within
the limit is reported rather than handed to an unbounded decoder. The ceiling is
RAYRAG_PDF_STREAM_LIMIT_BYTES (64 MiB by default).

Verified with a hand-built 256 KiB PDF whose page expands to 256 MiB:

WARN  Skipping PDF page content error=PDF page 1 content inflates past the 64 MiB safety limit
ERROR Document processing failed … PDF content could not be decoded:
      1 page stream(s) exceeded the 64 MiB safety limit

The container stayed at 84 MiB. A normal one-page PDF still extracts its text.

A refusal is never silent

Previously such a document would have been reported as DONE with 0 chunks — as if
it had parsed fine and simply had no text. Now each refused page is logged with its
page number and reason, and a document that produced nothing because of the cap
fails with that reason, so you can see what happened.

Also in this release

Three more unbounded reads are capped: the Blob and GitLab connector downloads and the
LLM raw response used for text-to-speech.

Upgrading

git pull
docker compose build rayrag
docker compose up -d rayrag

New optional setting: RAYRAG_PDF_STREAM_LIMIT_BYTES. Verified with 1609 Rust tests and
a real-browser walkthrough of every previewer (14/14) on the running container.