Skip to content

raggity v0.9.1 — ingest hardening hotfix

Choose a tag to compare

@IxMxAMAR IxMxAMAR released this 03 Jul 14:35

Hotfix from a real-world report: running rag ingest from a home directory with broad globs. Fixes: (1) built-in junk-directory pruningAppData, node_modules, .git, site-packages, __pycache__, venvs, package caches and the .raggity index itself are skipped when they appear below an include pattern's base (a pattern pointed inside such a dir still works), plus a new user-configurable sources.exclude glob list and a warning when a pattern matches >10,000 files. (2) Embedding OOM crash fixed — the chunker now hard-splits oversized single paragraphs (a multi-MB single-line file previously became one enormous chunk whose attention matrix tried to allocate 3.2GB), and embedding.parallel now defaults to in-process single-model instead of an all-cores multiprocess pool. (3) Quiet ingest — per-file skip messages demoted to info-level (summarized as skipped=N in the ingest summary), pypdf's internal warnings silenced. 387 tests, 0 warnings.