fix(rag): detect encrypted and corrupted PDFs with actionable errors (#451)

[kovtcharov]

Summary

Closes #451.

Password-protected PDFs were silently indexed as 0 chunks (pypdf returns empty text for encrypted pages without raising), and corrupted PDFs surfaced vague Failed to process document errors. Both cases left users with no actionable path forward.

This PR makes RAGSDK._extract_text_from_pdf detect three distinct failure modes and surface them with user-facing remediation guidance:

Mode	Exception	Trigger	Remediation hinted
Password-protected	EncryptedPDFError	reader.is_encrypted	qpdf --decrypt / pdftk commands
Malformed / truncated	CorruptedPDFError	pypdf.errors.PdfReadError at construction	Re-download / re-export / re-OCR
No extractable text	EmptyPDFError	All pages blank after extraction	ocrmypdf or enable VLM image extraction

All three inherit from a new PDFExtractionError base, which in turn inherits from ValueError so existing except ValueError: callers (including reindex_document) keep working unchanged.

Changes

• src/gaia/rag/sdk.py
  • New exception classes PDFExtractionError, EncryptedPDFError, CorruptedPDFError, EmptyPDFError, each carrying a stable short status code
  • _extract_text_from_pdf:
    • Wraps PdfReader(path) to catch PdfReadError and re-raise as CorruptedPDFError via raise ... from e (preserves the pypdf cause for operator logs)
    • Checks reader.is_encrypted up-front and refuses before attempting extraction
    • Detects all-blank pages (where pypdf returns empty text for every page) and raises EmptyPDFError
    • Adds pdf_status: "readable" | "encrypted" | "corrupted" | "empty" to returned metadata
  • index_document: adds a dedicated except PDFExtractionError clause that sets stats["pdf_status"] separately from stats["error"] so UIs can badge documents with the short status code instead of showing the full remediation blob
• src/gaia/rag/__init__.py: re-exports the four error classes

Tests

New suite at tests/unit/rag/test_pdf_extraction_errors.py (15 tests, all passing in < 1s):

• Fixtures built programmatically with pypdf.PdfWriter — no binary fixture files committed to the repo
• Encrypted: asserts raises EncryptedPDFError with qpdf/pdftk in message; asserts index_document returns pdf_status="encrypted"
• Corrupted: covers both garbage-bytes and truncated-valid-PDF; asserts __cause__ preserves the pypdf exception
• Empty: asserts raises EmptyPDFError with OCR hint on a blank-page PDF
• Hierarchy guards: all three remain ValueError subclasses; status codes are distinct

The fixture patches _check_dependencies, AgentSDK, and VLMClient so tests run without sentence-transformers / faiss / a live Lemonade server.

Test plan

• pytest tests/unit/rag/ -xvs — 15/15 pass locally
• pytest tests/unit/test_rag_tools.py — 17/17 still pass (no regression in adjacent RAG tests)
• black --check + isort --check-only on touched files — clean
• CI runs pass on the remote
• Manual verification with a real password-protected PDF via gaia chat

[kovtcharov]

@claude review this PR.

[github-actions]

Claude finished @kovtcharov's task —— View job

---

PR Review: fix(rag): detect encrypted and corrupted PDFs with actionable errors (#451)

Overall: 🟢 LGTM — Well-designed fix that properly addresses #451 with clean exception hierarchy and actionable user guidance.

---

🟢 Strengths

1. Clean exception hierarchy — All three errors inherit from PDFExtractionError(ValueError), preserving backward compatibility with existing except ValueError: blocks while allowing granular handling.

2. Actionable messages — Error messages include concrete remediation steps (qpdf, pdftk, ocrmypdf), not just failure descriptions. This is exactly what users need.

3. Status codes for UIs — The pdf_status field with short codes ("encrypted", "corrupted", "empty", "readable") enables clean UI badging without parsing multi-line error strings.

4. Proper exception chaining — raise CorruptedPDFError(...) from e preserves the pypdf exception for operator debugging while surfacing user-friendly messages.

5. Comprehensive test coverage — 15 tests covering all failure modes, hierarchy guards, and index_document integration. Fixtures built programmatically — no binary files in repo.

6. Existing test fixes — Updated tests/test_rag.py to set mock_pdf_instance.is_encrypted = False so Mock auto-attributes don't trip the new guard.

---

🟡 Minor Suggestions

1. Consider logging level (sdk.py:117, sdk.py:134, sdk.py:737)

   • Currently uses self.log.error() for all PDF failures. For empty PDFs (scanned docs), warning might be more appropriate since it's not an error per se — just missing OCR.

2. Type annotation for status (sdk.py:60)

   • Consider using ClassVar[str] for the class-level status attribute to make the typing more precise:

   from typing import ClassVar
   status: ClassVar[str] = "unreadable"

3. Test for VLM suggestion in EmptyPDFError (test_pdf_extraction_errors.py:477)

   • The test checks for "OCR" but the message also mentions VLM as an alternative. Consider extending the assertion to verify both remediation paths are present.

---

✅ Verified

• Exception classes properly exported in __init__.py
• __all__ updated alphabetically
• No breaking changes to existing API
• Test mocks appropriately bypass heavy deps (sentence-transformers, faiss, Lemonade)

---

Approve after CI passes. Nice work on making PDF failures actually actionable!