Repository navigation
GoldPan AI v2.0.0: Dual-Mode Local ONNX RAG, CLI & MCP Server
Highlights
GoldPan AI v2.0.0 introduces a 100% Offline Local ONNX Vector Engine, a Command-Line Interface (goldpan), and a Model Context Protocol (goldpan-mcp) Server for AI coding agents, alongside a complete algorithmic overhaul of the Markdown refinery pipeline.
✨ New Features
- Dual-Mode ChromaDB Knowledge Base (
local_onnx+gemini_api)- Runs 100% offline out of the box using local ONNX
all-MiniLM-L6-v2embeddings (384-dim) ingoldpan_kb_localwith zero API keys required. - Isolates cloud
gemini-embedding-001vectors (768-dim) ingoldpan_kb_geminiwith exponential backoff againstHTTP 429rate limits and automatic fallback to Local ONNX.
- Runs 100% offline out of the box using local ONNX
- Model Context Protocol Server (
goldpan-mcp)- Exposes 5
stdioJSON-RPC 2.0 tools (goldpan_extract,goldpan_kb_ingest,goldpan_kb_search,goldpan_kb_list,goldpan_kb_delete) andgoldpan://kb/<doc_id>resources for Claude Desktop, Claude Code, Cursor, and Antigravity. - Enforces
rea-tool-designcontracts: response character budgets (detail: summary | complete), explicitcoverageandresidual_unknowns, and untrusted-content provenance tags (_meta.provenance.untrusted_content = true).
- Exposes 5
- Command-Line Interface (
goldpan)- Installable via
pip install -e .(pyproject.toml), providing terminal commands forextract,ingest,search,kb-list,kb-delete,mcp, andversionwith Rich telemetry tables.
- Installable via
- Multi-Turn Grounded Chat & Offline Extractive Fallback
/api/kb/chatforwards multi-turn conversation history, attaches source citations, and falls back to extractive passage retrieval when running without an API key.
- Hybrid Local + Cloud Image Inspection (
HybridVisionCaptioner)- Extracts image dimensions, color modes, and EXIF metadata locally via Pillow when offline, or delegates OCR and table extraction to
gemini-2.5-flashwhen a key is configured.
- Extracts image dimensions, color modes, and EXIF metadata locally via Pillow when offline, or delegates OCR and table extraction to
🔧 Refinery & Algorithmic Improvements
- Fence-Preserving Semantic Chunker & Router (
SemanticRouterFilter&ChromaManager): Keeps fenced code blocks (```/~~~), Markdown tables, headings, blockquotes, and bullet lists intact across paragraph splits and sliding-window chunks. - Precision Boilerplate Stripping (
HeuristicFilter): Replaces greedy link regexes with bounded character classes so academic citations ([1] ... [Paper](url)) and long GDPR/cookie compliance paragraphs remain untouched while standalone nav links and short footer banners are stripped. - Code-Safe ATX Heading Formatter (
AIFriendlyMarkdownFormatter): Normalizes#Headingto# Headingoutside code blocks without mutating shebangs (#!/usr/bin/env) or C macros (#include) inside fenced code. - Multiline CSV Escaping (
LiteParseExtractor): Parses quoted multiline CSV fields viaio.StringIOand escapes pipe characters (\|) and line breaks (<br>) to prevent broken Markdown tables.
🛡️ Security & Repo Hygiene
- Upload & URL Hardening: Sanitizes uploaded filenames against path traversal (
../../), prefixes temporary files with UUIDs to prevent concurrent collisions, enforces the 5MB offline limit on actual disk bytes, and blocks cloud metadata SSRF endpoints (169.254.169.254). - Clean Git Tree & Dependencies: Untracked binary
chroma_data/artifacts, removed scratch scripts, and standardizedrequirements.txtto clean UTF-8. - 10/10 Automated Verification Suites: Added end-to-end
pytestcoverage across Web, MarkItDown, LiteParse, Local ONNX KB, MCP JSON-RPC contracts, CLI, algorithmic mutation guards, and FastAPITestClientendpoints (run_tests.py&backend/tests/test_api.py).