v3.0.0 - Streaming Batch Writes & Native Markdown Support

Release v3.0.0 🚀

Key Features

Streaming Batch Writes

• Per-batch database persistence for reliable large-scale document processing
• Checkpoint after every batch with cumulative progress tracking
• Memory-efficient processing of 1000+ documents

Native Markdown Support

• Extract and query document structure using Accept: text/markdown header
• Parse H1-H6 headers with anchors for structured navigation
• Handle special characters and nested header structures

Enhanced Checkpoint System

• Full DB state tracking with cumulative statistics
• Automatic checkpoint serialization and retrieval
• Consistency verification between checkpoint and database state

Crash Recovery with Automated Integrity Validation

• Resume from any failure point with zero data loss
• Automated integrity validation on recovery
• ACID guarantees: Atomicity, Consistency, Isolation, Durability
• Recovery performance: < 5 seconds

100% Test Coverage

• 97/97 tests passing across unit, integration, and acceptance suites
• 61 unit tests, 15 integration tests, 21 acceptance tests
• Performance benchmarks validated: <200MB memory, <60 seconds for 1000 documents

Breaking Changes

• Streamlined batch processing API: Mandatory document validation (id/content required)
• Checkpoint format: Now includes cumulative processing statistics

Installation

git clone https://github.com/Charpup/openclaw-docs-rag.git ~/.openclaw/skills/openclaw-docs-rag
cd ~/.openclaw/skills/openclaw-docs-rag
npm install

Documentation

• Full API documentation in SKILL.md
• Technical specifications in spec/v3.0/
• Test documentation in test/v3.0/
• Test report: TEST_REPORT_PHASE5.md

Full Changelog

See CHANGELOG.md for complete version history.