Skip to content

v0.1.0

Choose a tag to compare

@minhtridinh-kayden minhtridinh-kayden released this 12 Jul 06:10
6b6df49

First tagged release. Covers the full M0–M5 feature set (VIP-1 … VIP-59).

Added

  • Public APIviparse.load(source, *, output, encoding, ocr, normalize, max_bytes, cache, chunk, settings) and lazy viparse.load_batch(...), returning Unicode-NFC
    Documents as markdown / text / json.
  • Layered configurationoutput / encoding / ocr / normalize / max_bytes resolve
    from function args → VIPARSE_* env vars → a viparse.toml file → the built-in defaults. A
    validating Settings (via load_settings()) raises ConfigError on a bad value.
  • RAG chunking — opt-in chunk=ChunkOptions(max_tokens, overlap_tokens) splits a document
    into retrieval-sized, section-aware Chunks (never splitting a table row) with per-chunk
    section / page / sheet metadata and an ordinal index.
  • Framework integrationsto_langchain_documents(doc) / to_llamaindex_documents(doc)
    map a Document (chunk-aware) onto LangChain / LlamaIndex document types, provenance
    flattened into their metadata. Lazy behind viparse[langchain] / viparse[llamaindex].
  • CLIviparse <files> -o md|text|json (globs, directories, --out,
    --encoding/--ocr/--normalize) and viparse doctor (engine + binary availability).
  • Extraction engines — DOCX, XLSX, digital PDF, scanned PDF via OCR (viparse[ocr]),
    and legacy binary .doc/.xls via LibreOffice — all thin adapters behind extras so the
    core install stays dependency-free.
  • Vietnamese normalization (the moat) — legacy TCVN3 / VNI / VISCII → Unicode NFC,
    with font-signal detection, opt-in content-based detection (encoding="auto"), and text
    cleanup. Output is always NFC.
  • Structured output — headings, GFM tables, and a versioned JSON schema
    (viparse.SCHEMA_VERSION).
  • Untrusted-input safety — configurable file-size limit, zip-decompression-bomb guard,
    and per-engine process timeouts (UnsafeInput).
  • Caching — content-hash MemoryCache / DiskCache to skip re-parsing unchanged files.
  • Parallel batchload_batch(..., workers=N) with bounded concurrency and per-source
    error isolation.