Skip to content

Releases: TrizenX/viparse

v0.1.6

Choose a tag to compare

@minhtridinh-kayden minhtridinh-kayden released this 28 Jul 03:46
4691daf

Documentation and tooling only — src/ is byte-identical to v0.1.5, so upgrading changes no behaviour.

Cut so the PyPI project page, which renders the README, stops serving incomplete install instructions.

Fixed

  • The installation section omitted the rtf, langchain and llamaindex extras and never stated the Python floor, so three of the seven extras were undiscoverable to anyone arriving via PyPI rather than GitHub (VIP-77).

Changed

  • The README now documents the route → extract → normalize → structure pipeline, how encoding detection picks a charmap, and where viparse sits next to general-purpose loaders. It links the project website at https://viparse.trizenx.com (VIP-77).
  • ruff and mypy are capped to a minor series. Both gate CI, and an unpinned release of either turns quality red on unrelated work — ruff 0.16 did exactly that and blocked every open PR for three days (VIP-78, VIP-79).

Full changelog: v0.1.5...v0.1.6

v0.1.5

Choose a tag to compare

@minhtridinh-kayden minhtridinh-kayden released this 18 Jul 17:51
5a911b1

Added

  • RTF support — a new .rtf text-extraction engine (behind the viparse[rtf] extra, wrapping the pure-Python striprtf), detected by magic bytes (a leading UTF-8 BOM is tolerated). A legacy-encoded .rtf is normalized like any other font-less source — via content detection (encoding="auto") or an explicit encoding= override. The engine deliberately does not infer an encoding from the RTF font table (which lists declared rather than applied fonts) to avoid converting a Unicode body through a legacy charmap (VIP-75).

Full changelog: https://github.com/minhtridinh-kayden/viparse/blob/main/CHANGELOG.md

v0.1.4

Choose a tag to compare

@minhtridinh-kayden minhtridinh-kayden released this 18 Jul 17:11
b48e0dd

Added

  • VPS legacy encoding — the VPS (Vietnamese Professional System) → Unicode conversion table alongside TCVN3, VNI, and VISCII, selectable via an explicit encoding="vps" override. Cross-verified against four independent sources; intentionally excluded from content auto-detection to avoid mis-converting VISCII text (VIP-71).

Changed

  • Per-run encoding detection — mixed-encoding detection now works at run granularity: a single paragraph mixing a legacy .Vn*/VNI- run with a Unicode run is converted run-by-run, so a Unicode run (e.g. one containing ®) is no longer mangled. Same-encoding runs are coalesced so legacy base+mark forms are never severed; single-encoding and per-block output stay byte-for-byte identical (VIP-72).

Full changelog: https://github.com/minhtridinh-kayden/viparse/blob/main/CHANGELOG.md

v0.1.3

Choose a tag to compare

@minhtridinh-kayden minhtridinh-kayden released this 18 Jul 04:23
b068cd4

Fixed

  • Per-block encoding detection for mixed-encoding documents — a document that mixes a legacy .Vn*/VNI- run with already-Unicode runs is now detected and converted per block, by each block's own font signal, instead of applying one file-wide verdict. A Unicode run containing a character that is also a legacy surface byte (e.g. viparse® 2026) is no longer corrupted (viparseđ 2026), while the adjacent legacy run still converts. Single-encoding documents are unaffected — their output is byte-for-byte identical (VIP-65).

Full changelog: https://github.com/minhtridinh-kayden/viparse/blob/main/CHANGELOG.md

v0.1.2

Choose a tag to compare

@minhtridinh-kayden minhtridinh-kayden released this 12 Jul 07:04
d862d47

Dependency compatibility only — no code or behavior changes.

Changed

  • Widened dependency upper bounds so viparse installs alongside newer major releases:
    pillow<13 (extra ocr), langchain-core<2 (extra langchain), and reportlab<6
    (dev only). Verified against pillow 12, langchain-core 1.x, and reportlab 5 (#47, #48).

v0.1.1

Choose a tag to compare

@minhtridinh-kayden minhtridinh-kayden released this 12 Jul 06:45
91e3f70

Documentation and packaging only — no code or behavior changes.

Added

  • MIT license — added a LICENSE file and declared license = "MIT" in the package
    metadata (VIP-62).
  • README — installation instructions (pip install viparse and extras), a usage section,
    and released status linking PyPI (VIP-62).
  • PyPI publishing — a GitHub Actions workflow publishes to PyPI via Trusted Publishing
    (OIDC, no stored token) when a release is published (VIP-61).

Fixed

  • Project URLs — corrected the repository URL and added Homepage / Changelog links shown on
    the PyPI page (VIP-61).

v0.1.0

Choose a tag to compare

@minhtridinh-kayden minhtridinh-kayden released this 12 Jul 06:10
6b6df49

First tagged release. Covers the full M0–M5 feature set (VIP-1 … VIP-59).

Added

  • Public APIviparse.load(source, *, output, encoding, ocr, normalize, max_bytes, cache, chunk, settings) and lazy viparse.load_batch(...), returning Unicode-NFC
    Documents as markdown / text / json.
  • Layered configurationoutput / encoding / ocr / normalize / max_bytes resolve
    from function args → VIPARSE_* env vars → a viparse.toml file → the built-in defaults. A
    validating Settings (via load_settings()) raises ConfigError on a bad value.
  • RAG chunking — opt-in chunk=ChunkOptions(max_tokens, overlap_tokens) splits a document
    into retrieval-sized, section-aware Chunks (never splitting a table row) with per-chunk
    section / page / sheet metadata and an ordinal index.
  • Framework integrationsto_langchain_documents(doc) / to_llamaindex_documents(doc)
    map a Document (chunk-aware) onto LangChain / LlamaIndex document types, provenance
    flattened into their metadata. Lazy behind viparse[langchain] / viparse[llamaindex].
  • CLIviparse <files> -o md|text|json (globs, directories, --out,
    --encoding/--ocr/--normalize) and viparse doctor (engine + binary availability).
  • Extraction engines — DOCX, XLSX, digital PDF, scanned PDF via OCR (viparse[ocr]),
    and legacy binary .doc/.xls via LibreOffice — all thin adapters behind extras so the
    core install stays dependency-free.
  • Vietnamese normalization (the moat) — legacy TCVN3 / VNI / VISCII → Unicode NFC,
    with font-signal detection, opt-in content-based detection (encoding="auto"), and text
    cleanup. Output is always NFC.
  • Structured output — headings, GFM tables, and a versioned JSON schema
    (viparse.SCHEMA_VERSION).
  • Untrusted-input safety — configurable file-size limit, zip-decompression-bomb guard,
    and per-engine process timeouts (UnsafeInput).
  • Caching — content-hash MemoryCache / DiskCache to skip re-parsing unchanged files.
  • Parallel batchload_batch(..., workers=N) with bounded concurrency and per-source
    error isolation.