Releases: TrizenX/viparse
Releases · TrizenX/viparse
Release list
v0.1.6
Documentation and tooling only — src/ is byte-identical to v0.1.5, so upgrading changes no behaviour.
Cut so the PyPI project page, which renders the README, stops serving incomplete install instructions.
Fixed
- The installation section omitted the
rtf,langchainandllamaindexextras and never stated the Python floor, so three of the seven extras were undiscoverable to anyone arriving via PyPI rather than GitHub (VIP-77).
Changed
- The README now documents the
route → extract → normalize → structurepipeline, how encoding detection picks a charmap, and where viparse sits next to general-purpose loaders. It links the project website at https://viparse.trizenx.com (VIP-77). ruffandmypyare capped to a minor series. Both gate CI, and an unpinned release of either turnsqualityred on unrelated work — ruff 0.16 did exactly that and blocked every open PR for three days (VIP-78, VIP-79).
Full changelog: v0.1.5...v0.1.6
v0.1.5
Added
- RTF support — a new
.rtftext-extraction engine (behind theviparse[rtf]extra, wrapping the pure-Pythonstriprtf), detected by magic bytes (a leading UTF-8 BOM is tolerated). A legacy-encoded.rtfis normalized like any other font-less source — via content detection (encoding="auto") or an explicitencoding=override. The engine deliberately does not infer an encoding from the RTF font table (which lists declared rather than applied fonts) to avoid converting a Unicode body through a legacy charmap (VIP-75).
Full changelog: https://github.com/minhtridinh-kayden/viparse/blob/main/CHANGELOG.md
v0.1.4
Added
- VPS legacy encoding — the VPS (Vietnamese Professional System) → Unicode conversion table alongside TCVN3, VNI, and VISCII, selectable via an explicit
encoding="vps"override. Cross-verified against four independent sources; intentionally excluded from content auto-detection to avoid mis-converting VISCII text (VIP-71).
Changed
- Per-run encoding detection — mixed-encoding detection now works at run granularity: a single paragraph mixing a legacy
.Vn*/VNI-run with a Unicode run is converted run-by-run, so a Unicode run (e.g. one containing®) is no longer mangled. Same-encoding runs are coalesced so legacy base+mark forms are never severed; single-encoding and per-block output stay byte-for-byte identical (VIP-72).
Full changelog: https://github.com/minhtridinh-kayden/viparse/blob/main/CHANGELOG.md
v0.1.3
Fixed
- Per-block encoding detection for mixed-encoding documents — a document that mixes a legacy
.Vn*/VNI-run with already-Unicode runs is now detected and converted per block, by each block's own font signal, instead of applying one file-wide verdict. A Unicode run containing a character that is also a legacy surface byte (e.g.viparse® 2026) is no longer corrupted (viparseđ 2026), while the adjacent legacy run still converts. Single-encoding documents are unaffected — their output is byte-for-byte identical (VIP-65).
Full changelog: https://github.com/minhtridinh-kayden/viparse/blob/main/CHANGELOG.md
v0.1.2
Dependency compatibility only — no code or behavior changes.
Changed
v0.1.1
Documentation and packaging only — no code or behavior changes.
Added
- MIT license — added a
LICENSEfile and declaredlicense = "MIT"in the package
metadata (VIP-62). - README — installation instructions (
pip install viparseand extras), a usage section,
and released status linking PyPI (VIP-62). - PyPI publishing — a GitHub Actions workflow publishes to PyPI via Trusted Publishing
(OIDC, no stored token) when a release is published (VIP-61).
Fixed
- Project URLs — corrected the repository URL and added Homepage / Changelog links shown on
the PyPI page (VIP-61).
v0.1.0
First tagged release. Covers the full M0–M5 feature set (VIP-1 … VIP-59).
Added
- Public API —
viparse.load(source, *, output, encoding, ocr, normalize, max_bytes, cache, chunk, settings)and lazyviparse.load_batch(...), returning Unicode-NFC
Documents as markdown / text / json. - Layered configuration —
output/encoding/ocr/normalize/max_bytesresolve
from function args →VIPARSE_*env vars → aviparse.tomlfile → the built-in defaults. A
validatingSettings(viaload_settings()) raisesConfigErroron a bad value. - RAG chunking — opt-in
chunk=ChunkOptions(max_tokens, overlap_tokens)splits a document
into retrieval-sized, section-awareChunks (never splitting a table row) with per-chunk
section/page/sheetmetadata and an ordinalindex. - Framework integrations —
to_langchain_documents(doc)/to_llamaindex_documents(doc)
map aDocument(chunk-aware) onto LangChain / LlamaIndex document types, provenance
flattened into theirmetadata. Lazy behindviparse[langchain]/viparse[llamaindex]. - CLI —
viparse <files> -o md|text|json(globs, directories,--out,
--encoding/--ocr/--normalize) andviparse doctor(engine + binary availability). - Extraction engines — DOCX, XLSX, digital PDF, scanned PDF via OCR (
viparse[ocr]),
and legacy binary.doc/.xlsvia LibreOffice — all thin adapters behind extras so the
coreinstall stays dependency-free. - Vietnamese normalization (the moat) — legacy TCVN3 / VNI / VISCII → Unicode NFC,
with font-signal detection, opt-in content-based detection (encoding="auto"), and text
cleanup. Output is always NFC. - Structured output — headings, GFM tables, and a versioned JSON schema
(viparse.SCHEMA_VERSION). - Untrusted-input safety — configurable file-size limit, zip-decompression-bomb guard,
and per-engine process timeouts (UnsafeInput). - Caching — content-hash
MemoryCache/DiskCacheto skip re-parsing unchanged files. - Parallel batch —
load_batch(..., workers=N)with bounded concurrency and per-source
error isolation.