v0.5.0 — multi-track enhancements

v0.5.0 — multi-track enhancements

Five independent feature tracks, each with its own tests and a backward-compatible contract. +53 new tests, 994 passing locally, 0 regressions.

Track A — International LLM providers

Adds Groq, Mistral, Together AI, Cohere, and Fireworks via INTERNATIONAL_PROVIDER_PRESETS (mirroring the CHINA_PROVIDER_PRESETS pattern). Resolved as get_backend("intl:groq" | "intl:mistral" | "intl:together" | "intl:cohere" | "intl:fireworks"). All OpenAI Chat-Completions compatible, shared OpenAICompatBackend under the hood. Multimodal variants auto-picked with multimodal=True. Total registered backends: 23 (was 18).

Track B — New document types

• W2 (US wage and tax statement) with W2LineItem + templates/w2.md
• IDCard (driver's license / passport / state ID) + templates/id_card.md
• MedicalClaim (CMS-1500 / UB-04) with MedicalClaimLineItem + templates/medical_claim.md

All three ship with Markdown templates (frontmatter + field descriptions + "Common mistakes" section) and register in SCHEMA_REGISTRY so Pipeline(schema="W2"), extract_one(..., schema="W2"), and template auto-discovery all work out of the box.

Track C — Eval harness per-field report + CI regression gate

run_dataset() now additionally returns per_field_accuracy (aggregated across all cases × all strategies). New check_regression(results, thresholds) returns failing threshold descriptions; verify_against_thresholds() raises EvalRegressionError for CI. New format_report(results) is a plain-text pretty-printer. Threshold grammar: per_strategy.<name>.<metric> and per_field_accuracy.<field>.<metric>.

A new CI workflow .github/workflows/eval-regression.yml wires the gate end-to-end with deliberately-permissive 0.0 thresholds for v0.5 (tighten in v0.6 once real backends have a baseline).

Track D — Async pipeline

AsyncPipeline wraps the existing Pipeline so callers can await apipe.arun(doc) and await apipe.arun_many(docs). New Backend.acomplete() method defaults to asyncio.to_thread(self.complete, req) so all shipped backends transparently support await with zero risk to existing code. process_batch_async() in batch.py mirrors the sync helper with a bounded concurrency= semaphore and an on_progress callback. BatchItemResult gains .success and .latency_s fields, kept consistent with existing .ok / .elapsed_seconds via __post_init__.

A new example notebook examples/notebooks/04_async_pipeline.ipynb walks through all the async surface in 17 cells — all runnable with MockBackend, no API key needed.

Track E — Registry helpers

list_backends() enumerates every registered backend as {name, family, auth} dicts. Gated backends (slowmock, nanonets) surface only when their IDP_ENABLE_* env var is set. Backend.__repr__ defaults to <ClassName name=...> for nicer debug logs.

Backward compatibility

Every change is additive. No public API renames or removals. Sync process_batch callers see no behavior change from the BatchItemResult field additions. All public re-exports are additive (3 in idp.llm, 3 in idp.eval, 1 in idp.pipeline).

Test coverage

Test file	Tests
tests/test_international_providers.py	6
tests/test_new_schemas.py	11
tests/test_eval_regression.py	12
tests/test_async_pipeline.py	14
tests/test_backend_registry_helpers.py	10
New tests total	53

Full suite: 994 passed, 2 skipped (pre-existing test_enhance_benchmark.py needs pytest-benchmark plugin), 0 regressions. python -c "import idp" still works.

Install

pip install --upgrade py-idp

Or pin: pip install py-idp==0.5.0

Acknowledgments

This release was assembled with parallel sub-agent work streams (4 feature tracks + 1 integration track + 2 follow-on tracks) by the maintainer's local agent harness.

v0.3.1.post1 — README link-rewrite (metadata-only)

Same code as v0.3.1. This release exists solely so PyPI re-renders the README with the absolute URLs from PR #12 (fix(readme): rewrite relative links to absolute GitHub URLs).

The current v0.3.1 wheel has 20 broken relative links in its rendered README — particularly the 'Languages: English / 繁體中文 / 简体中文' switcher at the top. This wheel carries the same code with the fixed README, so pip install py-idp==0.3.1.post1 renders correctly.

What's new

• README.md, README.zh-CN.md, README.zh-TW.md: every relative link rewritten to an absolute https://github.com/rollroyces/py-idp/blob/main/. PyPI renders the README as-is, so this fixes the displayed description.
• Helper script: scripts/fix-relative-readme-links.py — idempotent, handles nested badge syntax.

What changed

• pyproject.toml version: 0.3.1 → 0.3.1.post1
• src/idp/init.py version: 0.3.1 → 0.3.1.post1
• CITATION.cff: 0.3.1 → 0.3.1.post1

Verification

508 tests pass, ruff + mypy (59 files) clean.

Recommendation for users

pip install py-idp # installs the latest (= 0.3.1.post1)
pip install py-idp==0.3.1 # also works; identical code

v0.3.1 — Discoverability & pre-existing-bug pass

What's in 0.3.1

A discoverability + pre-existing-bug pass. Mostly invisible improvements that make the project installable, the GitHub sidebar meaningful, and the test suite honest about where it runs.

Discoverability

• CITATION.cff — "Cite this repository" button now appears on the GitHub sidebar; surfaces py-idp in academic dependency graphs.
• docs/SECURITY.md — supported-versions table + private reporting channel + 72 h / 14 d SLA. Unlocks the GitHub Security tab and enables private vulnerability reports.
• CONTRIBUTING.md — how to add a backend/schema, commit conventions, release flow.
• .github/ISSUE_TEMPLATE/{bug,feature,question}.yml — structured intake for first-time contributors.
• README badges — now link to live CI, PyPI, Downloads, Stars, and Citation.
• New README sections — Security, Citing, Contributing.
• pyproject keywords — expanded from 7 to 12 (added rag, multimodal, information-extraction, idp, schema-discovery, invoice, pdf).

Bug fixes

• tiktoken is now a real [project] dependency (was promised by the README but never declared; fresh installs and CI failed with ModuleNotFoundError on idp.chunker.TokenChunker).
• New [pdf-render] extra (pdf2image, Pillow) for the multimodal path (PdfPagesParser, NanonetsVLBackend); declared in pyproject.toml, installed by CI.
• CI install line uses .[dev,eval,api,pdf-render] so test_security_adversarial.py collects cleanly.
• Six test files no longer hard-code /Users/hermes/py-idp/... as the repo path; they derive REPO from Path(__file__). Tests now pass on any machine, not just the author's.
• PolicyCache.flush_now() race with the background flusher: a caller that flushes synchronously and then reads the policy file can no longer observe a stray .tmp because the flusher re-checks _dirty.is_set() inside the lock and skips no-op writes.

Install

pip install py-idp==0.3.1

For the multimodal OCR path:

pip install py-idp[hf-vlm,pdf-render]==0.3.1

Stats

458 tests passing across Python 3.10 / 3.11 / 3.12 on every commit. ruff + mypy clean. No breaking changes since 0.3.0.