Skip to content

pdf2wiki 0.2.2 — resilience & security hardening

Choose a tag to compare

View all tags
@Sevthered Sevthered released this 23 Jul 12:49
v0.2.2
b4d41ad

Resilience + security hardening from a book-grounded review (Kubernetes / microservices / API-security
references). No API changes — every existing command and flag behaves the same. Test suite 93 → 103 passing.

pip install --upgrade pdf2wiki    # 0.2.2

Security

  • Zip-slip guard — the mineru.net result ZIP (fetched from a server-supplied URL) is now validated
    member-by-member before extraction; a ..//absolute member is rejected instead of overwriting
    arbitrary files.
  • HTTPS enforced on the API base URL, the presigned upload URL, and the result-download URL before
    the Bearer token or the PDF is sent — a config/response downgrade to http:// is refused.
  • Untrusted-response handling — submit-response fields are validated (clear error instead of a raw
    KeyError); upstream error bodies are truncated + single-lined; presigned URLs are redacted (query
    stripped) in error messages.
  • Token hygiene — pdf2wiki.toml is now gitignored and the config documents preferring
    MINERU_API_TOKEN / token_file over an inline token.

Added

  • Retry with backoff + jitter on the cloud HTTP calls (submit / upload / result download) and
    bounded tolerance for transient errors mid-poll — a momentary network blip or HTTP 429/5xx no longer
    fails an otherwise-healthy conversion. Transient (429/5xx/network) vs permanent (4xx/API-error) is
    classified; permanent errors still fail fast. Tunable: [mineru_cloud].retries, retry_base_delay,
    poll_max_transient.
  • Batch circuit breaker — after [remote].max_consec_fail (default 3) consecutive book failures the
    batch re-probes executor health and aborts if the dependency is dead, instead of fast-failing every
    remaining book. A healthy probe continues (content failures don't trip it).
  • Per-pass cloud resume — each cloud pass writes a .done sentinel; a re-run reuses a completed pass
    instead of re-uploading and re-paying.
  • Manifest now records an error_class per failed book (transient / permanent / timeout / fetch / phase5).

Changed

  • Remote convert timeout now kills the remote job — the SSH convert wraps the converter in a
    server-side timeout Ns reaper, so a client-side timeout no longer leaves a zombie MinerU/vllm job
    pinning GPU VRAM. SSH/scp calls gain ConnectTimeout + BatchMode=yes; fetch() scp transfers are
    now timeout-bounded and remote paths are shlex-quoted.
  • Local MinerU timeout kills the whole process group (start_new_session=True + killpg), so
    orphaned vllm/torch workers no longer survive a timed-out pass.

Full changelog: CHANGELOG.md ·
PyPI: https://pypi.org/project/pdf2wiki/0.2.2/

Assets 4
Loading