Skip to content

v0.4.1

Choose a tag to compare

@oldantique oldantique released this 29 Jul 07:37

Batch import, with rate limiting where it belongs

--arxiv and --doi now accept multiple identifiers (space- or comma-separated, URLs and bare ids mixed), and all request pacing moved into zotkit itself — callers, human or agent, never think about rate limits again.

What changed

  • zotkit create --arxiv id1 id2 id3 … — metadata for the whole batch is one export-API request (per 50 ids), entries mapped back to the ids you asked for regardless of response order or version resolution. A nonexistent or malformed id gets its own error line; the rest of the batch proceeds.
  • With --apply, PDFs download ≥3 seconds apart — arXiv's terms of use ask for 1 request per 3 s on a single connection, and v0.4.0's one-id-per-call design pushed that responsibility onto the caller (N quick calls → 429s, then hanging connections). The interval now lives in the request layer and applies to metadata and PDFs alike.
  • zotkit create --doi doi1 doi2 … — CrossRef has no batch endpoint, so requests go out sequentially with polite 1 s spacing (same mailto UA and type mapping as before).
  • Batch semantics match --file: dry-run prints every fetched record; --apply runs each item through the existing lint/dedup/create path and finishes with batch: created N, skipped M dup, failed K. Exit code is non-zero iff something failed.
  • 429 handling is unchanged (one retry honoring Retry-After — no aggressive backoff), and single-id invocations behave exactly as in v0.4.0.

Install/upgrade: pipx upgrade zotkit