Skip to content

Releases: oldantique/zotkit

v0.5.1

Choose a tag to compare

@oldantique oldantique released this 29 Jul 12:24

v0.5.1 — repository DOIs are not journal DOIs

Fixes the "SSRN Electronic Journal" trap: a preprint repository's own DOI (e.g. SSRN 10.2139/ssrn.*, registered in CrossRef as journal-article in a fake journal) could wrongly trigger the version-of-record upgrade, turning a working paper into a journalArticle.

  • The "is this a journal DOI?" decision is now a single two-layer predicate in zotkit.metadata, shared by create --arxiv and enrich --rebuild-record:
    1. repository-prefix blacklist — arXiv 10.48550, SSRN 10.2139, bioRxiv/medRxiv 10.1101, Research Square 10.21203, OSF 10.31219, ChemRxiv 10.26434, TechRxiv 10.36227, Preprints.org 10.20944 never count as journal DOIs;
    2. record-level guard — a CrossRef record whose container is "SSRN Electronic Journal" (or whose type is posted-content) is a repository posting, not a version of record.
  • Blocked upgrades keep the preprint and state the reason in the output.
  • Unchanged: genuine journal DOIs upgrade exactly as before; repository-DOI items still get plain fill-missing enrichment.

v0.5.0

Choose a tag to compare

@oldantique oldantique released this 29 Jul 09:59

zotkit enrich — complete existing items, in place

Incomplete library items — missing abstracts, missing DOIs, author lists truncated at import time, preprints that have since been published — could previously only be fixed by delete-and-recreate (unacceptable: the item key changes, and keys are the stable handle downstream manifests reference) or hand-written pyzotero scripts. Now it's a first-class command:

zotkit enrich --key K [K …]              # dry-run: per-item list of what would be filled
zotkit enrich --key K --apply
zotkit enrich --key K --rebuild-record --apply   # preprint → journal record, same item

The item key never changes. Every write is an in-place fetch→modify→update carrying the item version — a concurrent edit fails that one item with 412 (reported, batch continues) instead of clobbering.

What it does

  • Finds the authoritative identifier (DOI field, or arXiv id from archiveID / arXiv: Extra line / arxiv.org URL / arXiv's DataCite DOI; journal DOI wins per the v0.4.2 version-of-record rule) and pulls the record through the same rate-limited fetchers create uses. No identifier → skipped as needs-identifier.
  • Fills empty fields only — abstract, date, url, volume/issue/pages, ISSN, journal abbreviation, DOI, … Existing non-empty values are never overwritten.
  • Creators: extended only when the current list is a same-order prefix of the authoritative one (the classic 4-of-227 truncation); any other difference is reported and left alone.
  • Stale-stamp rule: an abstract-source: stamp with no abstract means an owner deliberately removed it — enrich refuses to re-add and reports NEEDS OWNER. Every abstract it writes is stamped with its true origin.
  • Extra only gains lines; tags, collections, relations, and attachments are never touched.
  • --rebuild-record: a preprint with a journal DOI is upgraded to the journal record in the same item — itemType, published title/date/venue/volume/pages — with arXiv: <id> preserved in Extra and the child-item count asserted unchanged.
  • Summary + exit code: enriched N / skipped M (needs-identifier, stale-stamp, up-to-date) / failed K; non-zero exit only when something failed.

Also

  • The CrossRef polite-pool contact is now configurable: export ZOTKIT_MAILTO=you@example.com (it should be a reachable address — distributors of zotkit-based tools, please set your own).

Install/upgrade: pipx upgrade zotkit

v0.4.2

Choose a tag to compare

@oldantique oldantique released this 29 Jul 08:08

Version of record for --arxiv

arXiv reports two kinds of DOI: its own 10.48550/* DataCite DOI, and a journal DOI meaning the paper was formally published. Citing the version of record is the correct scholarly practice, so --arxiv now acts on the difference:

  • Journal DOI present → the item is built from CrossRef instead of as a preprint: proper item type, venue, volume/pages (article numbers included — think PRL's 061102), formal publication date. The arXiv identity is preserved: arXiv: <id> in Extra, url stays the open-access abs page (the journal link is carried by the DOI field), the arXiv abstract fills in when CrossRef has none — very common — and the PDF still downloads from arXiv (green OA; --no-pdf unchanged). Dry-run announces it: 1602.03837 → journal DOI 10.1103/PhysRevLett.116.061102, building journal record.
  • CrossRef lookup fails (network, 404) → the item quietly degrades to the familiar preprint record with a warning; it never fails the batch.
  • Only a DataCite DOI, or no DOI → exactly the v0.4.1 preprint behavior.

Abstract provenance

Any abstract zotkit writes — via --arxiv or --doi — is now tagged in Extra with abstract-source: arxiv or abstract-source: crossref, naming where the text actually came from (the hybrid journal-record-with-arXiv-abstract case is labeled arxiv). No abstract, no line. Extra lines always append to existing content, never overwrite.

The CrossRef requests ride the existing per-host rate-limit layer — batches stay polite with zero caller involvement.

Install/upgrade: pipx upgrade zotkit

v0.4.1

Choose a tag to compare

@oldantique oldantique released this 29 Jul 07:37

Batch import, with rate limiting where it belongs

--arxiv and --doi now accept multiple identifiers (space- or comma-separated, URLs and bare ids mixed), and all request pacing moved into zotkit itself — callers, human or agent, never think about rate limits again.

What changed

  • zotkit create --arxiv id1 id2 id3 … — metadata for the whole batch is one export-API request (per 50 ids), entries mapped back to the ids you asked for regardless of response order or version resolution. A nonexistent or malformed id gets its own error line; the rest of the batch proceeds.
  • With --apply, PDFs download ≥3 seconds apart — arXiv's terms of use ask for 1 request per 3 s on a single connection, and v0.4.0's one-id-per-call design pushed that responsibility onto the caller (N quick calls → 429s, then hanging connections). The interval now lives in the request layer and applies to metadata and PDFs alike.
  • zotkit create --doi doi1 doi2 … — CrossRef has no batch endpoint, so requests go out sequentially with polite 1 s spacing (same mailto UA and type mapping as before).
  • Batch semantics match --file: dry-run prints every fetched record; --apply runs each item through the existing lint/dedup/create path and finishes with batch: created N, skipped M dup, failed K. Exit code is non-zero iff something failed.
  • 429 handling is unchanged (one retry honoring Retry-After — no aggressive backoff), and single-id invocations behave exactly as in v0.4.0.

Install/upgrade: pipx upgrade zotkit

v0.4.0

Choose a tag to compare

@oldantique oldantique released this 29 Jul 07:21

zotkit create --arxiv / --doi — one-shot import from an identifier

Two new ways to create items, no JSON file needed. Both are dry-run by default (printing the full fetched record), and --apply executes:

zotkit create --arxiv <id-or-url>

  • Accepts 2401.12345, versioned ids, old-style ids (math/0211159), or any arxiv.org abs/pdf URL.
  • Fetches from the arXiv export API: title, all authors, abstract (stored verbatim — line breaks and LaTeX untouched), date, abs-page URL, DOI (journal DOI when arXiv has one, else the 10.48550/arXiv.* DataCite DOI), as a preprint item with repository/archiveID set.
  • With --apply, the PDF is downloaded and attached automatically through the existing WebDAV / Zotero Storage auto-detection; --no-pdf skips that.

zotkit create --doi <doi-or-url>

  • Fetches from CrossRef (polite pool): full creator list, abstract, date, container title, volume/issue/pages, ISSN/ISBN, with CrossRef types mapped to proper Zotero item types (journal-articlejournalArticle, proceedings-articleconferencePaper, books, chapters, …). Unknown types are an error, never a guess.
  • No PDF download (publisher PDFs are usually paywalled) — the CLI tells you how to zotkit attach one manually.

Shared behavior

  • --collection "Name" and --tags field:ai,status:to-read file the new item directly; tag conventions are enforced as always.
  • Dedup by DOI/title applies as with --file; the three sources are mutually exclusive.
  • Readable errors for network failures, rate limits, unknown ids/DOIs.

Out of scope by design: translation-server / arbitrary-URL scraping — zotkit stays a daemon-free CLI.

Install/upgrade: pipx install zotkit or pipx upgrade zotkit

zotkit 0.3.0 — doctor, --version, 中文文档

Choose a tag to compare

@oldantique oldantique released this 11 Jul 08:08

Getting-started hardening before wider announcement:

  • zotkit doctor — one-command setup validation: config file, conventions, Zotero API reachability + write access, and attachment storage (probes WebDAV writability, or confirms Zotero Storage mode). Every failing check says how to fix it.
  • zotkit --version — for bug reports.
  • README.zh-CN.md — Chinese quickstart (install, config walkthrough, mirror hint, common commands).
  • README: badges, and an honest Limits & roadmap section (client-side search performance on very large libraries; group libraries untested; MCP wrapper planned).

zotkit 0.2.0 — Zotero Storage support

Choose a tag to compare

@oldantique oldantique released this 11 Jul 08:01

attach and fetch now work for both attachment storage modes, auto-detected from your .env:

  • WEBDAV_* configured → direct WebDAV read/write (unchanged, tested end-to-end)
  • no WEBDAV_*Zotero Storage via the Web API's official file endpoints (new)

Nothing to configure — just omit the WebDAV lines if you use Zotero Storage.

Note: attach()'s return field webdav_ok is now storage ("webdav" or "zotero").

zotkit 0.1.1

Choose a tag to compare

@oldantique oldantique released this 11 Jul 07:23

Windows compatibility fix: all text file IO (backups, .env, conventions.toml, item JSON) now explicitly UTF-8 — previously libraries with non-ASCII titles could crash backup or misread config on Windows, where the default locale encoding is not UTF-8.

zotkit 0.1.0

Choose a tag to compare

@oldantique oldantique released this 11 Jul 06:58

First public release.

  • Headless Zotero library management via the Web API: find / create (dedup, dry-run) / tag / status / move / backup
  • Direct WebDAV attachment upload & download (Zotero's K.zip/K.prop format) — no desktop app required
  • Optional tag conventions enforced in code via conventions.toml
  • zotkit CLI + from zotkit import Zot Python API