docflow zotero: add group-wide export, CLI export, and per-collection routing

[yarikoptic]

docflow zotero: add group-wide export, CLI export, and per-collection routing

Summary

While using docflow to manage the reference library for a multi-institution
grant proposal (OS4LS Track 2: "BIDS 2.0+ ecosystem"), we ran into three
gaps that block a "one command from Zotero to refs.bib" workflow:

1. No CLI for BibTeX export. ZoteroClient.export_bibtex() exists in the
   Python API but has no docflow zotero export subcommand. Every downstream
   Makefile has to reinvent the invocation.
2. export_bibtex() requires a collection key. For grants and papers
   where the whole group is the bibliography, there is no natural
   collection key — the top of the library is addressed as
   /groups/{id}/items with no /collections/{key} segment.
3. docflow zotero push targets a single collection at a time. When
   ingesting a curated DOI list where some entries belong in the primary
   collection and others in a "related / adjacent" sub-collection, users
   currently have to split the input and run push twice.

We worked around (1) and (2) with a small stdlib helper
(scripts/zotero-export-bib.py) that talks to the Zotero Web API directly
and paginates via Total-Results. Filing here so docflow can absorb it
properly.

Motivation / concrete use case

Setup: shared Zotero group 5111637 ("BIDS 2.0+ ecosystem"), grant
proposal repo, references cited via pandoc citeproc in
subs/biographies.md. We need make BIDS.bib to regenerate the local
.bib on demand.

Two flavours of citation are being curated:

• Core: BIDS spec, validators, PyBIDS, BIDS-Apps, OpenNeuro, DataLad,
  NWB, HED, brainlife, Nipoppy, Boutiques, etc. (≈18 refs) — belong in the
  top level of the group.
• Adjacent / biosketch context: pyAFQ, DIPY, QSIPrep, VAME, GuPPy,
  a Cell paper for a PI's dissertation lineage, etc. (≈8 refs) — cited in
  the PI/Co-PI biographies but only distantly related to the standard.
  Would live cleanly in a biosketch-refs sub-collection.

Right now the second bucket has to be either (a) inlined into the top-level
library and cluttering BIDS.bib, or (b) manually copied into a
sub-collection through the Zotero UI. Neither scales as new co-PIs are
added.

Proposed changes

1. docflow zotero export (CLI)

docflow zotero export [--collection KEY] [--format bibtex|csljson|ris] [--output PATH]

• If --collection omitted, export the whole group (root level).
• If --collection given, delegate to existing export_bibtex() logic.
• Default --format=bibtex; add csljson and ris behind the same
  Zotero format= parameter — free from the API's side.
• Default output = stdout; -o/--output writes to file.
• Config-driven default group/collection from .docflow/config.yaml, so a
  bare docflow zotero export works once configured.

2. Group-wide export in ZoteroClient

Make collection_key optional on export_bibtex():

def export_bibtex(self, collection_key: str | None = None, output_path: Path | None = None) -> str:
    ...

When None, call self.zot.items(format="bibtex") instead of
self.zot.collection_items(...). Everything else (writing to file,
logging) stays the same.

3. Pagination for format=bibtex

Zotero caps format=bibtex responses at 100 items per request — beyond
that, subsequent items are silently dropped. Add explicit pagination via
the Total-Results response header (or Link: rel="next") so libraries
larger than 100 export correctly. Reference implementation:
scripts/zotero-export-bib.py in the
consuming repo.

4. Per-collection routing on docflow zotero push (nice-to-have)

Right now --collection sets a single destination collection for all
input citations. For "some go here, some go there" flows, allow either:

• A per-input mapping in the citation JSON:

  {"doi": "10.1038/sdata.2016.44", "collection": "ABCD1234"}

• Or a --rules flag pointing at a small YAML:

  default_collection: ABCD1234       # main "BIDS 2.0+" collection
  routes:
    - match: {tag: biosketch}
      collection: EFGH5678           # sub-collection: biosketch refs
    - match: {doi_prefix: "10.1016/j.cell"}
      collection: EFGH5678

Even without routing, an ergonomic hook to bulk-import from a plain DOI
list would help:

docflow zotero add-dois --collection ABCD1234 < dois.txt
docflow zotero add-dois --collection EFGH5678 < biosketch-dois.txt

(Today the equivalent is a JSON blob per DOI — friction that discourages
curating references outside the Zotero UI.)

5. docflow init template update

The generated Makefile should include the bib / refs.bib target
alongside the existing zotero-check and zotero-push. Roughly:

refs.bib: ; docflow zotero export --output $@

That closes the loop for pandoc/LaTeX users who don't need the whole
comments-extraction pipeline but do want Zotero → .bib on demand.

What we're not asking for (out of scope for this issue)

• A general-purpose BibTeX ↔ Zotero round-trip (Better BibTeX already
  covers this well from the Zotero side).
• Support for non-Zotero backends (Mendeley etc.).
• Citation-key generation policies — leave to Better BibTeX or downstream
  tooling.

Willing to contribute

Happy to open a PR for (1) + (2) + (3) — the code is essentially the
helper script above merged into ZoteroClient plus a Click wrapper. (4)
and (5) are worth their own follow-up issues once (1)-(3) land.

Environment

• docflow: master @ (commit at time of filing)
• pyzotero: (version)
• Python: 3.12
• Zotero library type: group (id=5111637)

---

Filed as part of the OS4LS "BIDS 2.0+ ecosystem" Full Application
workflow (July 2026).