Repository navigation
PR 60 Crossref Reference Lists
James Kent edited this page Oct 9, 2026
·
1 revision
Opened 2026-10-09. Frozen once merged. The system as it stands is described in Design. Measurements: PR-58-Measurements.
#58 reads what a download marks. A PDF marks
nothing, about 30% of ACE pages hold no list, and an Elsevier list carries a DOI
on only 16% of its entries. This change adds reflist, a stage that fetches each
paper's reference list as its publisher deposited it with Crossref, and has
references use it where the download falls short.
Each of the four APIs tested could have supplied it:
- Crossref is ordered, carries a DOI on 92% of entries, and its references are open facts. Its gaps are whole publishers that deposit nothing (APA, AMA, some of J Neurosci), not partial lists.
- OpenAlex is unordered, so a numbered citation can't index it. It names Crossref's bare-DOI entries instead: authors, year and PMID, 100 DOIs per call.
- Europe PMC identifies entries almost only by PMID, and turns those it cannot match into "AUTHOR UNKNOWN" placeholders.
- Semantic Scholar withholds the list for most Elsevier papers.
-
Identifiers. The source's own entries borrow the DOIs and PMIDs they lack.
They are tied by DOI, then the publisher's key (Crossref often keeps the XML's
B12), then first author and year when only one entry matches, then the same position in a list of about the same length. Each copied id records its provider. -
Lists for sources that mark nothing. A PDF, or a page with no links, takes
Crossref's list. Its citations are then found by the heuristics in
services/citation_markers.py, each carrying the confidence its kind of marker earned against the exact XML links. -
Freshness.
referencesincludes the list's fingerprint, so an article read before its list arrived is read again once it does.
- Crossref's order differs from the printed numbering for about 8% of entries, so numbered matches against its list are 0.87 precise, against 0.997 with the XML's own list.
- Splitting a PDF's own reference section gets within 10% of Crossref's count for only 19 of 36 papers, because multi-column pages run entries together. It is not used.