Skip to content

OpenPapers 1.0.0

Latest
Latest

Choose a tag to compare

View all tags
@arrogance231 arrogance231 released this 06 Sep 05:24
· 1 commit to master since this release
v1.0.0
4b23241

OpenPapers 1.0.0

Provenance-first scholarly research infrastructure for Model Context Protocol clients: discovery, identification, retrieval, extraction, reconciliation, and evidence location across arXiv, Crossref, OpenAlex, Semantic Scholar, GitHub, and Hugging Face — with explicit refusal instead of fabrication.

Release integrity

  • Canonical evidence commit: 02c9ae4cf29fb4053ef4878a46954dfcee7c4e7a — every release gate below was executed at this SHA with recorded evidence. The tagged commit (4b23241) is its immediate successor and contains no shipped-source delta (documentation, recorded evidence, the new distribution gate, and a .gitignore entry only); npm run check was additionally re-run at the tagged tree.
  • The repository history was rewritten before this release to a clean release-candidate sequence; the v1.0.0 tag is the single immutable release point.
  • Version metadata is consistent across package.json, package-lock.json, server.json (MCP registry manifest), the runtime server version, and the changelog head, enforced by scripts/check-versions.mjs in npm run check.

Requirements and installation

  • Node.js >= 22.5 (tested on Node 22 and 24 in CI). No API keys required for core operation; provider keys improve rate limits.
  • npm install openpapers (npm publication pending), or run from source / Docker.
# stdio (default, for MCP clients)
npx openpapers

# Docker Compose stack (app + PostgreSQL/pgvector + GROBID)
docker compose up -d --wait

What's in the box

  • 37 MCP tools over stdio and HTTP transports covering search, identifier resolution, paper retrieval, PDF/HTML ingestion, fact/claim/parameter extraction, citation graphs, author and repository inspection, collections, ResearchPack import/export, and provenance-aware bibtex generation.
  • Provider adapters: arXiv, Crossref, OpenAlex, Semantic Scholar (scholarly), GitHub, Hugging Face (artifacts). Identifier-shaped queries probe arXiv/Crossref natively; arXiv-minted DOIs route to the arXiv probe.
  • Storage backends: SQLite (default, with ordered migrations) and PostgreSQL with pgvector. Both validated end-to-end from the published tarball.
  • Provenance model: every answer carries evidence with source class, locators, and provider-failure transparency; unavailable values are reported as NOT_REPORTED/UNKNOWN, never guessed.

Evaluation snapshot

All evidence files are committed under evals/results/ and name their producing commit.

Area Result
Offline retrieval (44 queries) Recall@1 0.420, Recall@5 0.750, Recall@10 0.856, MRR 0.773; identity accuracy 1.0 over 119 cases
Frozen retrieval holdout (5 queries) Recall@1/5/10 1.0
Live reliability (30 cases) Title-exact Recall@10 1.0, identifier resolution 1.0, identity correctness 1.0, zero silent provider failures; fuzzy discovery 0.5 (out of the 1.0 contract — see limitations)
Scoped canonical fact benchmark Precision 0.778, recall 1.0, F1 0.875 (LOPO F1 0.889)
Frozen real-source holdout (new, 20 cases / 40 tasks) Answer correctness 1.0, fact recall 1.0, fabricated answers 0, support-status accuracy 0.975 (single disclosed miss)
PDF parsing fidelity (new, 26-paper gold set) GROBID title token F1 0.976 (exact 0.885), author recall 0.774 / precision 0.825, abstract token F1 0.957; shipped PyMuPDF pipeline abstract first-page coverage 0.943
Runtime E2E (Docker) 12/12 steps including collection persistence across container restart; benchmark medians initialize 24.29 ms, tools/list 13.17 ms, tools/call 12.68 ms (fixture mode)
Distribution E2E (new) Pristine npm pack tarball install, spawned binary: 37 tools, search, collections, persistence across restart — passed on SQLite and PostgreSQL
Test suite 67 test files, 257 tests, npm audit: 0 vulnerabilities

Known limitations

  • Fuzzy discovery ≈ 0.5 Recall@10 — describing a paper from memory is not part of the 1.0 quality contract. Prefer a DOI or arXiv identifier. See limitations.
  • The default SQLite retrieval backend is a deterministic lexical-hash retriever, not semantic vector retrieval; semantic quality is unbenchmarked even with an embedding backend.
  • Extraction is heuristic and source-located, not independent scholarly verification. Cross-predicate exhaustive candidate-fact precision (0.134 on v4 dev, 0.563 on the v5 holdout) is intentionally lower than task-level metrics because the extraction layer is high-recall — see metric semantics.
  • GROBID fidelity is measured for header fields on arXiv-style papers; tables/equations/captions are not gold-scored.
  • Live evaluation covers one network region and timestamp; scheduled CI accumulates provider history over time.

Migration and compatibility policy

1.0.0 is the first stable release; there is no migration path from pre-1.0 development versions. Within 1.x: the 37-tool surface, tool schemas, provenance/evidence shapes, and storage schemas are stable — deprecations will be announced at least one minor version ahead. SQLite migrations are automatic and idempotent; PostgreSQL schema migrations are documented in postgres-migrations.md.

Security

Report vulnerabilities privately via GitHub security advisories. Downloaded papers and repository content are treated as untrusted data and never executed.

Full gate record

See RELEASE_CHECKLIST.md and the test plan for every claim's recorded evidence.

Assets 2
Loading