v0.45.3

v0.45.3

Security fix, and a correction to v0.45.2. Take this instead of 0.45.2 if you
run the HTTP transport, or if you call operate/fetch on records you did not
publish yourself.

Fixed

• A redirect bypassed the egress guard shipped in 0.45.2. That guard validated
  the URL it was handed, but the HTTP client follows redirects — so a record
  carrying a perfectly PUBLIC URL that 302s to http://127.0.0.1:9200/ reached
  it with the check satisfied. Measured before this fix: the guard was consulted
  once, about the entry URL, while the redirect target was fetched unchecked.

  0.45.2 does block a record pointing DIRECTLY at private address space. It did
  not block the same thing behind one redirect.

  Validation now also runs as an HTTP request event hook, which is the only layer
  that sees every address actually connected to, so it binds each hop rather than
  the first. The call-site checks remain: they fail before any I/O and name the
  file being refused, which a transport-level error cannot.

  After the fix the redirect target receives zero requests.

Unchanged from 0.45.2, and still true: a host that does not resolve is allowed
through, because it points nowhere and refusing it would block offline callers
while closing nothing; and this does not defeat DNS rebinding, which would
require pinning the checked address into the connection itself.

v0.45.2

v0.45.2

Security fix. Upgrade from 0.45.1 if you run the HTTP transport, or if you call
operate/fetch on records you did not publish yourself.

Fixed

• A record's file URL could make the server read addresses the caller cannot
  reach. operate and fetch take that URL straight from an upstream record,
  and every source that accepts uploads — Zenodo, HuggingFace, figshare, OpenML —
  lets it be anything. A record whose file was NAMED data.csv while pointing at
  http://127.0.0.1:9200/ or http://169.254.169.254/ was fetched, and with
  op='head' the body came back to the caller as rows.

  None of the existing controls covered this, each for a structural reason:

  • The DuckDB filesystem lock added in 0.45.0 is applied AFTER the source is
    materialized, and must be — a lazily-evaluated view would be blocked along
    with the legitimate read. It protects the user's SQL, not the source read.
  • The scheme allowlist governs HOW we fetch, never WHERE.
  • The operable-file check gates on the file's NAME, so the extension test never
    looked at the URL at all.

  A new egress guard resolves the host and refuses private, loopback, link-local,
  reserved, multicast and unspecified addresses before any request is made.

  Harmless over stdio, where the server is the caller's own child process and
  shares its network position. The exposure was SSRF with response exfiltration
  under --transport http, where the server may sit in a network the caller
  cannot otherwise reach.

  Set DATA_AGGREGATOR_MCP_ALLOW_PRIVATE_EGRESS=1 if you deliberately serve
  records from private address space.

  Two limits stated rather than implied: a host that does not resolve is still
  allowed through, because it points nowhere and refusing it would block offline
  callers while closing nothing; and this does not defeat DNS rebinding, which
  would require pinning the checked address into the connection itself.

v0.45.1

v0.45.1

Security fix. Upgrade from 0.45.0 if you rely on licence verdicts.

Fixed

• A hostile licence URL could mint a permissive verdict. host_matches already
  rejected a domain sitting in someone else's path, but the token scanner
  underneath splits a URL at any character it cannot consume — so one URL became
  two tokens and each half was read as a standalone URL:

  http://creativecommons.org@evil.example.com/licenses/by/4.0/
  https://creativecommons.org:8080@evil.example.com/licenses/by/4.0/
  https://evil.example.com#creativecommons.org/licenses/by/4.0/

  Each returned a real CC-BY-4.0 and ALLOW for commercial use. The first two read
  the URL's userinfo as the host; the third matched its fragment. Open Data
  Commons URLs were affected identically.

  This is reachable input rather than a curiosity: licence strings are record
  data, and anyone can upload a record and set that field on Zenodo, HuggingFace
  or OpenML. The compatibility verdict, the access flag and the FAIR score were
  all derivable from attacker-controlled text.

  A token is now rejected when the surrounding URL syntax proves it is not a
  host — followed by "@" (optionally through a port) is userinfo, preceded by
  "@ # ? & =" means it sits inside another URL's userinfo, fragment or query.

• The same scanner was quadratic on attacker-supplied text. Its host-label group
  nested a star inside a plus, so a 12 KB licence field of "by-by-by-…" cost
  about 1.5 seconds of CPU, and the scan runs three times per licence check — a
  page of such records was minutes of wall clock. Every repetition is now bounded
  by DNS's own limits and the scan is length-capped. 12 KB now costs about 12 ms.

Both symptoms came from one regex, and both are fixed in one change.

v0.45.0

v0.45.0

Licence answers get more accurate in both directions: a source that grants
everything stops being reported as all-rights-reserved, and a source that states
something we cannot parse stops being reported as silent.

Fixed

• A stated licence is no longer reported as "not stated". A record whose licence
  could not be parsed returned the same verdict reason as a record with no
  licence at all, while license_raw sat in the same response holding the value
  the message denied existed. The verdict (REVIEW) was right in both cases; the
  explanation pointed callers away from the one lead they had.

  OpenML made the cost concrete — it states licence: 'Public' on every dataset,
  so the answer was "licence not stated" for a source that always states
  something. Unrecognized values now quote themselves, which also surfaces the
  two known drops from the 17-source sweep, other-open (Zenodo) and
  Springer TDM.

  No verdict changed and no licence is promoted: Public stays REVIEW, because
  "publicly available" is not "public domain" and mapping it to CC0 would invent
  a specific grant from a vague word.

Added

• cellxgene now returns ALLOW instead of REVIEW "all-rights-reserved". CZ
  CELLxGENE Discover publishes every dataset under CC-BY 4.0 as a condition of
  submission, but its curation API exposes no licence field anywhere — verified
  across all 386 published collections and their nested dataset objects — so the
  archive with the clearest blanket grant was among the most pessimistic answers.
  The default applies only where the record itself is silent; a record-stated
  licence still wins.

• The project is formally citable. CITATION.cff and .zenodo.json ship in the
  repository, so GitHub renders a "Cite this repository" entry and a Zenodo
  deposition can mint a DOI. CI fails the build when the citation version
  disagrees with pyproject.toml.

Changed

• dataone, omicsdi, omics and biostudies are documented as deliberately having no
  blanket licence, each with the operator's own wording as the reason. The first
  two federate other repositories; the latter two place no ADDITIONAL restrictions
  beyond the original data owner's, and NCBI states outright that it has no rights
  to transfer onward. "No additional restrictions" is not permission, so
  defaulting a licence there would invent a grant the operator declined to make.
  A test pins each absent.

v0.44.0

v0.44.0

Closes the licence-coverage arc: every SPDX id the normalizer can emit now carries
a compatibility profile.

Added

• Creative Commons 1.0/2.0/2.5/3.0 are assessed, not merely identified. The
  normalizer already recognized CC BY 3.0, but only the 4.0 family carried
  compatibility flags, so a licence that plainly permits commercial use returned
  REVIEW while the identical 4.0 licence returned ALLOW. All 24 pre-4.0 ids (6
  families x 4 versions) are now hand-encoded from the licence texts, and their
  verdicts are asserted to match their 4.0 counterparts on every intent rather
  than assumed to.

  The profiles are not copies of the 4.0 ones: pre-4.0 CC says nothing about
  patents, and its only trademark clause disclaims Creative Commons' own marks
  rather than the licensor's, so those limitations are omitted.

• The UK Open Government Licence v1.0 and v2.0 are assessed too. All three OGL
  versions grant the same shape — copy, publish, distribute, adapt, and exploit
  commercially, conditioned only on attribution — so they share one profile,
  verified against the legal text of each version. Prose and short-code forms now
  work for v1.0 and v2.0 as well. Bare OGL stays unrecognized: the verdict would
  be identical for any version, but spdx_id is the field callers cite, and
  naming a version the source never stated invents a fact.

• Every licence id the normalizer can emit now carries a compatibility profile,
  pinned by an invariant test. The failure it guards is silent — an alias or URL
  pattern added without a matching profile downgrades a plainly-stated licence to
  REVIEW, which reads as caution rather than as the gap it is.

Fixed

• OGL-UK-3.0 wrongly reported that the licence is silent on patents. Its profile
  omitted patent-use on that stated ground, but every OGL version's exemption
  list explicitly carves out "other intellectual property rights, including
  patents, trade marks, and design rights". patent-use is now asserted for all
  three versions. Verdicts are unaffected — they derive from permissions alone —
  but the reported limitations were misdescribing the licence.

v0.43.0

v0.43.0

Highlights:

• Source-level blanket licences, for sources that publish one. 10 of 17 sources
  state no licence at all, and answering "all rights reserved" for an archive
  whose operator has dedicated the whole thing is a wrong answer, not a safe
  one. pdb is CC0-1.0 (the wwPDB dedicates the archive; entries carry no licence
  field) and uniprot is CC-BY-4.0 (every UniProtKB flat-file record states it
  in-band, a notice the JSON our adapter reads drops). Applies only when the
  record itself is silent — a licence on the record always wins — and the
  verdict says where the licence came from. gwas is deliberately excluded:
  individual studies carry their own Usage License, so a blanket default would
  be wrong precisely where it matters.

• The README now documents the streamable HTTP transport: --transport http and
  every flag with it, the /mcp/ endpoint and the 307 that /mcp redirects with,
  the always-on DNS-rebinding rules, and that fetch(dest=...) writes to the
  server's filesystem rather than the caller's. Plus four environment variables
  the code read but nothing documented. The transport shipped in 0.42.0 but
  appeared nowhere someone installing from PyPI would look.

Fixes:

• NCBI pacing now follows the request host, not the service label. GEO's
  supplementary-file listing fetches from ftp.ncbi.nlm.nih.gov under the label
  "GEO suppl listing", which failed a startswith("NCBI") test, so it drew from
  the default bucket at 10 req/s — over three times NCBI's keyless ceiling —
  purely because of what the call was named. Honest scope note: this was found
  while triaging an intermittent GEO 403 and does not demonstrably fix it; that
  403 never reproduced in isolation. The pacing gap is a real defect on its own
  terms, but the causal link is unproven.

• A Creative Commons licence stated without a version is no longer discarded.
  CC 2.0/3.0/4.0 differ on attribution and on the effect of a DRM clause, so
  picking one would be a fabrication; identify_cc_family reports "family known,
  version not" as a third outcome instead of throwing the information away.

v0.42.0

v0.42.0

Highlights:

• Streamable HTTP transport (--transport http) alongside stdio; mcp pinned to

  =1.28.1,<2 for 3 high-severity CVEs.

• Four new sources: BioStudies (EBI/ArrayExpress), GBIF (biodiversity),
  data.gov (US federal open data), and NASA CMR (Earth science) — 13 -> 17.
• Search now reports unresolved ontology params instead of silently dropping
  them, with capability-gated elicitation.

Security:

• operate with op='sql' can no longer make the server fetch arbitrary URLs.
  httpfs stayed loaded after the local-filesystem lockdown, so a crafted SELECT
  made the server issue the request and return the body as rows — SSRF with
  response exfiltration under --transport http. Both filesystems are now
  disabled before the configuration is locked.

Fixes:

• bioproject: resolve was 100% broken (wrong NCBI search field).
• EuropePMC open-access full text was reported as paywalled (PMCID quoting).
• uniprot: resolve/fetch were unreachable (missing dispatch entry).
• OpenNeuro manifests, DANDI licences behind an spdx: prefix, and 4 advertised
  id_examples that did not resolve.

See CHANGELOG.md for the complete list.

v0.41.1 — version-sync fix

Patch release. v0.41.0 shipped with __init__.__version__ still at 0.40.0 (pyproject/server.json were bumped, the module attribute was not), so the module version and the version stamped into generated RO-Crate/dossier provenance disagreed with the distribution. All four version sources are now synced to 0.41.1. No functional changes beyond the version string; the 0.41.0 features (UniProtKB connector, search --json CLI) are unchanged.

v0.41.0 — UniProtKB connector + search --json CLI

Added

• UniProtKB connector (#15) — UniProtKB is now a first-class default source:
  full-text search reads the accurate x-total-results header (cursor-paginated,
  so like huggingface it contributes to page 1 only — offset>0 returns no rows),
  resolve attaches a FASTA FileEntry (unverified — no upstream checksum), and an
  injection-safe accession guard fails before any network call.
• search --json one-shot CLI subcommand (#16) — an explicit search first-arg
  diverts to a one-shot CLI that prints the SearchResult.results array as JSON,
  enabling lightweight non-MCP consumers (e.g. recap's DaProvider). Bare invocation
  still starts the MCP server unchanged.

Fixed

• Registry server.json description now fits the MCP registry's 100-char limit,
  with a guard test to keep it there.

Changed

• Dependency bumps: starlette 1.1.0→1.3.1 (#10), aiohttp 3.14.0→3.14.1 (#11),
  cryptography 48.0.0→48.0.1 (#12), pydantic-settings 2.14.1→2.14.2 (#13),
  python-multipart 0.0.29→0.0.31 (#9), actions/checkout 6.0.3→7.0.0 (#14).
• Re-recorded the stdio demo against the v0.40.0 tool surface.

v0.40.0

Full-project audit release: 20+ verified fixes across the core engine, connectors, and security surface, plus a truthful MCP tool surface.

Highlights

Truthful MCP surface — the search tool now advertises all 12 sources (DANDI, OpenML, RCSB PDB, GWAS Catalog, and CZ CELLxGENE were hidden from LLM clients); fetch lists every wired backend with its verification status; server.json documents the full env-var surface (LLM_*, EMBEDDING_*, UNPAYWALL_EMAIL) so deployment tooling can surface the knobs behind understand=, multi_query=, and rank=semantic.

Security hardening — archive extraction now streams in chunks counting actual bytes (no whole-member RAM loads, no header-trust), extraction shares the fetch byte budget (closing a 2× disk-write hole), and fetch/operate enforce an http(s) URL scheme allowlist against poisoned upstream metadata. The registry publish workflow SHA-pins its checkout and sha256-verifies the mcp-publisher binary.

Correctness — NCBI_EMAIL alone no longer over-claims 10 req/s (inviting 429s); search fan-out survives sub-task cancellation with partial results instead of crashing; cursors are type-validated; collapse_mirrors finds transitive merges; DataONE Solr queries escape user input; GWAS pagination math fixed for size>50; OSF listing page-capped; DataCite/Scholix error-taxonomy escapes closed; OpenNeuro queries use GraphQL variables.

README — full 12-source roster in the intro and sources table, new architecture diagram, PyPI-safe absolute links.

See CHANGELOG.md for the complete list.

🤖 Generated with Claude Code