Skip to content

Rete 0.3.7

Choose a tag to compare

@github-actions github-actions released this 28 Oct 11:50
· 46 commits to main since this release
2845fa2

Where 0.3.7 is published: every channel, from the one v0.3.7 tag.
crates.io (rete-core, rete-graph, rete-cli, rete-wasm), PyPI and npm
(rete-graph), the GitHub release with its native archives, browser bundle,
Blender add-on, the Claude Desktop extension (rete-0.3.7.mcpb, and
rete.mcpb behind the releases/latest/download/rete.mcpb link) and the
Pyodide wheels, the container images on GHCR (rete-cli, rete-relay,
rete-dev), and the desktop File Explorer installers. It is the first release
cut the new way: no py-v* / js-v* tags, no follow-up PR, no manual upload.

What 0.3.7 changes on top of 0.3.6:

  • Behaviour change: SPARQL comparisons go by datatype, not by what a string
    looks like.
    =, !=, IN, <…>=, ORDER BY, MIN/MAX and the
    arithmetic follow SPARQL 1.1 §17.3 and §15.1: "01" = "1" is false,
    FILTER(?year > 2000) over string years returns nothing (cast it:
    xsd:integer(?year) > 2000), and ORDER BY is a total order. The query
    warnings name the new cases. Details under Fixed.
  • The Claude Desktop extension no longer times out on long queries. Every
    query runs as a job on a worker thread: a call answers within 40 s with the
    rows or a job_id, and query_status, query_result (every row, by cursor
    or saved to a CSV/TSV/JSON file) and cancel_query follow it. With
    partition_by a long extraction runs as chunks checkpointed on disk, and
    the same call resumes after a cancel, a crash or a restart; keys_job runs
    it for the entities an earlier job listed — the works citing a paper,
    listed in crossref-citations, then their details from crossref —
    without copying them into the query. The extension also queries RDF shared
    in the chat (load_rdf), and its instructions now say what a query costs
    and when and how to split one.
  • Node reads batched ranges in parallel (npm rete-graph, and the
    extension that bundles it): listing the 4,758 works citing one paper on
    the 60 GB Crossref graph took 3.5–4.9 s instead of 23–35 s.
  • Split a query by one variable, from the engine:
    rete_core::partition_plan / eval_query_partitioned, and in JS
    Graph.partitionPlan() and queryRaw(q, { partition }). The keys come
    from a VALUES list on the variable, else its cheapest constant pattern.
    SparqlError gains a Partition variant.
  • --query-file <path> (or - for standard input) on every CLI command
    that takes a query, for queries past the OS argument limit.
  • Security: zstd 0.14.3 / zstd-safe 8.2.0 / zstd-sys 2.1.2
    (RUSTSEC-2026-0391, -0395 to -0406, -0410), PyO3 0.30.1 in the Python
    client (RUSTSEC-2025-0020, RUSTSEC-2026-0177), tokio 1.54.1 in the File
    Explorer (RUSTSEC-2026-0386), and in the extension the MCP SDK 1.32.1,
    body-parser 2.4.0 and zod 4.6.7. Every lockfile is now audited in CI
    and nightly against one exception list.

Added

  • The Claude Desktop extension runs queries as jobs, so a long one no
    longer times out or blocks everything else.
    Claude gives up on a tool
    call after 60 s, and the engine's range reads are synchronous: a query on
    the extension's own thread froze it until the query ended, a client that
    had given up could not cancel it, and every later call — even a trivial
    one — queued behind it and timed out too. Asking for title, venue and year
    of the ~4,800 works citing one paper on the 60 GB Crossref graph did
    exactly this. Every query now runs on a worker thread (up to three at
    once). sparql_query waits up to 40 s and otherwise answers with a
    job_id while the query keeps running; new tools start_query (do not
    wait), query_status (wait up to 45 s; reads, bytes, elapsed; the first
    rows once done), query_result and cancel_query. The 200-row answer
    page now carries next_cursor: query_result pages through every row
    (up to 1000 a page) or writes all of them to a .csv / .tsv / .json /
    .jsonl / .nt file in a granted folder, never over an existing file —
    agents had been folding rows into one GROUP_CONCAT string to get past
    the cap. A client that cancels a sparql_query call stops its query.

  • The extension tells the model what a query costs. Its instructions now
    explain that a published graph costs one network round trip per separate
    place a query touches — so per-entity details for thousands of scattered
    entities are expensive while a COUNT is cheap — how to follow a job, and
    how to split a large extraction into a cheap key list plus VALUES
    batches; they also ask the model to check in dataset_schema that the
    relation it needs is in the graph it is querying. list_datasets returns
    whole descriptions when a filter narrows it to three graphs or fewer: the
    caveat that matters most for a large graph ("keep queries selective") sat
    past the 320-character cut.

  • The Claude Desktop extension queries RDF shared in the chat. Claude
    Desktop does not hand chat attachments to an extension, so a .ttl the
    user attached could not be queried at all — and the path the model sees for
    an upload (/mnt/user-data/uploads/…) is in Claude's own sandbox, not on
    the user's machine. The new load_rdf tool takes the file's text (Turtle,
    TriG, N-Triples, N-Quads; syntax from the file's extension), builds it into
    a graph held in memory, and every tool then takes it as dataset
    chat:<name> — with no folder granted. A file too long for one call goes
    in parts (append; a part that does not parse is refused and the graph
    keeps the parts before it); path reads an RDF file the user saved in a
    granted folder instead; save_as keeps the graph as a .rete there. The
    instructions tell the model to convert CSV / JSON / RDF/XML / JSON-LD to
    Turtle first, and to ask for a granted folder or a URL for a .rete, which
    is binary and cannot travel through the chat.

  • A long query can run as resumable chunks, split by one variable. A job
    that ran for many minutes was lost when Claude Desktop restarted or the
    query was cancelled, and an agent splitting the work by hand into VALUES
    batches had to keep track of the batches itself. The engine now plans the
    split: rete_core::partition_plan(rete, query, var) refuses one that would
    change the answer, saying why — ASK or DESCRIBE, ORDER BY, an aggregate not
    grouped by the variable, a variable some solution may leave unbound — and
    writes the keys query from the one constant pattern on the variable with
    the smallest tiles (read off the tile directory). eval_query_partitioned
    runs one chunk, its keys seeded as VALUES where the pushdown turns each
    into an index lookup. Both are on the wasm graphs (partition_plan,
    query_partition) and in the JS client (Graph.partitionPlan(),
    queryRaw(q, { partition: { var, keys } })). The Claude Desktop extension
    drives them with partition_by on sparql_query / start_query: keys
    first, then chunks two at a time, each sized to take about 15 s, every
    finished chunk written to disk. Sending the same call again after a cancel,
    a crash or a restart carries on from the last finished chunk (a checkpoint
    made on other bytes is discarded); an identical call joins a running one.
    The instructions now say when to split and how to choose the variable.
    SparqlError gains a Partition variant for the refusals.

  • A split takes its keys from a VALUES list, or from an earlier job.
    The citations of the Crossref graph moved to crossref-citations.rete, so
    "title and year of every work citing a paper" lists the works in one graph
    and asks the other for their details. With the works in a VALUES ?work
    list, the plan now lists the keys from that list as it stands (keys_from: "values", no read) instead of evaluating the whole query once, and each
    chunk cuts the list down to its own keys. In the Claude Desktop extension,
    keys_job: "<job_id>" beside partition_by takes the keys from an earlier
    job's rows — the listing on the other graph — so the model never copies
    thousands of IRIs into a query. The plan's keys_from_where flag becomes
    keys_from ("values", "pattern" or "where").

  • mcpb-test.yml: the extension's stdio suites run on pull requests.
    They ran only in the release workflow, so a change to clients/mcpb
    reached a tag unchecked. The new workflow builds the engine, the JS client
    and the bundle, runs the suites on Node 22 and on the Node 18 floor, and
    validates the manifest.

  • --query-file <path> on every CLI command that takes a query, with
    - reading standard input: sparql, sparql-url, federate (in place of
    --query), cost, progressive, cypher and manifest query. The query
    used to be a command-line argument only, which the OS caps (128 KiB per
    argument on Linux, a 32,767-character command line on Windows), so a query
    with a large VALUES block failed before rete ran (OpenAIRE QA, E5). The
    file and the argument are mutually exclusive, and one of them is required:
    both or neither is a usage error (exit 2). The file must be UTF-8 (a
    leading byte-order mark is skipped); an unreadable or blank one is an error
    (exit 1). See docs/cli.md.

Security

  • The 2026-10 zstd, tokio, zod and body-parser advisories are cleared.
    zstd is 0.14.3 (was 0.13.3; the workspace requirement moves from 0.13 to
    0.14, no source change), with zstd-safe 8.2.0 (was 7.2.4) and zstd-sys
    2.1.2 (was 2.0.16), fixing RUSTSEC-2026-0391, RUSTSEC-2026-0395 to
    RUSTSEC-2026-0406 and RUSTSEC-2026-0410 in the workspace, Python, R, desktop
    explorer and fuzz lockfiles. tokio is 1.54.1 (was 1.53.1) in the desktop
    explorer, fixing RUSTSEC-2026-0386. The Claude Desktop extension resolves
    body-parser 2.4.0 (was 2.3.0, GHSA-27r3-jxmh-p828) and zod 4.6.7 (was
    4.4.3, GHSA-mrhx-c7xc-m3w5). No exceptions were added.
  • PyO3 0.30.1 in the Python client (was 0.23.5), fixing RUSTSEC-2025-0020
    (possible buffer overflow in PyString::from_object) and RUSTSEC-2026-0177
    (missing Sync bound on PyCFunction::new_closure closures) in the PyPI
    and Pyodide wheels. The Python API is unchanged: same functions, same
    signatures, same 61 tests, abi3 wheels still cover CPython ≥ 3.9, and the
    module keeps declaring that it uses the GIL. Both Pyodide wheels (0.29 and
    314) build and answer queries.
  • The Claude Desktop extension's MCP SDK is 1.32.1 (was 1.29.0), with
    qs, proxy-addr, fast-uri, ip-address, hono and
    @hono/node-server moved to patched releases: 32 npm advisories in the
    extension's runtime dependency set, one critical (proxy-addr IP spoofing).
    The bundled server actually carried the SDK and fast-uri (through ajv);
    the rest belong to HTTP transports the stdio server does not bundle. The
    update is a lockfile refresh within the existing version ranges; the
    extension's stdio smoke suite passes and the bundle still runs on Node 18.
  • fuzz/Cargo.lock refreshed. It still recorded rete-core 0.3.2 with
    ruzstd 0.7.3 (RUSTSEC-2026-0344, decompression bomb); cargo re-resolved it
    on every fuzz run, so the harness built the current engine, but the
    committed file described a dependency set nobody built.
  • Every lockfile is audited, in CI and nightly. cargo audit used to see
    the workspace lockfile only. scripts/audit_lockfiles.py now audits the
    workspace, Python, R, Java (resolved fresh), File Explorer and fuzz
    lockfiles and, with npm audit --omit=dev, the npm packages that ship,
    against a single exception list, lockfile-audit.toml: each exception names
    one advisory and the lockfiles it applies to, with a reason and a review date
    at most a quarter away. Unlisted lockfiles, stale or expired exceptions and
    lockfiles out of step with their manifests fail the run. CI's security job
    runs it on lockfile and manifest changes and on the nightly schedule, and the
    crates.io publish gate (scripts/publish_crates.sh) runs the same script on
    the workspace lockfile instead of its own copy of the ignore list. The
    accepted exceptions: the two quick-xml findings (blocked on Oxigraph, as
    before), paste (unmaintained) in the R client, and the File Explorer's
    Linux GTK backend crates and unic-* tables (informational; the app ships
    for macOS and Windows only). See docs/release.md#dependency-audit.

Changed

  • Remote and local queries in Node fetch batched reads in parallel. The
    engine asks for some reads as a batch — the dictionary terms a result
    decodes, the tiles a join is about to probe — and offers a hook,
    globalThis.reteReadMany, to fetch such a batch concurrently. The
    playground has always installed one; the Node client did not, so under
    Node every range of a batch was its own blocking round trip. Its sync-fetch
    bridge now installs the hook: up to 16 ranges in flight, one wait for the
    whole batch, and the engine plans for that fan-out. On crossref.rete
    (60 GB) over a residential link, listing the 4,758 works citing one paper
    took 3.5–4.9 s instead of 23–35 s, and title/venue/year for 100 of them
    19–25 s instead of 46–65 s (two cold runs), with identical rows. This
    reaches the npm package and the Claude
    Desktop extension, which bundles it. A batch with any range that is not a
    206 at the requested offset comes back null and the engine re-reads it
    sequentially, as before. The hook is installed only with the bridge's own
    XHR (never over a caller-supplied one), and
    globalThis.reteReadMany = null before the first open() opts out.
  • One vX.Y.Z tag releases every channel. The same tag that builds the
    GitHub release and publishes the crates now also publishes rete-graph to
    PyPI (python-client-publish.yml) and npm (js-client-publish.yml) and the
    container images; the separate py-v* and js-v* tags are retired (the old
    ones stay as history). The release no longer has a tag order: the relay
    image and the Blender add-on used to pip install rete-graph from PyPI while
    they were built, so v* had to wait for the py-v* upload and for a second
    PR raising the PyPI floors. Both now use wheels built in their own run from
    the tagged commit, by a new reusable python-wheels.yml that also builds the
    PyPI wheels, and their builds fail unless the engine they carry is the tag's
    version. relay-test.yml and the Docker PR check build the relay the same
    way, on the PR's own wheel. A release is now one PR
    (scripts/sync_versions.py --set X.Y.Z stamps the floors and the Binder pin
    too; --set-published is gone), its merge, and the tag. Every workflow the
    tag starts first runs sync_versions.py --check-tag, so a tag that does not
    match every channel's version publishes nowhere, and --check now requires
    every client to carry the engine version exactly rather than its
    MAJOR.MINOR. The pypi and npm GitHub environments admit v* tags only,
    like crates-io. The trusted-publisher entries on PyPI and npm are
    unchanged: they name the workflow files and environments, which keep their
    names. See docs/release.md.
  • The Pyodide 0.29 wheel ships with the GitHub release; a release has no
    manual step left.
    python-client-publish.yml attaches the two
    pyemscripten_* wheels and the copy retagged pyodide_2025_0_wasm32 for
    Pyodide 0.29 (whose installer predates PEP 783 and cannot use the PyPI
    wheels) to the tag's GitHub release, waiting for release.yml's draft the way
    the File Explorer builders do. scripts/publish_pyodide_wheel.sh, which
    uploaded the retagged copy to data.graphplaza.com/wheels/ by hand after
    each release, is gone, and so is the versioned URL sync_versions.py
    stamped into docs/python.md. The bucket copies stay, frozen at 0.3.6.
    GitHub serves release assets without CORS headers, so a notebook cannot
    install one by URL: docs/python.md now says to download the wheel, upload
    it into JupyterLite and %pip install emfs:./<file>.whl. The new assets are
    not in SHA256SUMS or the attestations (docs/release.md).
  • crates.io publishing needs no manual approval. The crates-io GitHub
    environment no longer has a required reviewer: a v* tag publishes the
    crates as soon as the release is assembled. It still admits v* tags only,
    authenticates through trusted publishing, and stops on the publish-time
    cargo audit / cargo deny gate (docs/release.md).
  • npm publishing is tag-only and publishes the tarball it tested.
    js-client-publish.yml now has a build job (tag/version check, wasm build,
    tests, npm pack, npm publish --dry-run on tags) and a publish job that
    uploads that tarball with --provenance, only for a pushed release tag; a
    manual workflow_dispatch run used to publish from whatever branch it was
    started on and is now a dry run. npm is pinned to the 12.x major with a
    check for the 11.5.1 trusted-publishing floor. Authentication is unchanged:
    rete-graph has been published through npm trusted publishing (GitHub
    OIDC, no token) since 0.2.0; the "npm tokens that bypass 2FA are being
    restricted" notice in the 0.3.6 log is npm's announcement to all publishers
    and does not apply to that path. docs/release.md now lists how each
    registry authenticates.

Fixed

  • SPARQL: comparisons go by datatype, not by what a string looks like.
    This changes query results.
    rete compared any two terms numerically when
    both texts parsed as numbers and by their token text otherwise. So the
    plain strings "01" = "1", "1.0" = "1", "1e3" = "1000" and
    "inf" = "Infinity" were true, "NaN" = "NaN" was false ("NaN" read as
    a float NaN), ?year > 2000 compared a string year with a number, and
    ORDER BY put literals before IRIs and a string after the longer strings
    it prefixes ("Retired and archived" before "Retired"). Found by the
    OpenAIRE output QA (e_nan_cmp.rq). Now, as SPARQL 1.1 §17.3 maps each
    operator by the operands' datatypes:

    • = / != / IN: numbers by value with type promotion
      ("1"^^xsd:integer = "1.0"^^xsd:decimal, "01"^^xsd:integer = 1;
      integers and decimals exactly, a NaN equal to nothing); simple literals
      and xsd:string by code point; booleans, the XSD date/time types and
      durations by value; language-tagged literals by text and tag, tags
      case-insensitively ("a"@EN = "a"@en). A string never equals a number.
      Two different literals of a datatype rete does not know, or ill-typed
      ones ("abc"^^xsd:integer), are a type error (RDFterm-equal), as is a
      zoned and an unzoned date/time less than 14 hours apart.
    • < > <= >=: defined for the same pairs; anything else (a string
      and a number, two IRIs) is a type error, which drops the row in a
      FILTER. A NaN compares false.
    • ORDER BY, MIN, MAX: unbound, blank nodes, IRIs, literals, quoted
      triples (§15.1); among literals text first (strings, tagged and unknown
      literals together, by text), then numbers by value, booleans, date/time
      values and durations. It is now a total order.
    • SUM, AVG, + - * /, ABS / CEIL / FLOOR / ROUND and the
      SUBSTR positions take numeric-typed literals only: a string is a type
      error whatever its text, and the derived types (xsd:int, …) now count.

    Check queries that compare strings with numbers. FILTER(?year > 2000)
    over string years now returns nothing; cast (xsd:integer(?year) > 2000)
    or compare text on both sides. The query warnings say so:
    incomparable (a type error: the operator is not defined for the pair) and
    a string-vs-number hint for an = that is false only because a string's
    text spells the number ("2020" = 2020). The Claude Desktop extension
    words the new kind. sameTerm, joins, DISTINCT and GROUP BY are
    unchanged: they compare terms, not values.

    W3C SPARQL 1.1: 239 / 312, unchanged. SPARQL 1.0 suite (not gated, CI
    harness): 111 → 121, newly passing open-world date-2, date-3,
    open-cmp-01/02, open-eq-04/06/08/10/11/12; none newly failing. A new
    Oxigraph differential (comparison_matrix_agrees_with_oxigraph, 30
    queries, every value compared whole) agrees; where Oxigraph departs from
    the spec rete follows the spec (NaN < 1 false, false < true true,
    <a> <= <a> an error, the zoned/unzoned date pair an error), listed in
    docs/sparql.md#compare.

    Measured on the same files with the old and the new binary: the DataCite
    QA battery, 64 of 64 answers unchanged. The OpenAIRE output QA, 192
    queries: four answers change — the repro, two self-joins that counted 7
    agents named "Nan" as different from themselves, and one dataset titled
    "nan" — and a_years changes order on three shards, because years of
    fewer than four digits are not valid xsd:gYears and now sort as text,
    before the valid years. Published queries (276 playground examples and
    181 dataset-card starter queries, run on the live files): one example
    broke and is fixed — the bcul "Treasures" example filtered
    FILTER(?date > 1450) on xsd:gYear dates and now compares
    "1450"^^xsd:gYear, with the same 50 rows; 3 examples' ORDER BY … LIMIT window and 9 examples' order move because strings sort by their
    text rather than their N-Triples token (a string before its own
    extensions, an embedded " by its code point); 11 temporal-extent card
    starters (MIN/MAX) and one bounding box change and 15 histogram
    starters change order, all over values that are not of one comparable
    type — strings that hold numbers, ill-typed years, IRIs mixed with numbers
    (IRIs now sort first, §15.1); and three histogram starters (vidy,
    bne, wikidata-xxl shard 3) that panicked with "comparison function
    does not correctly implement a total order" now answer. 409 of the 457
    answer exactly as before; 5 could not be compared within a 900 s / 8 GB
    limit on either binary (gharchive, deps-dev, the federated
    openaire-2021; the two openaire-2021 ones answer the same on the
    shard that holds their data). Cards are written at build time, so a
    published card changes only when its file is rebuilt.


Install, artifact list, download verification and known limitations: release guide. Full history: CHANGELOG.md.