v0.5.1

What's Changed

No breaking changes, and no query returns different rows.

Planner:

• #97: MATCH (a)-[r1]->(b)-[r2]->(c) [WHERE ...] RETURN count(*) is counted from per-node degrees instead of building every path, when each WHERE condition reads only one hop's variables and the hops fan out enough to make it worthwhile. On graph-benchmark (100K Person, 2.4M FOLLOWS), q8 drops from 120 to 20 ms and q9 from 92 to 19 ms. The counts match path expansion exactly, including self-loops.
• #101: grouping by a node, as in WITH forum, collect(friend), groups by the node's id when its label has an in-memory node index, and looks the node's other properties up afterwards, once per node, instead of carrying them through the aggregate for every row. In memory mode, IC3 drops from 53.4 to 25.4 ms (-52%), IC7 from 16.2 to 14.4 ms (-11%) and IC5 from 85.1 to 78.9 ms. explain() shows the lookup as NodeLookupExec. Lance node tables read lazily keep the previous plan.

CSR:

• #98: optimize_csr() (Python and Rust) folds a CSR's accumulated delta into a new base at the pinned version, the counterpart of Lance's optimize_indices(). refresh() still applies changes as a delta and rebuilds on its own past 25% of the base. The CSR guide explains when to use which.
• #100: filters on an in-memory relationship's columns (WHERE membership.joinDate > $minDate) run per chunk in every partition, instead of one partition building a mask while the others wait. IC5 in memory mode drops from 97.1 to 89.1 ms on its own.

IC5, slower in memory mode since 0.5.0's #87 (#92), is back to 0.4.0's time: 78.9 ms against 0.4.0's 79.8 ms (0.5.0: 97.1 ms). In lazy mode it's about 110 ms, against 104.0 ms in 0.4.0 and 116.4 ms in 0.5.0.

Known trade-offs: none new. Two follow-ups are open: #99 (looking up grouped nodes' properties for Lance node tables) and #102 (large named hops that return relationship columns from Lance, an experiment parked there).

Other: RELEASE.md documents the release process, and CI skips the test workflows on main pushes that don't touch code.

Benchmarks: measured PR by PR against main with the harness in CONTRIBUTING.md, both suites, both modes, on an Apple M5; every query's answer matched. Each PR lists every query.

PRs

• perf(planner): count two-hop paths from per-node degrees by @prrao87 in #97
• feat(csr): fold the delta into a new base with optimize_csr() by @prrao87 in #98
• perf(csr): filter in-memory relationship columns per chunk by @prrao87 in #100
• perf(planner): look up a grouped node's columns after grouping by id by @prrao87 in #101
• chore: bump version 0.5.0 → 0.5.1 by @prrao87 in #103

Full Changelog: v0.5.0...v0.5.1

v0.5.0

What's Changed

Breaking:

• #88 (Python and Rust): the deprecated GraphConfigBuilder.with_node_label and with_relationship are removed, and in Rust with_relationship_endpoints too. Use node_table(label, primary_key) and rel_table(rel_type, from_table, to_table, src, dst).
• #87 (Rust): CsrIndexBuilder::add_edge, Adjacency::build and with_changes, EdgeChanges and CsrExpandNode::try_new changed signatures. The Rust API is marked experimental, so expect more changes like this. The Python API is unaffected.

CSR:

• #87: a hop that names its relationship, such as (a)-[k:KNOWS]->(b), stays on the CSR and fetches only the columns of k the query reads. Filters on those columns run inside the expansion. count(r) on a relationship now works. Refresh no longer needs the old dataset version, so compaction followed by cleanup_old_versions() applies as a delta instead of a rebuild. Building the CSR takes about 95 bytes per edge at peak and is about 1.35 times slower.
• #94: a hop that fetches a large share of a Lance node table reads the target columns once instead of once per batch. In lazy mode, IC3 and IC9 are about 60% faster than in 0.4.0 (403 → 166 ms, 796 → 310 ms).
• #93: csr_stats() reports base_version (the version of the last full build) and unindexed_edges (live edges outside the base CSR).

Known trade-off: IC5 is about 20% slower in memory mode than in 0.4.0 (79.8 → 95.7 ms), because its named hasMember hop now runs on the CSR. In lazy mode it's within 7% of 0.4.0 (104.0 → 111.1 ms). #92 tracks the fix.

Other: #85 (parallel Python tests), #90 (leaner Rust CI caches), #89 and #91 (Python and Rust tabs in the docs).

Benchmarks: both suites, both modes, against the 0.4.0 wheel from PyPI, on an Apple M5. Every query's answer matched.

PRs

• test(python): run the test suite in parallel with pytest-xdist by @prrao87 in #85
• refactor(config)!: remove deprecated with_node_label and with_relationship by @prrao87 in #88
• docs: add python and rust tabs to the home page example by @prrao87 in #89
• docs: rust tabs for getting started, keeping the csr fresh, sidebar order by @prrao87 in #91
• ci: keep the rust caches lean and stop rebuilding for the example by @prrao87 in #90
• feat(csr)!: keep named relationships on the csr with edge rows by @prrao87 in #87
• feat(csr): report how far the base csr trails the data by @prrao87 in #93
• perf(csr): read target columns once when a hop fetches much of a table by @prrao87 in #94
• chore: bump version 0.4.0 → 0.5.0 by @prrao87 in #95

Full Changelog: v0.4.0...v0.5.0

v0.4.0

What's Changed

Breaking (Rust):

• #70: GraphConfig's node and relationship mappings are private. Code that wrote to them directly must use GraphConfigBuilder. A deserialized config is now validated like a built one, so a saved config with an unnormalized key, or a relationship whose endpoint label has no node mapping, fails to load. The Python API is unaffected.

Query results that change:

• #82: within one MATCH, two single-hop relationships of the same type no longer bind the same edge, so some queries return fewer rows (openCypher relationship uniqueness).
• #83: a relationship variable written again, as in (a)-[r:KNOWS]->(b), (c)-[r]->(d), matches that same relationship. It used to return wrong rows or fail to plan. Reusing one in a variable-length segment now raises an error within one MATCH.

Python:

• #71: the public classes are typed, documented Python classes wrapping the compiled ones. Names, signatures and behaviour are unchanged. ExecutionStrategy, SqlDialect and DistanceMetric are IntEnums with the same members and values, and passing the wrong class raises TypeError naming the parameter. The API reference is generated from the docstrings.

Fixes: #65 (distinct counts on empty input), #66 (physical column casing in graph scans), #67 (one trailing semicolon accepted).

Docs: #63, #64, #68, #69.

PRs

• docs(rust): show with_execution_deadline in the API docs by @prrao87 in #63
• docs: move documentation links to lance-csr.hdclabs.ai by @prrao87 in #64
• fix(parser): accept one trailing Cypher semicolon by @stugorf in #67
• fix(planner): preserve physical column casing in graph scans by @stugorf in #66
• fix(query): preserve empty-input distinct counts by @stugorf in #65
• docs: sync benchmark tables for lance-csr 0.3.1 by @prrao87 in #69
• docs: explain contribution rationale and benchmark harness by @prrao87 in #68
• refactor(config)!: make graph config mappings private by @prrao87 in #70
• refactor(python): wrap the compiled classes in typed, documented python classes by @prrao87 in #71
• fix(planner): bind each relationship once per match by @prrao87 in #82
• fix(planner): match a reused relationship variable as the same edge by @prrao87 in #83
• chore: bump version to 0.4.0 by @prrao87 in #84

Full Changelog: v0.3.1...v0.4.0

v0.3.1

lance-csr 0.3.1 is the first PyPI release of the 0.3 line. It contains everything in 0.3.0, plus two Cypher fixes. 0.3.0's PyPI publish stopped at a flaky test before uploading anything, so 0.3.0 exists only on crates.io.

See the 0.3.0 notes for this line's main changes:

• the Rust CypherEngine on crates.io;
• UNION, CALL subqueries, named paths, reduce and the new scalar functions;
• query timeouts and type stubs;
• about 22–25% faster GDC Interactive complex reads.

Fixes

• ORDER BY places nulls as openCypher does: last in ascending order and first in descending order. Previously nulls always came first. (#60)
• count(v) works for paths and plain values: count(DISTINCT p) on named paths, and count(x) for UNWIND and WITH values, now return counts instead of failing to plan. Parallel edges count as distinct paths. (#61)
• A timed-stream test no longer depends on partition order. It had failed intermittently on release builds. The engine's behaviour is unchanged.

Behaviour change

Ascending sorts over columns with nulls return the nulls last. To keep nulls first, sort on a flag computed in WITH, for example MATCH (p:Person) WITH p.name AS name, p.name IS NULL AS missing RETURN name ORDER BY missing DESC, name.

Full changelog: v0.3.0...v0.3.1

v0.3.0

Note: 0.3.0 was published to crates.io but not to PyPI; its PyPI publish stopped at a flaky test before uploading. Python users should install 0.3.1, which includes everything below plus two fixes.

lance-csr 0.3.0 makes the engine a Rust library, broadens the supported Cypher considerably, and speeds up the GDC Interactive complex reads.

Highlights

• Use it from Rust. The engine is now on crates.io as lance-csr. lance_csr::CypherEngine is the same CSR-backed engine as the Python class, with an async API, refresh(), statistics and cooperative query deadlines. See Using from Rust.
• Much more Cypher:
  • UNION / UNION ALL, read-only CALL { ... } subqueries, and named path values with relationships(), startNode() and endNode().
  • keys(), labels(), reduce, and the any / all / none / single predicates.
  • Numeric functions (abs, round, floor, ceil, sqrt) and string and list functions (left, right, substring, replace, split, trim, reverse, last, tail, size, range with a step).
  • List and map parameters, parameters in SKIP / LIMIT, and integer lists that keep their precision.
• Query timeouts. execute, execute_batches and CypherQuery.execute take a keyword-only timeout= in seconds. Rust callers use lance_csr::query::with_execution_deadline.
• Faster traversal. The CSR expands high-degree nodes in bounded chunks, traverses variable-length paths iteratively (including zero-hop paths), reuses filter columns, and searches shortest paths from both ends. Repeated queries reuse per-snapshot plan templates.
• Type stubs. The package ships .pyi stubs and a py.typed marker.
• target_partitions on CypherEngine (1 to 1024) sets DataFusion's query partitioning.

Performance

Compared with 0.2.0 on the same machine (release builds, both benchmark suites, all result checks passing):

• GDC Interactive complex reads (IC1–IC14): about 22–25% faster overall. IC12 is about 7× faster and IC13 about 4×.
• 30-query workload: level overall, but Q30 is about 10–14% slower. IC6 is also about 14% slower in memory. Both are tracked in #56.
• Engine construction, warm queries and refresh: unchanged.

The published benchmark tables will be refreshed with 0.3.0 in the benchmark repositories.

Behaviour changes

• The Rust crates are renamed to lance-csr and lance-csr-catalog (use lance_csr::...). The Python package name and import are unchanged.
• Concurrent refresh() calls run one at a time, so an older snapshot can no longer replace a newer one (#12).
• Mixing UNION and UNION ALL in one query raises ValueError, as openCypher requires.
• Type errors in comparisons and arithmetic name the expression and operand types. String-number comparisons such as p.age = '30' still run, as before.
• Named path records use the metadata fields __lance_csr_properties, __lance_csr_identity, __lance_csr_start and __lance_csr_end.

Known issues

#56 (two slower queries), #58, #59, #60 and #61 (Cypher behaviours that differ from openCypher, with reproductions).

Thanks

Thanks to David Hughes (@stugorf) for the Cypher, CSR and execution work in this release.

Full changelog: v0.2.0...v0.3.0

v0.2.0

lance-csr now runs the official LDBC Interactive complex (IC) queries as openCypher text instead of hand-written ports. Its config builder uses Kuzu-style node and relationship tables, and the project's tooling moves to uv.

Highlights

• The official LDBC IC Cypher runs as written. 13 of the 14 queries run from the official text (as adapted to the SF1 schema) with no changes. IC14's text uses reduce(), which openCypher's standardisation scope excludes, so it runs as a port built from in-scope constructs. All 14 match the golden answers in every engine mode.
• New config builder: node_table and rel_table. The names match Kuzu's and Ladybug's node and rel tables, and every relationship table names the node tables at both ends.
• Faster planning for queries with several subqueries. EXISTS checks, pattern comprehensions and OPTIONAL MATCH no longer copy the query's input exponentially. LDBC IC14 drops from 641 to 225 physical operators and from 188 to 62 ms.

Upgrading

uv add 'lance-csr>=0.2.0'

Rename the builder calls. Each relationship now names its endpoint node tables:

# 0.1
config = (
    GraphConfig.builder()
    .with_node_label("Person", "id")
    .with_node_label("Company", "id")
    .with_relationship("WORKS_AT", "src", "dst")
    .build()
)

# 0.2
config = (
    GraphConfig.builder()
    .node_table("Person", primary_key="id")
    .node_table("Company", primary_key="id")
    .rel_table("WORKS_AT", from_table="Person", to_table="Company", src="src", dst="dst")
    .build()
)

with_node_label, with_relationship and with_relationship_endpoints still work in 0.2 with unchanged behavior. They raise a DeprecationWarning that names the replacement, and will be removed in a later release. In Rust they are marked #[deprecated].

Behavior changes

• collect() over no values returns [] instead of null, as Cypher specifies.
• Unaliased variables in RETURN come out under lowercase names (RETURN postCount gives a postcount column), consistent with aliases, which were already lowercased.
• A list literal that mixes types, such as [uni.name, studyAt.classYear], becomes a list of strings. Before, it failed to cast.
• The tests and dev extras are gone from the package: development tools now live in a uv dependency group (see below).

New Cypher support

• Parameters: engine.execute(query, params), and the same for execute_batches and explain, with numbers, strings, booleans, None, and datetime/date values.
• Nodes as values: a = b, a IN [x, y], count(DISTINCT a) and a IS NULL compare nodes by id, and a path used as a value is its length (path IS NULL).
• Lists of nodes: collect(node), then UNWIND back to nodes, and x IN list for any list expression (friend IN friends, tag.id IN tags).
• Chained comparisons: $start <= m.creationDate < $end.
• Inline properties that read a variable: (t:Tag {id: tagId}).
• Label inference: an unlabeled node takes the label the relationship's endpoint tables decide, as in (p:Person)<-[:HAS_CREATOR]-(post).
• Predicates as values: NOT (a)-[:KNOWS]-(b) AS isNew, and CASE path IS NULL WHEN true ....
• Maps and fields: map literals (a node inside a map keeps its properties), and .field on maps and dates (latestLike.msg.id, birthday.month).
• Functions: head, toInteger, toFloat, date, datetime and duration.inSeconds.
• Aggregation: aggregates inside expressions (head(collect(...)), count(x) + 1), and collect() in the order of a preceding WITH ... ORDER BY.
• Comprehensions: list comprehensions [x IN list WHERE p | v], pattern predicates on the element included, and pattern comprehensions [(a)-[:R]->(b) WHERE p | v].

Fixes

• Semantic analysis now ends variable scope at WITH, as the planner already did. Before, WITH collect(friend) AS friends UNWIND friends AS friend failed with "redefined with different type".
• A pattern after WITH may start at a node no earlier clause bound. It used to fail to plan.
• Variable-length paths kept only their end nodes' columns, dropping scalars and other nodes carried from earlier clauses.
• An OPTIONAL MATCH whose WHERE reads a list carried by WITH (WHERE friend IN friends) now plans.

Performance

On the LDBC SNB SF1 IC queries as openCypher text (release build, Apple M5), lance-csr's in-memory CSR mode runs 1.96× faster than Neo4j 2026.09 and wins 9 of 14 queries. It runs 4.36× faster than Kuzu 0.11.3 and wins 10 of 14 (geometric means). These are independent, unaudited measurements, not LDBC Benchmark Results, and the Neo4j and Kuzu numbers come from an earlier run on the same machine. The 30-query suite is unchanged from 0.1.0. Its results, next to Neo4j, Kuzu, Ladybug and lance-graph, are in the README and in graph-benchmark-ldbc.

Packaging, documentation and tooling

• uv instead of pip: dependencies are locked in uv.lock, and development setup is uv sync --locked --no-install-project followed by uv run --no-sync maturin develop. make install, make test and make lint wrap these steps.
• Documentation: the README states the project's goals: a single-machine, openCypher-compatible engine on Lance, multimodal knowledge graphs, and DataFusion for graph traversal. It marks distributed execution as out of scope, and credits lance-graph and Apache DataFusion. DEVELOPMENT.md and the build steps match the current setup, including the protoc requirement.
• Release CI: each built wheel is tested by its file path, so PyPI's copy of the same version can't stand in for it. Pull requests build wheels only when the wheel build itself changes.
• The package metadata names the lance-csr authors.

Known limitations

• Integer-only list literals such as [1, 2, 3] parse as 32-bit float vectors (the vector-search syntax), so integers above 2^24 lose precision (#3).
• Unbounded variable-length alternation (-[:A|B*0..]->) unrolls into many branches, and planning them dominates LDBC IC12's run time.

Full changelog: v0.1.0...v0.2.0

v0.1.0

lance-csr 0.1.0

lance-csr is a fork of lance-graph at commit 3014793. It is a research test bed for making multi-hop path traversal fast with a CSR adjacency index and Apache DataFusion over the Lance format. This first release replaces the tree with work developed on top of that commit. Thanks to the lance-graph maintainers and contributors for the foundation: the Cypher parser, semantic analysis, the DataFusion planner, the graph configuration and the Python bindings.

Breaking changes

• The Python distribution is lance-csr, and the import is lance_csr (previously lance-graph / lance_graph). The Rust crates keep their lance-graph* names for now.
• Python 3.12 or newer is required.
• Removed: Delta Lake and Unity Catalog support, the catalog connector layer, and the knowledge-graph application (CLI, web service, Streamlit app, LLM extraction). No deltalake release supports DataFusion 54, and the rest sat outside the Cypher engine.

Dependencies

• Lance 13 (Rust and pylance >=13,<14), DataFusion 54, Arrow 58 and PyO3 0.28. lance-graph's Lance 1.0 reader panics on files written by current pylance.
• The Rust toolchain is pinned in rust-toolchain.toml, and the extension crate now has its own Cargo.lock.

Engine: Lance snapshots, refresh and streaming

• CypherEngine opens each Lance dataset as a lazy DataFusion table provider instead of copying it into memory. You choose per table: pass lance.dataset(...) to read from disk, or an Arrow table to keep it in memory.
• Each dataset is pinned to the version you pass in. refresh() builds and validates a full replacement, then swaps it in atomically. Queries already running keep their snapshot, and a failed refresh changes nothing.
• New methods: execute_batches() (streams a pyarrow.RecordBatchReader), explain(), refresh(), dataset_versions(), csr_stats() and node_index_stats().
• In-memory tables are split into batches across all cores, so operators that don't repartition still run in parallel.

CSR traversal (use_csr=True)

• The engine builds a compressed sparse row (CSR) adjacency index for each relationship type in both directions, from the relationship table alone, so node-table writes never invalidate it. The index runs inside DataFusion as extension operators, and the default (use_csr=False) behavior is unchanged.
• Hops: single hops and every hop of a variable-length path, whether directed, incoming or undirected, expand through the CSR instead of a relationship scan plus hash join. Any hop the CSR can't express falls back to a join in the same query.
• Target lookup by row position: target node columns are fetched directly by Lance row id (or Arrow row index) instead of through a second hash join. Columns no expression needs are pruned, and the target id is rebuilt from the neighbor value rather than read.
• Filters and closing edges: WHERE conditions on the target run inside the expansion. They fetch only the filter's columns, then the remaining columns for rows that pass. When the target is already bound (a cycle), the CSR keeps only the matching neighbor instead of expanding everything.
• Incremental maintenance: refresh() diffs Lance versions fragment by fragment and applies appended edges and deletions (as tombstones) on top of the shared base index. It rebuilds once changes exceed 25% of the base, or when the diff can't be computed. There's no API change: commit with pylance and call refresh().
• Hashing: the CSR's integer-keyed maps use ahash instead of SipHash, which dominated expansion time in profiles.

Planning

• Re-rooting: a MATCH starts at its most selective node (an id lookup first, then equality, then range filters), reversing path segments as needed.
• Statistics: DataFusion gets distinct counts for node-id and edge-endpoint columns, so its join sizing and build-side choices are right, including for equality filters pushed into Lance scans.
• Continued paths: when a pattern continues earlier rows (OPTIONAL MATCH, or MATCH after WITH) and both ends are bound, expansion starts from the end with the smaller mean fan-out. On LDBC IC5 this cut the query from about 1.7 s to about 0.24 s. A path bound only at its far end is now read from that end.

Cypher support

Enough to run all 14 LDBC Interactive complex queries:

• Patterns: relationship-type alternation [:A|B], also inside variable-length paths; multi-label nodes (m:Post:Comment) as a union of labels; optional endpoint labels per relationship type (with_relationship(type, src, dst, source_label, target_label)).
• Clauses: chains of WITH, WITH DISTINCT, WITH ... WHERE, whole nodes carried through WITH, OPTIONAL MATCH, and WHERE between reading clauses.
• Predicates: EXISTS { ... } and pattern predicates (NOT (a)-[:R]-(b)), in WHERE and in CASE conditions.
• Paths: shortestPath and allShortestPaths, as a breadth-first search over the CSR (or over an edge scan without it), plus length(p) and nodes(p).
• Expressions: arithmetic with precedence and parentheses, searched and simple CASE, coalesce, list literals, indexing, size, range, toString, date_part, to_epoch_ms, collect(DISTINCT ...), and the NULL literal in any case.

Correctness fixes to behavior inherited from lance-graph

Each fix has a regression test.

• Lance inputs were silently capped at 10,000 rows, because the bridge called a pylance method that doesn't exist and fell back to take(range(10000)).
• Undirected patterns dropped every match that needs an incoming edge. For example, a person with 30 neighbors returned 29.
• Hops inside variable-length paths could be silently dropped, because path temporaries had no label mapping.
• Multi-label nodes scanned only their first label.
• collect() kept nulls.
• Aggregates with camelCase aliases (count(*) AS postCount) failed to plan, and so did a camelCase scalar carried through a second WITH.
• WITH didn't end variable scope.
• WITH b (a whole node) failed to plan.
• Overflowing numeric literals panicked, and LIMIT -1 / SKIP -1 wrapped around to 2^64 − 1.
• Empty Arrow inputs raised an error instead of returning empty results.
• Subclasses of lance.LanceDataset were loaded into memory and ignored by refresh().

Packaging

• Prebuilt abi3 wheels for Linux x86-64 (manylinux_2_28, via zig) and native Apple Silicon (macOS 11+), plus an sdist. Every wheel is tested on CPython 3.12, 3.13 and 3.14 in a clean environment without Rust.
• The CPU baseline is unchanged from lance-graph: haswell (AVX2/FMA) on Linux x86-64 and apple-m1 on macOS.
• Releases publish to PyPI through trusted publishing.

Known limitations

• The CSR handles integer node ids only. Other id types, inline target property filters and bound relationship variables use the join path.
• A CSR expansion keeps its input's partitioning, so a pattern rooted at one node runs its fan-out on one thread. Splitting large expansions at run time is future work.
• Relationship uniqueness is enforced for undirected variable-length paths of up to 3 hops, and only while no two edges join the same pair of nodes. It isn't enforced across separate pattern segments, which matches lance-graph.
• OPTIONAL MATCH and EXISTS plan their input twice.
• Alternation expands to at most 64 branches. Variable-length alternation is limited to 20 hops and to directed, unnamed segments.
• shortestPath supports a single relationship type, with no relationship variable or properties.