Repository navigation
v4.4.0
Added
- Compiled native plans (spec 257). A native read with one or more hops is walked by a class written for its
plan's shape (IVGC.P<n>, extendingGraph.KG.Cypher.Exec). Its loops have the hops' types and directions written
in, the node checks and the grouping inline, properties read fromrdf_props' storage globals by name, and a
variable-length hop's ends walked as the interpreter walks them. A new shape is answered by the interpreter while a
background job compiles it.Graph.KG.Cypher.Gen.Mode("sync" | "off" | "job")sets this per process, and
^IVG.CypherGen("mode")the namespace default; a deploy that installs classes drops every compiled class. Answers are the interpreter's, row for row (148 E2E cases; the SQL path holds each but
the three five-hop BI14 cases, whose SQL statement does not finish preparing). SF1, laptop, fixed-length plans
compiled against interpreted (results/local-compiled-sf1/): median 1.12x; BI14 2.6x, BI09 1.6x, BI04 and BI03
1.4x; BI18 1.9x in one process. Variable-length plans were compiled later and have no separate measurement.
With the sort-key class below, SF1 on a laptop against Neo4j 5.24 and Memgraph 3.9, one engine at a time: median
IVG/Neo4j 1.01 (20 of 39 at or ahead, 36 within 3x), IVG/Memgraph 1.00 (REPORT_TEMPLATE.md).
On AWS (r7g.4xlarge, SF1, every engine capped at 900 s): median IVG/Neo4j 0.76 (28 of 40 at or ahead, 37 within
3x), IVG/Memgraph 0.54, against 1.00 and 0.77 at 4.3.0.
SF10 on AWS (r7g.8xlarge): median IVG/Neo4j 0.90 (22 of 39 at or ahead), against 1.13 at the last SF10 run;
IVG/Memgraph 0.57 against that run's Memgraph.
Changed
-
IVG now writes and compiles classes in the namespace at run time (
IVGC.P<n>, built by a backgroundJob), and
records them in^IVG.CypherCompiled. A namespace under change control sets^IVG.CypherGen("mode") = "off"
(ADMIN_GUIDE, "Classes IVG writes at run time"). A process that cannot record a shape (a role that may only read
the database, a read-only database) runs the read interpreted, natively. -
Upgrading: re-run
initialize_schema()to deployGraph.KG.Cypher.GenandGraph.KG.Cypher.SortKey. The native
plan version is unchanged (2), so until then reads run on the 4.3.0 executor: the same answers, without compiled
plans. Every fix below is in the Python package and applies without a redeploy. A deploy deletes the compiled
classes (IVGC.*); each shape compiles again on its first run. -
The native executor computes ORDER BY sort keys with
Graph.KG.Cypher.SortKey.Key, which holds the code of the
SQL path'sCY_SORT_KEYas a plain class method. It used to call the UDF's generated class, and one key on an SNB
timestamp cost 44.7 us that way against 1.8 us now. An ordered read computes a key per match: IC2 files about 10,000.
A unit test holds the class and the UDF to the same code, and an E2E test holds them to the same keys. -
Grouped reads end sooner. A compiled walk's count(DISTINCT) members are moved to
^||IVG.CypherRun("gd")only in a
worker, whose export reads them; every run did it before.EmitGroupsreads a group's sort keys from its stored
values rather than making and parsing its JSON. In one process on SF1 (1ade0533): BI04 100 to 84 ms, BI18 48 to 43. -
A compiled lone variable-length hop counted by count(DISTINCT) of its end checks each end inline rather than through
NodeOK: IC3 (5,000 ends, a label and a<>each) 22.7 to 15 ms on SF1, one process (24ed922a).
Fixed
- The SQL path refused every
USE GRAPHquery whose undirected hop goes through the hop's edge CTE, which included two
hops of one type, an unorderedLIMITand a cross filter. The query failed with SQLCODE -29, "Field 'E3.GRAPH_ID'
not found", because the CTE did not projectgraph_id. USE GRAPHdid not reach the reads inside aWITHstage: theMATCHbefore aWITHread every graph. With one
node id in two graphs,USE GRAPH 'g1' MATCH (a:L) WITH a RETURN count(*)answered 4 where the answer is 1, on both
paths, and any staged query could mix other graphs' rows into its answer. Each stage's reads are now scoped before it is
built (tests/e2e/test_with_stage_graph_scope_e2e.py).- In a multigraph, relationship equality took parallel relationships for one:
r1 = r2kept 5 pairs where 3 are equal,
r1 <> r2kept 0 of 2. The comparison now includesedge_id(tests/e2e/test_multigraph_rel_identity_e2e.py). - A duplicate
create_edgecost about 30 ms, and so did each duplicate inbulk_ingest_edges' SQL fallback. The
insert was refused by the unique key, and the refused row was rolled back through the journal. Both now insert with
... WHERE NOT EXISTS: a duplicate costs a probe, and the result is unchanged (False, or the temporal row adopted). ledger.verify()right afterledger.enable()reporteddiverges/unrecorded_writesfor every default-graph
relationship written before enable. The genesis records the default graph as"\u0000"and the tables read it back
asNone; the comparison now keys both as the default graph.- In a multigraph, the SQL path took two parallel relationships for one when a pattern's second hop could reuse the
first's. It dropped the path, counting 4 where there are 8. The hop-to-hop exclusion now comparesedge_idas well. approx_count_distinctanswered what its HLL route reads, whatever the query asked: a far end's label or WHERE, a
minimum of 0 or above 1, a start's label,USE GRAPH, another RETURN column, a count of the start andSKIP/LIMIT
were dropped, soMATCH (s {id: $x})-[*1..2]->(m:B)counted every end. Its source was the first string among the
statement's bound values, which could be anUNWIND's, a WHERE's property key or an unused parameter. A query with
no variable-length hop, one that did not translate, one whose source was not found and a source^NKGdid not hold
answered 0. The route now reads its source from the start'snode_idpin and the rest from the parsed query; every
other query is counted exactly, ascount(DISTINCT ...), with a warning that says why, and takes as long as that
count (checkresult.metadata.warningsfor "counted exactly"). A column with noASwent through the translator's
BFS route instead, which took the far end's label for the source and returned BFS rows; it now takes the same route.
A query that cannot run (an undefined variable) now fails ascount(DISTINCT ...)does, rather than answering 0.- With native Cypher on, a query over a label mapped to a SQL table (
map_sql_table) answered no rows, with no error:
the native executor does not read mapped tables. It now leaves such queries to the SQL path
(tests/unit/test_sql_table_bridge.py, which failed three cases at 4.3.0, passes).