Skip to content

26.8.1

Latest
Latest

Choose a tag to compare

View all tags
@lvca lvca released this 03 Aug 21:15
26.8.1
87bdc67

ArcadeDB 26.8.1

Overview

This is a major release: 381 issues and pull requests closed under the 26.8.1 milestone - 281 issues and 100 PRs - out of 298 pull requests merged and 669 commits in total since 26.7.2. It also carries everything shipped in the 26.7.3 hotfix.

The headline work is in four places:

  • Security - 13 advisories closed, most of them on the wire protocols and the MCP endpoint.
  • Concurrency - concurrent writes to unrelated records of the same page no longer conflict, which is what made ArcadeDB usable under real multi-writer load on types with few buckets.
  • Graph integrity - a family of vertex/edge delete defects that could silently lose edges under concurrency is closed, and deleting a super-node vertex got 5x faster along the way.
  • Storage and indexes - bloom filters on compacted LSM series, a schema dictionary that is no longer capped at one page, a geospatial index that costs one entry per point instead of eleven, and TimeSeries TAG columns that are dictionary-encoded.

Upgrading is strongly recommended for every deployment. There are breaking changes and behaviour changes - they are collected in Breaking changes and upgrade notes and each is called out again where it belongs. No schema migration is required, and no existing database is rewritten.

Contents: Highlights · Security advisories · New features · Major fixes and improvements · Breaking changes and upgrade notes · Dependency updates

Highlights

Concurrent writes to unrelated records of the same page no longer conflict

ArcadeDB detects write conflicts per page, so two transactions that touched the same bucket page raised a ConcurrentModificationException even when they wrote completely unrelated records that merely happened to share it. On a type with few buckets and many concurrent writers this made a retry pointless: the retry ran straight into the same collision (#5279).

All three halves of that are gone:

  • Inserts into one page reserve their slot per in-flight transaction, so concurrent inserts get different positions (and different RIDs) instead of all being handed the same one.
  • Updates are replayed by the commit-time disjoint-slot merge whenever they stayed inside the page, which now includes a record that grew - a longer string, one more property - and not only an overwrite of the same size or smaller. Growth is the normal update shape, so leaving it out kept concurrent updates conflicting.
  • Deletes of a plain in-place record are replayed too (#5569). Such a delete only zeroes one slot-table entry, so it commutes with writes to every other slot.

Measured on the reported workload (one single bucket, attempts=1, no retry):

Scenario Before After
Concurrent inserts ~1750 conflicts / 2000 0
Concurrent sub-graph creation (6 vertices + 5 edges per transaction) ~270 / 320 0
10 transactions updating 10 different records of one page 9 failed / 10 0
Sustained updates, 8 writers on their own records of one page ~2083 / 2880 0
8 deletes + 8 updates of 16 different records of one page 15 failed / 16 0
10 transactions deleting 10 different records of one page 9 failed / 10 0

A ConcurrentModificationException is still raised - by design - when two transactions really write the same record: a byte-for-byte pre-image check makes sure no concurrent write is ever silently overwritten. Nothing changes for single-writer workloads and no application change is needed. The merge can be switched off with arcadedb.txPageSlotMerge=false.

The merges also prove their coverage now instead of trusting every writer to declare its pages (#5596): a page carrying even one byte written outside a declared, replayable write is refused and falls back to the ordinary retry. The three merge counters (edgeAppendMerges, txPageSlotMerges, mergesDeclinedByCoverage) are now visible to an operator in the PAGE-MANAGER block, in /api/v1/server, in Studio and on /prometheus (#5608).

Deleting a vertex no longer loses its edges - and is up to 5x faster

Four defects, all on the same path, all capable of committing a graph with edges pointing at a vertex that is gone (#5670, #5680, #5725, #5760):

  • An edge-list chunk that is momentarily unreadable - a normal MVCC window on a hot vertex, not a fact about the graph - was read as "nothing to remove here". The removal ended having removed nothing and the edge record was deleted anyway, leaving a back-reference on a neighbour. It is now a retryable ConcurrentModificationException, so the transaction re-reads a consistent view and completes the removal.
  • The same window on the vertex's own list meant no edges collected, and the vertex record deleted on top of that empty view.
  • An edge appended while the delete was running survived the delete with a live out and an in naming a record that no longer exists. A vertex delete now pins every page its edge list can grow through, at the version it read the list at.
  • Each edge was disconnected from both endpoints, one of which is always the vertex being deleted - whose lists are dropped wholesale moments later. Each edge is now disconnected from its far end only.

That last one is also where the performance is. 100k edges into one hub, on an Apple M-series laptop:

layout before after
promoted super-node (striped) 2374 ms 446 ms
classic single chain 350 ms 308 ms

The removal walk streams now: deleteVertex no longer materialises every edge into a list first, so there is no per-degree allocation on the path at all (tens of megabytes of retained heap on a million-edge super-node).

Visible effect. vertex.delete() / DELETE VERTEX and edge.delete() / DELETE EDGE can now raise a retryable ConcurrentModificationException where they previously "succeeded" while losing an edge. It is a NeedRetryException, so database.transaction(...) and the server's auto-retry for single-request commands absorb it. A client-managed explicit transaction over RemoteDatabase spans several HTTP requests, so its commit is not auto-retried and the caller should retry the transaction. A delete that keeps failing however often it is retried means the list is genuinely broken: the error now names the repair, CHECK DATABASE RECORD #12:3 FIX, and the retry after it goes through.

Bloom filters on compacted LSM indexes (enabled by default)

An LSM index lookup walks every compacted series from newest to oldest, and a series whose key range covers the key still costs a root-page search and a data-page read to discover it does not hold it. Each compacted series now carries a bloom filter that answers from a single 8 KB page (#5517).

Measured on 2M keys across 9 series (LSMTreeBloomFilterBenchmark):

measurement filters off filters on gain
absent-key lookups (a duplicate check) 199,743/s 386,138/s 1.9x
pages read for those lookups 1,490 705 2.1x fewer
bytes read for those lookups 372 MB 176 MB 2.1x fewer
present-key lookups 244,000/s 281,150/s 1.2x
  • On by default at a 1% target false-positive rate: arcadedb.indexBloomFilterRate=0.01. Set 0 to disable.
  • ~1.2 bytes per key on disk, about 3% of the index it describes, in a <index>_bf.bfidx component.
  • No rebuild and no migration. Filters are written by compaction, so an existing index gains them at its next compaction. They replicate over HA and are included in backups.
  • Helps most when compacted series overlap in key range - an email, a UUID, a business id. Ascending keys give each series a disjoint slice the root page already rules out. Range scans never consult them.
  • Observability: bloomSkippedSeries and bloomProbedSeries in the index statistics.

Backward and forward compatible with no version bump: an older ArcadeDB does not recognise the .bfidx extension and reads the index exactly as it does today.

The schema dictionary is no longer capped at a single page

Every type and property name is mapped to a small integer id, and that table lived in one page: 48,396 short names measured. Past it, CREATE PROPERTY and inserting a document with a new field name failed permanently with No space left in dictionary file, with no way to grow and no way back (#5560).

Names now roll over onto further pages, so the cap is gone.

  • No migration, and existing databases are not rewritten. A dictionary written by an earlier version is a dictionary of one page and loads unchanged; it gains rollover on the next write that needs it.
  • Appending a name is no longer quadratic. Growing to 500,000 names took 11.2s of pure array copying; it now takes 2ms.
  • New databases use a 65,536-byte dictionary page instead of 327,680, so a new name dirties and flushes 5x less. Existing databases keep the page size they were created with.

Rolling upgrade: upgrade followers before, or together with, the leader. Dictionary pages replicate as raw pages, and a follower on an older build writes page 1+ but reloads only page 0. A database that has rolled over can no longer be opened by an older ArcadeDB at all - loudly, not silently.

Geospatial index: a point costs one entry instead of eleven

A GEOSPATIAL index stored the whole GeoHash ancestor chain, so a single point wrote one entry per tree level - 11 by default. Everything an index write costs was multiplied by 11, and the continent-sized cells at the top of the tree collected one posting per record and grew without bound, which is why a bulk load with a geospatial index got slower the longer it ran and finally failed with ReplicatedEntryTooLargeException (#5478).

On an LSM-Tree, "every cell below C" is simply the key range [C, C+\uFFFF], so the ancestors do not need to be materialised. The index now stores only the frontier cells - exactly one for a point - and answers a query with a prefix range scan. On a 1M-point load into one country-sized box (GeoIndexIngestBenchmark):

arm wall clock index entries
no index (the floor) 10.8 s -
new layout 13.0 s 1,000,000
old layout 22.4 s 11,000,000

The index costs 5.3x less, the whole load is 1.7x faster, and the index is 11x smaller on disk. Area shapes index fewer cells still - a complete set of sibling cells collapses into its parent, 57% fewer for a small square and 74% for a jagged outline. Queries are also more selective and stream their candidates instead of materialising the whole set, and a POINT search shape now uses the index at all (geo.equals / geo.contains used to find nothing and fall back to a full scan).

Existing indexes keep working and are not rewritten - the layout is recorded per index. Opening the database says so once per index, and Studio shows a banner with the ready-to-run statement: REBUILD INDEX `Address[location]`. Note the shapeRel half of the fix lives in the shared query walk, so an index still on the old layout also stops skipping covering cells the moment the jar is swapped.

TimeSeries TAG columns are dictionary-encoded: 36x less page traffic

A TimeSeries mutable row is fixed-stride, so a STRING TAG column reserved 258 bytes whether the tag was us-east-1 or empty. Tags are low-cardinality by definition, so nearly all of that was padding that still had to be written, flushed and shipped through the WAL (#5519).

A TAG column now holds a 4-byte id into a per-type append-only dictionary component:

arm stride rows per 64K page
1 tag, 3 fields 290 B → 36 B 225 → 1819
10 tags, 3 fields 2612 B → 72 B 25 → 909
10 tags, 10 fields 2668 B → 128 B 24 → 511

The ten-tag arm went from writing 50.0 MB of pages for 2.1 MB of payload (23x amplification) to 1.4 MB (0.7x), and from 29.9 ms to 6.0 ms. Corroborated independently on real TSBS data (2,592,000 points) by @tae898 in the issue.

  • arcadedb.timeSeriesTagDictionaryMaxSize caps distinct values per type, default 1M.
  • STRING fields stay inline - a field is where high-cardinality text belongs.
  • Existing types keep the inline layout. The row format is versioned per type; a new TimeSeries type gets the encoding, an existing one has to be recreated to gain it. If you are benchmarking, point the harness at a fresh database or you will measure the old layout.

MCP: a module of its own, new tools, and per-principal scoping

The MCP server is now a dedicated arcadedb-mcp module (#5692) and gained the tool surface that makes it usable as a GraphRAG back-end:

  • vector_search (dense and sparse), hybrid_search, full_text_search and a bounded sample_records (#4860, #4861, #4862, #4863).
  • Prompts graphrag_query and build_knowledge_graph, the latter with an enforceable source-text fence (#4866, #5586).
  • Configurable tool profiles, per-database permission scoping and per-principal profiles on a shared endpoint (#4867, #4868, #5445).
  • MCP 2025-03-26 transport conformance: batches, notifications, GET and Origin handling (#5394), plus proper JSON-RPC -32602 for malformed members instead of HTTP 500 (#5585, #5620).

Much of this arrived from @justinblethrow-cloud.

Experimental: GraalVM native-image build of the server

An experimental native-image build of the ArcadeDB server is now part of the build (#5544) - a single self-contained binary with no JVM start-up cost. Experimental means exactly that: it is not yet part of the published distribution.

Security advisories

This release closes 13 security advisories, on top of the three closed in 26.7.3. Each is published in full - impact, affected versions and credit - as a GitHub Security Advisory on the repository; the summaries below are only a map of what changed. Upgrading is strongly recommended for any deployment that exposes a wire protocol, the MCP endpoint, or accepts queries from untrusted callers.

Authentication and authorization on the wire protocols

  • GHSA-fq9c-x968-g278 - the MongoDB protocol accepted commands without authenticating the caller. It now authenticates and enforces per-database authorization.
  • GHSA-m46c-jh3x-xwrp - the Redis protocol required no authentication. It now requires AUTH, supports the HELLO handshake, and can be served over TLS.
  • GHSA-c287-v325-j5jx - the Gremlin protocol did not enforce per-database and per-type authorization.

The authenticated principal is now bound on every execution path

The engine's per-user permission gates are deliberately no-ops when no principal is bound on the thread, which is how embedded and replication contexts skip them. Three paths reached the engine without binding it, so every gate silently passed:

Privileged operations that were not gated

  • GHSA-pff6-hp53-pj54 - the server-administration MCP tools (set_server_setting and the profiler controls) gated only on the global allowAdmin flag and ignored the caller. They are root-only now, matching POST /api/v1/server.
  • GHSA-vv82-qvpf-rjwv - DELETE FUNCTION did not require UPDATE_SCHEMA.
  • GHSA-hfp5-6gcp-8c75 - Cypher LOAD CSV could read local files without administrative privilege.
  • GHSA-qwgr-2c45-63xx - the database name was not validated when creating or dropping a database, allowing path traversal outside the configured database directory.

Untrusted input reaching the host

  • GHSA-4w2m-77c8-83mw - a caller-supplied URL was validated only on its first hop and re-resolved after the check, so a redirect or a DNS rebind could reach an address the validation had rejected (SSRF). Every hop is validated now, and the validated address is pinned for the duration of the fetch.
  • GHSA-wx28-2265-f788 - the scripting host-class allow-list was matched as a regular expression, so an entry could admit far more classes than it names. It is matched literally now.
  • GHSA-xmjm-8q85-g778 - range() materialised every element, so one query could exhaust the heap. The list is lazy now and a range beyond arcadedb.queryMaxRangeSize is refused as a client error.

Also hardened in this release

  • MongoDB protocol: field names and filter values can no longer inject SQL. A MongoDB command is translated into SQL, and the field names and values taken off the wire were embedded without escaping. A filter value containing a single quote closed the string literal, so updateMany({name: "v1' OR 'x' = 'x"}, ...) updated every document; a $unset / $inc field name containing a back-tick removed a property the client never asked for. Values are now bound as parameters and every field name is back-tick quoted, one dot-separated segment at a time (#5579, #5583).
  • Server and cluster status endpoints scope their per-database output to the caller. GET /api/v1/cluster, the ha.databases array of GET /api/v1/server?mode=cluster and the metrics.sparseVectorIndexes map now reduce every per-database entry to the databases the caller is authorized for. POST /api/v1/cluster/bootstrap-state and GET /api/v1/cluster?presence=true move behind the root check the seven mutating Raft endpoints already use.
  • Studio no longer carries schema names in inline onclick handlers (#5580).

New features

Storage and indexes

  • CHECK DATABASE RECORD <rid> - check and repair named records only, instead of two full passes over a type (#5680). Combines with FIX and COMPRESS.

  • CHECK DATABASE FIX reclaims orphaned edge-list segments (#5375), and rebuilds unreadable edge lists from the surviving edge records.

  • Live progress for CHECK DATABASE, REBUILD INDEX, COMPACT INDEX, BACKUP DATABASE and IMPORT DATABASE, in the engine, over HTTP, in the console and in Studio (#5372, #5376).

  • COMPACT INDEX is reachable from SQL/HTTP, not only from the Java API (#5144).

  • GEOSPATIAL indexes have a builder of their own, so precision and tokenization are settable from SQL: CREATE INDEX ON Location (coords) GEOSPATIAL METADATA {"precision": 6}.

  • An LSM_VECTOR index compacts itself once its file is mostly garbage (#5516).

  • HASH indexes can key on a LINK, so an edge type's @out/@in pair can be indexed UNIQUE_HASH - the structural way to enforce edge de-duplication, which used to be accepted and then fail on every insert claiming page corruption (#5677):

    CREATE INDEX ON INITIATED (`@out`, `@in`) UNIQUE_HASH

Server and operations

  • Opt-in SLF4J logging. Slf4jLogger routes ArcadeDB's logs through the SLF4J facade instead of writing to stdout, keeping java.util.logging as the default, so an embedding host application gets consistent logs (#4276). arcadedb.log.impl is now a proper GlobalConfiguration entry (#5543). Contributed by @ruispereira.
  • The OpenAPI spec matches the registered HTTP route surface (#4895).
  • arcadedb.server.httpQueryDefaultLimit makes the HTTP row cap configurable (default 20000, -1 for unlimited), and truncated responses say so - see below.

Query engines

  • Parallel top-K for LSM_SPARSE_VECTOR - a sparse top-K is split into parallel RID ranges (#4085), 3.4x on real SPLADE data.
  • Cypher CREATE CONSTRAINT ... IS UNIQUE / IS NODE KEY upgrade a plain index in place, so a Neo4j migration script that creates indexes and then constraints keeps working.

Major fixes and improvements

Bulk load

  • A failed bulk load answers immediately instead of waiting for the rest of the upload. POST /api/v1/batch rejects a payload it cannot use, but did so only after reading the rest of the upload, so on a 25M-line load the client was told nothing for fifteen minutes and then nothing at all (UT000002: The response has already been started). The verdict is now delivered as soon as it is reached (#5470). A valid line is no longer blamed for a truncated upload, and the truncation check never blocks.
  • GraphBatch.close() restores the configured WAL flush, not the default, so durability is no longer silently downgraded after any bulk load (#5378).
  • Two concurrent GraphBatch instances on the same database are refused instead of silently losing edges (#5666).
  • Time-series HTTP ingest batches into one append transaction per measurement.

Graph engine

  • Deleting an edge no longer leaves its back-reference behind under concurrency, on all three operations that disconnect one: DELETE EDGE, moving an edge, and DELETE VERTEX (see the highlight above).
  • Ghost-edge pruning during iteration no longer rolls back and replaces the caller's transaction (#5694).
  • TX_RETRY_DELAY is read from the database configuration instead of the global static, so a per-database override applies (#5693).
  • Broken multi-page records are deletable again by every path, and CHECK DATABASE FIX repairs them.

Indexes

  • countEntries() no longer counts tombstones as live entries. Deleting every record of a type left the index reporting 1 with zero records in the database (#5601). The contract is now stated on Index.countEntries().
  • An index cursor never hands out a null entry, hasNext() is exact, next() throws NoSuchElementException once exhausted, and getRecord() / getKeys() describe the entry next() last returned (#5635). Two user-visible consequences fixed with it: SELECT min(...) / max(...) could answer with a deleted key, and a delete-heavy index reported one entry too many.
  • Cursor is AutoCloseable, and a leaked cursor no longer pins a retired index file for the lifetime of the database - the retire guard now holds weak references (#5662). A dozen call sites that abandoned a cursor partway now release it.
  • A HASH index refuses a page size its bucket pages cannot address. Above 65536 bytes the 16-bit slot offsets truncate and the index destroys itself on insert, reported as Detected cycle in hash index. The page size is validated at creation now (#5713).
  • Partitioned types, three fixes. A lookup on a secondary index was pruned to the bucket the lookup key hashed to rather than the partition key's, so it read the wrong bucket and a secondary UNIQUE index stopped rejecting duplicates (#5589). A lookup key boxed differently than the stored value hashed differently, so on a LONG partition key every negative value missed (#5595). And a partition key whose bucket is not a function of the index key - BINARY, DECIMAL, zone-carrying DATETIME, COLLATE CI - is now refused instead of quietly breaking UNIQUE (#5603). The strategy is also persisted now, so a partitioned type no longer comes back round-robin after a restart (#5637).
  • CREATE INDEX IF NOT EXISTS answers for the index asked for. It matched on the property set alone, so a NOTUNIQUE index answered "already there" to a request for a UNIQUE one and the constraint was never created (#5675). It now compares the index kind, the uniqueness, and the settings the METADATA clause named (#5765).
  • An existing index is never dropped implicitly any more - the old rebuild-on-mismatch could leave the type with no index at all if the rebuild then failed on the stored data. Opt in with withReplaceIfIncompatible(true).
  • Manual indexes work. ManualIndexBuilder.create() registered the wrong object, so the very next commit fenced the database. Three further defects on the same path are fixed with it (#5765).
  • Copying a type copies its records and its index definitions, not just their names. Every index on the copy came out empty, every copied record came out with no properties, and every index setting outside the three-argument overload was replaced by a default (#5723).
  • REBUILD INDEX no longer resets a non-default GeoHash precision, the same defect fixed for FULL_TEXT in #4732: TypeIndexBuilder shadowed IndexBuilder's metadata field.
  • An index METADATA key is now either applied or reported. A typo was indistinguishable from a correct clause (#5639). Four dense-vector settings that were unreachable behind that silence (efSearch, inactivityRebuildTimeoutMs, neighborOverflowFactor, alphaDiversityRelaxation) are now settable and persisted.
  • Full-text BM25 is comparable across buckets (#5267), and a combined full-text index expands per property (#5181).
  • Bounded range scans over multi-series compacted indexes no longer drop rows or mis-order descending results (#5214), an accented partial prefix no longer returns rows of other keys (#5321), and a mixed-type index range is bounded by type category (#5225).
  • LOCK TYPE / LOCK BUCKET lock every index component file, including compacted sub-indexes.
  • A STRING property no longer reads back as a geometry. Deserializing any string whose first characters were POINT, POLYGON, ... parsed it as WKT and handed back a Shape, so a description column holding "POLYGON shaped, see attached" did not round-trip (#5600).

Vector search

  • A search aimed at a deleted region finds the survivors instead of nothing. Two independent causes: the traversal was told every node was an acceptable answer, so a beam that filled with tombstones declared itself finished; and a tombstone was scored through a placeholder vector whose cosine similarity came back Infinity, making every tombstone the best candidate in the beam (#5558). getStats() gains a bruteForceScans counter.
  • countEntries() is no longer torn by a concurrent graph rebuild - a rebuild now publishes a fully populated replacement with a single reference assignment (#5568).
  • The location cache is gone and locationCacheSize is refused. It was never a cache bound: an evicted location is unrecoverable and every reader reads a missing location as deleted, so a cap of 100 over 1000 live vectors made countEntries() report 100 and dropped neighbours from searches (#5559). See the breaking-changes section.
  • In-memory bloat is fixed - VectorLocation objects were never removed from the map, an OOM after weeks of operation (#5516) - and LSMVectorIndex.remove() no longer scans every vector id per call, which made any record update on a vector-indexed type O(index size) (#5318).
  • A committed vector is no longer intermittently missing from search (#5615), and GraphSearcherPool can no longer hand out a searcher bound to a replaced graph (#5648).
  • A query with an allow-list narrower than k no longer costs a full index scan every time (#5748), and a DOT_PRODUCT index warns about non-unit vectors (#5750).
  • Sparse vectors: compaction no longer fails past the 2GB WAL buffer (#5189), the top-K heap no longer allocates a Float on every comparison (#5473), and block-max skipping cuts the p50 that tracked total posting length (#5388).
  • Studio can create a vector index - the Add Index dialog built a statement without the METADATA the engine requires, so it was simply impossible (#5607). dimensions is now enforced at creation: an index created without it accepted writes and indexed nothing, forever, without a warning.

TimeSeries

  • DATE / DATETIME_* / DECIMAL / BINARY fields no longer silently corrupt the next column, and the sealed layer restores the declared column type (#5475).
  • Unbounded last-point-per-tag no longer scans the whole series (208 ms → 2.5 ms) via a per-shard latest-ts shortcut (#5414), and an unbounded descending scan no longer reads the whole unsealed tail (#5416).
  • Ingest no longer boxes every numeric sample (#5474).

HA / Raft clustering

  • A materialized view no longer makes the leader ship page versions followers never received (#5492) - the WALVersionGapException / non-converging-resync / lost-write shape. recordFileChanges now ships the WAL its callback committed, and both SQL and Cypher statements execute against the replicated database instance rather than the inner LocalDatabase (#5655).
  • A follower's index can no longer be short after an LSM compaction, which silently returned fewer rows (#5443).
  • Concurrent transactions on a shared follower Database handle no longer corrupt records or lose committed writes (#5503).
  • A leader crash between the Raft commit and the phase-2 apply no longer loses a locally-originated write (#5407), and the phase-2 ticket is released when an abandoned entry applies, so Raft log compaction is no longer pinned until restart (#5410).
  • The Raft log no longer grows unbounded until disk-full on low-write clusters (#5345), and a permanently wedged follower replication channel now escalates instead of staying dead until a leader restart (#5346).
  • The leader is no longer advertised as available before it is ready to serve (#5453), and DROP DATABASE no longer deletes files synchronously inside the Raft apply, blocking the state machine (#5454).
  • Kubernetes: a recreated follower is no longer permanently stranded (#5268), a peer is no longer silently removed from the Raft configuration when its pod is deleted (#5275), a stranded follower's Raft server is no longer left CLOSED (#5271), the crash-loop on newTI < oldTI is gone (#5291), the default raftStorageDirectory lands inside server.databaseDirectory (#5272), and database bootstrap metadata no longer lives inside the Raft storage directory (#5277).
  • Cluster status is accurate: the configuration is re-emitted rather than logged once at bootstrap (#5304), the LATENCY column reports replication RTT rather than heartbeat age (#5314), a never-appended follower is no longer reported HEALTHY (#5295), and POST /api/v1/cluster/leader validates its body and reports whether leadership actually transferred (#5276).
  • Polymorphic count(*) FROM V no longer returns node-dependent totals from bucket counter drift (#5297), and parameterized Gremlin commands work on followers (#5187).
  • ServerSecurityDatabaseUser no longer floods the logs with a per-access INFO line for every fileId created after the security config load - thousands per second, rotating logs in seconds (#5269).
  • A failed startup no longer leaves an unkillable JVM: the shutdown hook blocked on the lifecycle lock (#5450), and non-daemon background threads no longer keep an embedder JVM alive after a leaked Database (#5418).
  • schema.load() no longer runs before checkForRecovery(), which left a database unopenable after a crash before the dictionary page was flushed (#5325).

OpenCypher

  • A hop onto an already-bound vertex counts every relationship joining the pair. The optimizer's operator for such a hop was built as a semi-join, so it answered once per input row and threw the rest of the pair away. The shape where it shows is a cycle, whose closing hop always has both endpoints bound: anything aggregating over such a pattern under-reported wherever parallel edges exist between a pair, which is normal in transaction and payment graphs (#5663). Row counts can go up, and that is the fix.
  • An unbound $parameter is an error, not null. A query referencing a $name the caller never bound evaluated it to null and ran to completion against a value nobody supplied, so a de-duplicating WHERE NOT EXISTS { ... {id: $id} ... } CREATE guard degraded into an unconditional CREATE. ArcadeDB now raises Neo4j's own Expected parameter(s): id (#5501, #5561). EXPLAIN is exempt; bound-to-null is not unbound.
  • A subquery body is part of the query, not a string it carries. EXISTS { }, COUNT { } and COLLECT { } held their body as text, edited it once per outer row to correlate it, ran it as a standalone statement and absorbed any failure into the expression's neutral value - false, 0, [] (#5656, #5657, #5658). Bodies are ASTs now, run with the outer row as a seed, and a failing body is reported. That removes the whole class of text-rewriting bugs behind #4995/#5165,
    #5464, #5461 and #5541, and every validation phase reaches inside a body
    (#5626).
  • A non-numeric argument to abs() and friends is a 400, not a 500 (#5484), with the message phrased in the vocabulary of the language: Type mismatch: abs() expects an INTEGER or a FLOAT argument but got STRING. A literal is rejected before the query runs, and every argument position is covered. The five follow-ups (#5602) close the arity-guard blind spot, extend parse-time validation beyond RETURN/WITH, implement charLength() and isNormalized(), unregister charAt(), and make case folding independent of the server's default locale.
  • Arithmetic errors are client errors. 64-bit overflow, division and modulo by zero answer HTTP 400 and Bolt's Neo.ClientError.Statement.ArithmeticError (#5545, #5647). Floating-point is untouched.
  • The two count push-downs agree. RETURN count(*) LIMIT 0 returned a row; MATCH (m:Label) RETURN count(*) scanned for a number the type counter held; a pattern that cannot match cost 200 record reads to answer 0; MATCH (a)-[:LINKS]->(b) RETURN count(*) with an unlabelled anchor answered 0 (#5715, #5686).
  • Inline WHERE predicates apply everywhere they are written - in MATCH, in EXISTS { }, in pattern comprehensions, on variable-length patterns and in both shortestPath evaluators (#5460, #5462, #5463, #5464, #5480, #5481, #5489, #5490).
  • Planner: an inline property map is planned as the equality predicate it stands for, so it no longer keeps the whole statement out of the cost-based optimizer (#5446); a bound-target expansion filters on the segment's neighbour pointer (#5660); composite indexes are seeked by their whole key (#5444); and bounded variable-length paths use indexed and IN-list anchors (#5357, #5387, #5393).
  • Plus a long tail of semantics fixes: stdev()/stdevP() on empty input (#5459), the self-loop counted twice in an undirected pattern comprehension (#5456), UNWIND as the first clause of a subquery (#5461), head()/size() on an unsupported argument (#5476, #5477), an explicit null in an optional argument (#5629), a procedure's wrong argument count (#5627), and the Duration sign in abs() (#5649).

SQL

  • Integer arithmetic fails on overflow instead of wrapping, and division by zero is a client error (#5164, #5647, #5494).
  • Back-tick quoted names containing a backslash are no longer mis-parsed. A name that arrived already escaped grew one backslash on every parse and re-emission until it no longer resolved, and a name ending in a backslash left the closing back-tick indistinguishable from an escaped one, so the quoted token absorbed the SQL that followed it. See the breaking-changes section.
  • MATCHES works when the regular expression contains multiple dots (#5258), the MATCH rid filter is no longer discarded (#5315), and TRAVERSE/SELECT from a bound RID collection no longer NPEs (#5505).
  • UPDATE ... MERGE supports parameterized payloads (#5347).
  • A large batch script no longer dies with StackOverflowError when closing its execution plan (#5708), and a script that hits RETURN or BREAK releases every line before it (#5720).
  • ceil()/floor() return FLOAT (#5382), || rejects non-STRING operands (#5298), chained comparisons are honoured (#5284), split() with an empty delimiter no longer appends a spurious element (#5390), and datetime(map) honours epochSeconds/epochMillis (#5274).
  • dateTimeImplementation=java.time.Instant no longer breaks reading DATETIME values. The moment a DATETIME column crossed the JSON boundary it threw UnsupportedTemporalTypeException: Unsupported field: YearOfEra, which took out HTTP, the remote driver, Studio, toJSON() and the SQL .format() method. Instant is now anchored to UTC before the pattern is applied.

Server, HTTP and observability

  • A truncated query response is no longer indistinguishable from a complete one. The HTTP endpoints serialize at most 20,000 rows and reported that nowhere: same 200, same body shape (#5711). A limit the caller states is now honored as written, a query's own LIMIT raises the cap, and the response carries {"limit": 20000, "returned": 20000, "truncated": true}. arcadedb.server.httpQueryDefaultLimit configures the default; RemoteDatabase.setMaxResultRows(Integer) sets it per connection; Studio marks the row count (truncated).
  • Query and HTTP metrics survive an in-process server restart. The server added a Micrometer registry to the JVM-wide global registry on every start and removed none on stop, so after a restart arcadedb.query.duration and arcadedb.http.requests read back as 0 while the gauges kept reporting the stopped server (#5565). The subsystem is now dismantled on stop, reference counted across several servers in one JVM.
  • Unexpected internal server errors stay visible in production-mode logs (#5374), with the full stack trace.
  • HTTP status classification: client errors are classified on the Postgres, Redis, MongoDB and GraphQL wire paths (#5628), the command API no longer answers 500 ClassCastException for a non-string language or command field (#5222), and JSON array payloads are accepted (#5415).
  • The "not found" message for a missing bucket reaches the user again. Schema.getBucketById(int) raises in exactly the cases a caller would test for null, so every if (bucket == null) written after one was dead code - including the branch holding the recovery guidance for an EXTERNAL property whose bucket is not loaded (#5636). Schema now exposes null-returning getBucketByIdIfExists(int) / getBucketByNameIfExists(String).
  • Studio shows a profiler counter sitting at zero instead of hiding it - for a health signal whose good state is zero, hiding it was backwards.
  • The remote driver honors the server's 503 "please retry", and the console no longer terminates a comment on a semicolon (#5457).

Wire protocols

  • PostgreSQL: quoted identifiers are identifiers, not string literals (#5369); the schema path no longer collapses ARRAY_OF_* / DATETIME_* to VARCHAR, and ARRAY_OF_SHORTS no longer fails queries (#5311); LIST OF EMBEDDED is advertised as json[] (#5289); nested arrays serialize (#5366); and a single-column projection no longer returns every column (#5367).
  • Gremlin: a traversal result carrying a raw RID serializes (#5309); ArcadeGraphManager no longer caches a stale graph after the underlying database is reopened (#5307); hasLabel() no longer returns elements of the wrong kind (#5223).
  • Bolt: two concurrency defects behind the flaky concurrentSessions are fixed, and a missing parameter answers Neo.ClientError.Statement.ParameterMissing rather than SyntaxError.
  • MongoDB / Redis: see the security section.

Studio and bindings

  • Studio: the Indexes tab no longer triggers a SQL syntax error (#5469), the Add Index dialog can create dense and sparse vector indexes, geospatial indexes needing a rebuild are flagged with a ready-to-run statement, and schema names travel in data-* attributes instead of inline onclick handlers.
  • Python bindings: the max_connections default is aligned with the engine (#5352), vulnerable numpy and py7zr floors are raised, and the dependency floors are audited in CI (#5616).

Breaking changes and upgrade notes

No schema migration is required and no existing database is rewritten. The items below change behaviour that existing code or scripts may depend on.

SQL and Cypher

  • Inside a back-tick quoted name, a backslash escapes the next character. A literal backslash has to be doubled: SELECT FROM `C:\\data` where `C:\data` used to work. Only names that actually contain a backslash are affected. The Postgres wire protocol now writes back-tick identifiers the way it already read them.
  • Cypher: an unbound $parameter raises Expected parameter(s): x instead of evaluating to null. To keep the old behaviour for a specific query, bind the name explicitly to null.
  • Cypher: a failing EXISTS { } / COUNT { } / COLLECT { } body returns its error instead of the expression's neutral value. If a body of yours errors on a subset of rows, a query that used to return rows will now raise. The old answer was wrong, not merely quiet.
  • Cypher: parse-time validation reaches every clause and every subquery body. A query whose bad call sits in a clause the validation never walked - or in a branch that never executes - is now rejected before it starts. Also: a variable's kind now survives WITH *, so MATCH p = (a)-[:KNOWS]->(b) WITH * RETURN p.name is rejected as the path-property access it always was.
  • Cypher: a wrong argument count now raises CommandSemanticException (a CommandParsingException subclass), where the runtime guards used to throw CommandExecutionException. Embedded code catching the latter around a call should catch CommandParsingException.
  • Cypher: row counts can go up where parallel edges join a pair and the pattern has a hop onto a bound vertex (typically a cycle). A saved report or a threshold calibrated against the old numbers should be re-checked.

Indexes

  • CREATE INDEX ... METADATA refuses an unknown or malformed key, on LSM_VECTOR, LSM_SPARSE_VECTOR and FULL_TEXT (GEOSPATIAL already did). A stored migration carrying a stray or misspelled key used to run and is now refused - which is the point, since the key was never doing anything. Also, a METADATA clause on an index type with no settings at all (LSM_TREE, HASH) now fails the statement instead of being ignored.
  • A guarded CREATE INDEX IF NOT EXISTS naming a setting MAY NOW RAISE where it used to be a no-op. Any re-runnable script of the shape CREATE INDEX IF NOT EXISTS ON Doc (embedding) LSM_VECTOR METADATA {"dimensions": 384, "efSearch": 120} raises HTTP 400 if the index already there carries a different value for any key the clause names. Drop the keys you do not actually require, or align the value.
  • An existing EUCLIDEAN or DOT_PRODUCT vector index changes its search results on the first reopen. The persisted definition names the metric similarityFunction while the reader looked only for similarity, so such an index has been scoring with COSINE since the first restart after it was created, against a graph built with the right metric. It now scores with the metric it was created with - distances and result ordering change, and they change to what was asked for. Nothing to re-create or rebuild. COSINE indexes are unaffected.
  • arcadedb.vectorIndex.locationCacheSize and the per-index locationCacheSize metadata are refused for any positive value. Remove the key from any CREATE INDEX script before upgrading. The global setting is still tolerated (a startup line must not stop a server booting) and warned about once per index; -1 and 0 are still accepted. Plan for ~90 bytes per live vector - the figure getStats() now reports as estimatedLocationIndexBytes, up ~3.75x from the 24-byte payload it used to quote. An index that appeared to work under a cap on a large corpus may now need a larger heap.
  • getOrCreateTypeIndex no longer upgrades an incompatible index: asked for a UNIQUE index where a NOTUNIQUE one covers those properties, it raises IllegalArgumentException naming both definitions instead of dropping and rebuilding. Use buildTypeIndex(...).withReplaceIfIncompatible(true) if replacing really is what you mean.
  • withPageSize(0) now means "use the default", for every index type. A caller passing 0 expecting a failure now silently gets a working index at the default size.
  • withPageSize(262_144) on a HASH index is now refused with a message naming the 256-65536 range, where it used to silently produce a 65536-byte index.
  • A LINK HASH index created with this release is not readable by an earlier build. Such an index could not hold data before this release anyway (it failed on the first insert); it has to be dropped before downgrading.

Metrics and reported statistics

  • The monotonic engine metrics are Prometheus counters now, so the exported series are renamed with a _total suffix: arcadedb_engine_page_cache_hits_total, ..._misses_total, arcadedb_engine_pages_read_total, ..._written_total, arcadedb_engine_wal_bytes_written_total, arcadedb_engine_mvcc_conflicts_total, arcadedb_engine_page_merges_edge_append_total, ..._slot_total, ..._declined_total, arcadedb_engine_tx_write_total, ..._read_total, ..._rollbacks_total, arcadedb_engine_queries_total, arcadedb_engine_commands_total. Existing dashboards and alerts on the old names need updating. The three genuinely instantaneous readings (wal_files, files_open, databases) stay gauges and keep their names. Those totals are also all-time JVM totals now: six of them used to be summed over the currently open databases only, so closing one made the total go backwards and Prometheus fabricated a rate spike.
  • A server restart in the same JVM resets the query and HTTP counters, because the values belong to the server that recorded them. An application that wants its own meters to survive the server's shutdown should register them before starting it.
  • CHECK DATABASE output changed: a corrupt vertex or edge produces one warning per record instead of two, the , removing it wording is gone from a run that removes nothing, totalWarnings now counts distinct messages rather than occurrences, and the progress total for a vertex/edge step is the record count rather than twice it.

Java API

  • Schema gains getBucketByIdIfExists(int) and getBucketByNameIfExists(String) - source-incompatible for anyone implementing com.arcadedb.schema.Schema outside the project. An unknown bucket in SQL now raises CommandExecutionException / CommandSQLParsingException carrying the specific message, where several paths previously let a SchemaException escape.
  • AfterRecordReadListener must return a mutable record when returning a different one than it was handed (typically record.modify(), or one built from scratch). reload() renders the replacement through the serializer now rather than taking its buffer (#5755) - which also fixes the live defect where a replacement record aliased the per-thread scratch buffer and started answering with another record's values after an unrelated save() on the same thread.
  • BucketLSMVectorIndexBuilder no longer exposes its settings as public fields. Every fluent withX() method is preserved (with withEfSearch added), and getVectorMetadata() returns the whole configuration.
  • TypeLSMVectorIndexBuilder.withLocationCacheSize(N) is deprecated and refuses a positive N.
  • A manually named LSM_VECTOR index now keeps its name across TRUNCATE TYPE, where it used to come back under the auto-derived form.
  • BucketSelectionStrategy.getBucketIdByKeys(List, Object[], boolean) is the new contract; the single-argument form and DocumentType.getBucketIndexByKeys(Object[], boolean) are deprecated and never prune.

Operational

  • Partitioned types: a database that ran partitioned(...) on a type carrying more than one index may already hold duplicates in a secondary UNIQUE index, admitted while the check was reading the wrong bucket. The constraint is enforced again from this release, but existing rows are not retro-validated - check those indexes and REBUILD INDEX them.
  • Geospatial: existing indexes keep the old layout and are not rewritten, but they change query behaviour the moment the jar is swapped (geo.equals / geo.contains go from returning nothing to returning the right rows). Run REBUILD INDEX `Type[prop]` to get the ingest and selectivity gains.
  • Rolling HA upgrade: upgrade followers before, or together with, the leader, because of the multi-page schema dictionary.

Dependency updates

Around 80 dependency bumps landed in this cycle, almost all through Dependabot. The notable ones:

  • Security-driven: the Jackson family is pinned to 2.22.1 via jackson-bom, Ivy raised to 2.6.0 to clear CVE-2026-26032, eight security-critical Studio packages bumped as a group, and the numpy / py7zr floors in the Python bindings raised. Dev-scoped npm alerts in the E2E harnesses were cleared and the gaps that let them accumulate closed.
  • Runtime and engine: GraalVM 25.2.4, JVector 4.0.0-rc.9, Groovy 4.0.33, Logback 1.6.1.
  • Build and test: JUnit Jupiter 6.1.2, Cucumber 7.34.6, maven-enforcer-plugin 3.6.3, maven-jar-plugin 3.5.1, license-maven-plugin 5.1.1, frontend-maven-plugin 2.0.2, native-maven-plugin 1.1.6, Playwright 1.62.0, testcontainers ≥ 4.15.0, prettier 3.9.6, and the usual GitHub Actions refresh (setup-node, setup-go, setup-python, setup-dotnet to their v7/v6 majors).
  • Studio: ApexCharts 6.6.1, swagger-ui-dist 5.32.11, FontAwesome 7.3.1, marked 18.0.7, sax 1.6.1, plus the routine build-tooling churn.
  • Go E2E: 21 modules updated as a group, and Dependabot now also updates indirect Go modules.

Two versions are deliberately frozen and documented as such: the Gremlin ANTLR runtime and Groovy majors, which TinkerPop cannot take.

Thanks

To everyone who reported, reproduced, reviewed, tested and fixed - in particular @adepase, @alphafarmer, @borutjures, @cmettier, @danieljuhl, @focusmacula, @gramian, @ironluca, @ivanfrias, @justinblethrow-cloud, @kl-demi, @KyaniteSolutions, @LepsyMikolaj3301, @mdre, @rthuffman, @ruispereira, @Rupert1987, @shulei5831sl, @sunil-pateel, @tae898, @TobiasJoseHermann, @vivekjustthink, @waterWang, @xdevsapps, @YaeSakuraQ,.

Full Changelog: 26.7.2...26.8.1

Assets 10
Loading