Repository navigation
VelesDB v8.0.0
[8.0.0] - 2026-11-02
MAJOR release. The breaking changes and how to move past each one are in
docs/guides/MIGRATION_v8.0.0.md.
API deprecated one major or more ago is removed: in Rust,store_node_payload,
OperationalMetrics::shared,effective_ef_search,CollectionStats, the
7.0 update-check entry points,TraversalMetricsand velesdb-migrate's
PineconeConfig::environment, andEdgeStoreloses its serde impls; in
Python, the nativeCollection.search, thetop_kalias and
get_graph_store. Point payloads and edge properties nested deeper than 124
levels are refused (VELES-017) where they used to be written and then fail
to read back or to restore; a secondSPARSE_NEARis refused where it was
dropped, and a failing branch fails a hybrid dense + sparse search. The WASM
VelesQL executor validates with core'sQueryValidator. The Tauri plugin and
the mobile bindings carry core'sVELES-###codes, andMobileGraphStoreis
a core graph collection.velesdb-wasmstorage modes return core's results
and keep f32 vectors in every mode. Graph edges with properties now survive a
flush and a reopen, and aMATCHrange predicate returns every matching node.
enable_streamingreturnsResultand refuses a config that aborted the
process; caller-chosen sizes no longer abort it either. Stored floats read
back bit for bit (serde_json'sfloat_roundtrip, at a parse cost stated
below), and the Python binding raises on a value nested past 127 levels.
Every read a VelesQL statement makes now asks theDatabaseObserver
about the collection it reads — aggregates, UNION arms after the first,
JOIN secondaries,MATCHenrichment and the REST point routes included.
A browser page can no longer drivevelesdb-serverwithout a configured
origin andHost, andvelesdb-mobile'sget/getByIdcarry a denial.
VelesQL compares numbers by exact value with no epsilon, resolves an
ORDER BYaggregate's alias, refuses aHAVINGit cannot evaluate, and
MIN/MAXreport each column's exact extreme in
AggregateResult::min_values/max_values.
Do not downgrade to 7.x a graph collection whose edges carry properties:
7.x cannot read the 8.0 edge snapshot.
Security
-
The read gate decides every read a
VelesQLstatement makes, aggregates
included. In 7.1.0 aDatabaseObserver'son_query_requestwas never
asked about an aggregate:Database::execute_aggregate, the server's
/aggregateand the aggregates/queryroutes there, the CLI and the Tauri
querycommand ranCOUNT,SUM,AVG,MIN,MAX,GROUP BYand
HAVINGover every stored row, ignoring anAllowWithScopefilter and a
Denyalike. A scalar subquery reduced by an aggregate (WHERE amount > (SELECT MAX(score) FROM other)) readotherthe same way, in aSELECT,
anUPDATE, aDELETEor anINSERT. Other reads the gate missed or
decided wrongly:- every operand of a
UNION,INTERSECTorEXCEPTafter the first read
its collection unscoped, and undecided when it named another collection; - a JOIN read its secondary collection undecided;
- a
MATCHaddressed through the_collectionparameter (how/query,
the REPL and the SDKs send one) was decided on an empty collection name,
and a scope was dropped, since a traversal never reads theWHEREit was
folded into; the collections its@collectionnodes enrich from were not
decided at all; SELECT EDGESwas not decided;- REST
GET /collections/{name}/points/{id},
POST /collections/{name}/points/scrolland
GET /collections/{name}/points/{id}/relationswere not decided; velesdb-mobileget/getByIdwere not decided, and now throw a
MobileObserver's denial;velesdb-wasmdecided a JOIN on its first collection only, so a
WasmObservernever saw the secondary one.
In 8.0.0 each of these reads asks the observer about the collection it
reads. ADenyaborts the statement or request with the observer's error.
The remaining reads that skipped the gate are decided too:- collection metadata reads —
DESCRIBE COLLECTION, a plan-onlyEXPLAIN
(VelesQL and REST/query/explainwithanalyze: false), REST
GET /collections/{name}/empty/sanity/config/stats
/diagnostics, Pythoncount()/len()/info()/is_empty()/analyze()/
explain(), mobilecount()/all_ids()/get_stats()/diagnostics(),
wasmlen/is_empty/DESCRIBE COLLECTION, Tauriget_collection/
list_collections/is_empty/list_indexes— which leakedpoint_count
and statistics undecided; aDenynow refuses them and a scope they
cannot honour refuses withVELES-009rather than run unscoped; GET /collections/{name}/graph/traverse/stream(SSE), which never
consulted the observer at all;- every
AgentMemoryread — thequery_semantic/query_episodic/
query_proceduralVelesQLbridges and the nativequery/recent/
recall/list_all/serialize/relation accessors — which read the
_semantic_memory/_episodic_memory/_procedural_memorycollections
undecided (the Tauri memory commands ride on them); - the remaining Tauri command reads —
search,search_ids,
batch_search,text_search,hybrid_search,multi_query_search,
sparse_search,hybrid_sparse_search,get_points,
scroll_collection,get_edges,traverse_graph,
traverse_graph_parallel,get_node_degree— which ran on detached
collection leaves and never consulted an observer passed to
VelesDbState::new_with_observer.
A scope is AND-composed into theWHEREof the statement's target and of
every set-operation operand, so it can only narrow. Where a read cannot
apply one — a traversal, a JOIN's secondary side,@collection
enrichment,SELECT EDGES— the statement is refused withVELES-009
rather than run unscoped, as the server's graph and/matchendpoints
already answered (a point's relations now answer the same way). Over REST
a point outside the scope reads as not found, and a scroll returns only the
scoped points. Insidevelesdb-corethe
executors that read for a statement take anAuthorizedQuerythat only
the gate builds, so a statement path that skips the gate does not compile.
What an integrator sees change is in
MIGRATION_v8.0.0.md§2.
The follow-up adversarial pass closed what it still missed:
- a scope filter the row-filter lowering cannot express — a
similarity()/vector/graph conjunct (lowered to a match-all tautology,
running unscoped), a full-textMATCH(lowered to substring
CONTAINS, narrowing by the wrong predicate), a$paramor subquery
value (no policy value of its own) — is refused withVELES-009at the
gate itself, on every surface.CONTAINS_TEXTstays legal: it already
is a strict substring predicate, so its lowering is faithful; - a scope column qualified with a JOIN table (
other.col) is refused for
the same reason — pushdown would apply it to the join side, where a
LEFT JOINkeeps every base row anyway; SemanticMemory::edge_count/count/is_emptyand
EpisodicMemory::get_with_embedding, decided like their siblings —
theedge_countgate had been written insideclear()(a mutation)
instead;- every
PyGraphCollectiondirect read (get_edges,get_outgoing*/
get_incoming*,has_edge,node_degree,get_node_payload,
all_node_ids,traverse_*,edge_count,__len__/count/is_empty,
get,__contains__), gated asGraphTraversal; - the remaining metadata reads: Python
analyze()/get_collection_stats/
analyze_collection/collection_diagnostics, mobileanalyze(); the
index-metadata getters on both (Tauri already gateslist_indexes), and
the collection-config getters (dimension,metric,storage_mode,
is_metadata_only,pending_reopen,guard_rails,is_delta_active,
check_auto_reindex_divergence) that duplicate whatinfo()already
gates; Database::explain_query/analyze_collection/collection_diagnostics
now gate in-core, so a surface can no longer forget to decide them —
RESTPOST /analyzewas never decided.get_collection_statsstays
surface-gated by design: core consults it on every uncached plan build;- Python
explain()now also authorizes the query'sFROM— a plan for
SELECT * FROM deniedno longer ships to a caller who cannot read it; on_querytelemetry for aMATCHaddressed through_collection
reported an empty collection name; it now reports the gate's target.
- every operand of a
Added
-
velesdb_core::MAX_JSON_NESTING(127) andMAX_VALUE_DEPTH(124), also
undervalidation: the nestingserde_jsonreads back, and the deepest
edge properties and point payloads core accepts (see the twoVELES-017
changes under Changed). -
Agent-memory semantic search at an explicit quality in Python, Tauri and
mobile. The bindings now reach core 7.1.0's
SemanticMemory::query_with_quality: Python's
memory.semantic.query(embedding, top_k=10, quality=None)and the Tauri
semantic_querycommand's optionalqualityfield (guest-js
SemanticQueryRequest.quality) take a mode string, parsed by core's
parse_search_modeasCollection.search_with_qualityandsearchalready
parse theirs:fast,balanced,accurate,perfect,autotune/auto,
custom:<ef>oradaptive:<min_ef>:<max_ef>, each ef in 16–4096. An
unusable mode is aValueErrorin Python andVELES-009in Tauri, never the
default quality. Mobile'sVelesSemanticMemorygains
query_with_quality(embedding, top_k, quality)(queryWithQuality), taking
theSearchQualityenumVelesCollection.search_with_qualitytakes; it stays
on that gated collection search rather than core'sSemanticMemory, which
reads the collection leaf without the observer read gate every mobile read
passes (#1392). Omitting the quality keeps the collection's own everywhere. -
api_types::validate_search_qualitychecks a typedSearchQualityby
the rules ofparse_search_mode. EveryCustom/Adaptiveef must be in
theef_searchrange and anAdaptivemin_efmust not exceedmax_ef,
so a binding's enum is accepted exactly when itscustom:/adaptive:
string would be. Mobile'sSearchQualityconversion goes through it, and
now refusesAdaptive { min_ef: 512, max_ef: 32 }, which core's parser
already refused and which ran as a search that never escalated. -
collection::graph::dfs_traverse(&EdgeStore, source, &TraversalConfig):
core's DFS over a standalone edge store. It is the loop a collection's
traverse_dfs_configruns (moved, not copied, into
collection/graph/traversal_dfs.rs, with its #906 push-time bound), so a
binding holding a bareEdgeStoregets the same order, depths, paths and
bounds as a collection; a test pins the two answers equal. -
velesql::Query::result_kind()classifies a statement by the result it
produces. It returns the newvelesql::QueryResultKind(Rows,Mutation,
Deletion,Ddl,Train,Admin), withSELECT EDGESasRows. The
mobile and WASM bindings each carried their own copy of this rule, and the
copies disagreed; both now map from this one, and the enum is exhaustive so
that a new kind fails to compile in every binding that reports it. -
GraphCollection::get_edge(edge_id)returns one edge by ID. Finding one
edge tookget_edges(None)and a filter, which clones every edge in the
graph; this clones only the one asked for. The mobile graph store's
getEdgeuses it. -
GraphCollection::try_remove_edge(edge_id)reports a removal that did not
happen.remove_edgeanswersfalseboth for an absent edge and for a
removal the edge write-ahead log refused (or an index found out of sync with
the edge's source shard), so a caller could not tell "already gone" from "left
in place".try_remove_edgereturnsOk(true)for a removal,Ok(false)for
an absent edge, and core's storage error (VELES-006) carrying the reason
otherwise, the codeTRUNCATEalready gives the same failures.
remove_edgeis unchanged. The mobile graph store'sremoveEdgeuses it. -
GraphCollection::truncate()removes every edge, then every node, by the
rule VelesQLTRUNCATEfollows. It attempts every edge, removes the nodes
only if no edge removal failed, and returns(edges, nodes)removed, or
core's storage error (VELES-006) naming the first failed edge with the
nodes left in place.TRUNCATEand the mobile graph store'sclearcall
it, where the mobile store had its own copy of the loop. -
The Python binding exposes three core contracts as module functions.
velesdb.hash_id(value)isvelesdb_core::hash_id, the canonical string →
u64point id;velesdb.metric_higher_is_better(metric)is
DistanceMetric::higher_is_better, the order core sorts results by; and
velesdb.validate_search_quality(mode)runs core'sparse_search_mode, the
parsersearch_with_qualityuses, raisingValueErrorwith its message. The
MIT integrations call them instead of carrying copies (below), so a change in
core reaches them with the next wheel. -
FusionStrategy::from_name: one fusion strategy name grammar in core.
velesdb_core::FusionStrategy::from_name(name, FusionParams)readsrrf,
average(aliasavg),maximum(aliasmax),weightedand
relative_score(aliasrsf), case-insensitively, fills every parameter the
caller leaves out with core's default, and builds the weighted strategies
through the validating constructors. An unknown name is the new
FusionError::UnknownStrategy, whose message lists the accepted names. A
singlerelative_scoreweight implies its complement.FusionParamsis
#[non_exhaustive], built fromFusionParams::default()and itswith_*
setters. The defaults it fills are now constants too:DEFAULT_RRF_K(60),
DEFAULT_RELATIVE_SCORE_DENSE_WEIGHTandDEFAULT_RELATIVE_SCORE_SPARSE_WEIGHT
(0.5 each), beside the existingDEFAULT_WEIGHTED_*ones. Python gets the same
grammar asFusionStrategy.from_name(name, *, k=..., avg_weight=..., ...). -
FusionStrategy::fused_direction: the direction of fused scores, in core.
velesdb_core::FusionStrategy::fused_direction(&[ScoreDirection])returns the
polarity of whatfuse_with_directionsproduces:LowerIsBetteronly when
every branch is a distance and the strategy keeps the input score space
(Average,Maximum,Weighted),HigherIsBetterforRRF,WeightedRRF,
RelativeScoreand any mixed or empty set of branches.
fuse_with_directionsreads it instead of repeating the rule. Python exposes
it asFusionStrategy.fused_higher_is_better(metric), the direction of the
scoresmulti_query_searchreturns with that strategy;velesdb_common.scores
gainsfused_higher_is_better(strategy, fusion, metric), which calls it and
falls back to a literal set pinned to it by a drift test for a wheel older
than 8.0.0, andto_similarity(score, higher_is_better). -
parse_search_modeandSearchQualityno longer needpersistence.
SearchQualitymoved fromindex::hnsw::paramsto its own ungated module.
It is re-exported atvelesdb_core::SearchQualityand at its old path.
api_types::parse_search_modeandmode_to_search_qualitynow build without
thepersistencefeature, so adefault-features = falseconsumer
(velesdb-wasm) reads search modes with core's parser instead of its own copy. -
The executor conformance fixture covers the WASM executor's own statement
families.conformance/velesql_executor_cases.json(v2.1.0) adds, each
asserted againstvelesdb-coreand thevelesdb-wasmexecutor: NEAR ranking,
hybrid NEAR + metadata andsimilarity()thresholds with per-row scores
(near_cases),NEAR_FUSEDrrf/average/maximum (fusion_cases), queries both
engines must refuse (rejected_cases), UPDATE/DELETE followed by a SELECT
(dml_cases) and DISTINCT/named-column projection (projection_cases).
LIKE, ILIKE and OFFSET join the scalarcases(X015-X020), which the CLI
runs too. Three accepted gaps are recorded asdocumented_divergences
D007-D009: DELETE takes any WHERE on WASM but onlyid = N/id IN (...)
on core, UPDATE may setvectoron core only, and an unranked row scores
1.0 on core and 0.0 on WASM. -
velesdb_core::velesql::distinct_evaluator. DISTINCT deduplication
(apply_distinct) moves out of thepersistence-gated collection query
engine, besidewindow_evaluator, so the WASM executor dedups through the
same key as core. -
Guard-rails have one live source in core.
Database::guardrail_limits()
returns the limits in force, andDatabase::update_guardrails_with(|limits| ...)edits them in place and propagates the result as one step, so two
concurrent partial updates both land.RateLimiter::set_limit_qpsand
CircuitBreaker::reconfigurechange a limiter or a breaker in place, keeping
its buckets and its state.velesdb_core::metrics::LimitType, the argument
ofGuardRailsMetrics::record_limit_exceeded, is now exported.
QueryLimits::validaterefuses arate_limit_qpsor a
circuit_failure_thresholdof0, which would reject every query or open
the breaker on the first failure now that both take effect;
update_guardrails_withapplies it to the edited limits and returns
Error::Config(VELES-009), changing nothing, when they fail. The
existingupdate_guardrailskeeps its unchecked signature. -
VectorCollection::pending_reopennames the streaming settings not in
force yet. It returns thePendingReopensettings (DeferredIndexing,
AsyncIndexBuilder, withfield_name()) whose persisted value differs from
the component the collection runs, and that the next open will apply. Python
Collection.pending_reopen()and Mobilepending_reopen()return the field
names. -
Database::refresh_storage_gauges(&OperationalMetrics)sets
velesdb_documents_total(points across every collection) and
velesdb_index_size_bytes(the summedindexes_memory_usage) from the live
database;/metricscalls it before every export. -
The learning-loop
Stopchecklist names the memories and the outcomes
behind feedback. The agent hooks record the ids of every memory a
successfulrecall,recall_where,recall_fusedorwhyreturned (with
the server'sstaleverdict), and the checklist names exactly those, instead
of asking for feedback on "every recalled memory". Beside them it lists
verified outcomes: on Claude Code, a single test, build or lint command that
failed and then passed or still fails (a newpost-tool-use-failure.shhook
reads the documentedExit code N; a pipe or a chain is never counted), and
on both hosts a commit reverting another in an edited repository since the
last checkpoint. No outcome drivesfeedbackby itself: the checklist asks
for feedback only on a listed memory whose effect an outcome demonstrates,
citing it. The records follow the edit records' rules (private directory,
atomic files, links and malformed records refused) and travel in the
pending/delivered manifest, so an interruptedStopre-emits them. Codex
does not record check outcomes: how its shell results report an exit status
has not been measured.scripts/sync-agent-hooks.pyinstalls the new hook. -
Database::gated_search_outcome:gated_searchwith the read gate's
refusal kept apart from the search's own outcome. -
QueryLimits::client_timeout_ms(the budget a request's own
timeout_msresolves to) andQueryLimits::timeout_budget(whose budget
ran out — the one classificationVelesQLand the REST search share). -
velesql::Grouper,GroupInputandGroupOutput(both
#[non_exhaustive],GroupInput::newbuilds one): the grouping core's
SQL and vectorGROUP BYand the WASM executor share.Grouper::push
fails withError::Configpast the group budget,Grouper::newwith
Error::Query(V018) for a non-numeric HAVING threshold. -
velesql::compare_json_values(ORDER BY's value order, also still at
collection::search::query::ordering),velesql::row_field,
SelectOrderBy::directed, andSelectStatement::grouped_sort_keys,
order_by_aggregates_as_fieldsandis_vector_group_by. -
ValidationErrorKind::OrderByNotInResult(V016),
DuplicateOutputColumn(V017) andInvalidHaving(V018). -
Filter::matches_pointandfilter::Condition::matches_point: evaluate a
filter against a point, its id included, which a VelesQLidcolumn
reads.velesql::explain::IndexType::PointId(PointIdLookup). -
LlamaIndex
GraphLoader.load_from_nodes(extract_relations=True)writes
node relationships as edges. The option was documented but never read, and
the call always returned"edges": 0. EachSOURCE,PREVIOUS,NEXT,
PARENTandCHILDrelationship whose target is in the graph — loaded by
the same call or, with a native graph collection, by an earlier one —
becomes an edge labelled with the kind's name, with a deterministic id, so a
reload writes nothing new and returns the same counts. Any other
relationship is skipped, never written as a dangling edge, and counted in
the newskipped_relationskey of the result.
Changed
-
BREAKING (behavior) — a query with two
SPARSE_NEARclauses is
rejected. A query runs one sparse search, and a secondSPARSE_NEAR
(underANDorOR) was dropped without a word, the query answering for
the first sparse vector alone; withUSING FUSIONit even passed fusion
validation as a second branch. It now fails with a query error
(VELES-010) naming the count; the guard that existed for it was never
called. Run one query per sparse vector and fuse the results. -
BREAKING (behavior) — edges whose properties nest deeper than 124 levels
are refused.add_edgeandadd_edges_batchreturnVELES-017
(SchemaValidation), before any lock or write, for an edge whose
properties nest more thanvalidation::MAX_VALUE_DEPTH(124) levels of
arrays and objects, the properties map counting as the first. Core writes
edges into JSON documents, whichserde_jsonreads back up to
validation::MAX_JSON_NESTING(127) levels; the deepest, the agent-memory
snapshot, holds an edge three levels down. A deeper edge was written, then
skipped with a warning on edge-WAL replay, or failed the agent-memory
restore. The edge snapshot refuses to write an edge it could not read
back (pastMAX_JSON_NESTING, should one reach anEdgeStoredirectly),
rather than write a file it cannot read; one a 7.x edge WAL replays at 125
or 126 levels still snapshots. -
BREAKING —
EdgeStoreno longer implementsSerialize/Deserialize.
Its derived serde form was theedge_store.binlayout, a private
persistence format; the snapshot now has its own private form. Persist a
store withto_bytes/from_bytesorsave_to_file/load_from_file. -
BREAKING (behavior) — LangChain
GraphLoaderwrites one persistent graph
collection. Its relations went to an in-memoryvelesdb.GraphStore, lost
when the process ended, and its entities to a separate vector or metadata
collection under alabelpayload key. Entities are now the nodes of the
graph collectioncollection_name(created on the first load, with the
dimension of the first embedding, or without embeddings when there is no
embedding_fn), labelled with their type through_labelsso
MATCH (n:TYPE)finds them, and relations are its edges. Migration: a
collection_namethat names a collection the previous loader created is not
a graph collection, so the load fails; delete it or pass a new name. -
The LangChain, LlamaIndex and Haystack integrations require
velesdb>=5.1.0, the first release whoseSearchOptionstakesk, which
they now pass instead of thetop_kalias removed below. -
BREAKING (behavior) — point payloads nested deeper than 124 levels are
refused. Every payload ingest path (upsert,upsert_bulk,
upsert_bulk_from_raw,upsert_metadata, graph node payload writes)
returnsVELES-017(SchemaValidation) naming the point, before anything
is written, for a payload nesting more thanvalidation::MAX_VALUE_DEPTH
(124) arrays and objects.serde_jsonreads back 127 nested levels: a
payload deeper than that, reachable from the Rust API and the Python
binding (the REST API's JSON parser already stopped earlier), was written
to the payload log, and then every read of that point failed; one at 125
to 127 levels read back, but made the agent-memory snapshot holding it,
three levels deeper, fail to restore. The bound is the one edge properties
have, so every stored value fits every document core writes it into.
Upgrading: a payload a 7.x store holds at 125 to 127 levels stays
readable, but writing it back unchanged (a VelesQLUPDATE, any
read-modify-write) is now refused; flatten it first. -
BREAKING (behavior) — LangChain relevance scores are higher-is-closer under
every metric.similarity_search_with_relevance_scoresreturned the raw
score and keptscore >= score_threshold; undereuclideanandhammingthe
raw score is a distance, so a threshold kept the farthest documents and
dropped the closest. A distancedis now returned as1 / (1 + d)(the
metric's direction comes from velesdb-core throughvelesdb_common.scores),
so the threshold, andas_retriever(search_type="similarity_score_threshold")
which calls it, keep the closest documents. Cosine keeps its[0, 1]
rescaling, dot and Jaccard their raw similarity, and fused dense + sparse
scores pass through.GraphRetriever.score_thresholdnow filters seeds on
these relevance scores, and the seeds'relevance_scoremetadata carries
them.similarity_search_with_scorestill returns the raw distance; a sparse
search that falls back to dense-only is now scored as the dense search it
ran, so its cosine scores are normalised too. Migration: a threshold written
as a maximum distancedon aeuclideanorhammingstore becomes
1 / (1 + d). The asyncasimilarity_search_with_relevance_scores, and the
asyncsimilarity_score_thresholdretriever that calls it, run the same
method; they raisedNotImplementedErrorfrom LangChain's default
_select_relevance_score_fn. -
BREAKING (behavior) — Haystack
Document.scoreis higher-is-closer under
every metric whenscale_score=True(the default). Aeuclideanor
hammingscore was the raw distance, soDocumentJoinerand rankers, which
sort descending, put the farthest document first. A distancedis now
1 / (1 + d); cosine keeps its[0, 1]rescaling, dot and Jaccard their raw
similarity, andscale_score=Falsestill returns raw scores. A fused score
(fusion=...) that is still a distance (average,maximumorweighted
over a distance metric, as core'sfused_directionstates) is mapped the
same way whateverscale_scoresays. -
LlamaIndex similarities on a
euclideanorhammingstore change range
for fused, hybrid and text scores. They pass through as core returns them
(see Fixed): RRF scores near1 / 60, BM25 scores that can exceed1,1.0
for acontains_text_searchmatch, where they were all squeezed through
1 / (1 + s). Asimilarity_cutofftuned on those paths needs re-tuning. -
velesdb_common.scores.to_similaritykeeps the order of negative
lower-is-better scores.weightedfusion over distances subtracts a hit
bonus, so a very close match can score below0; it was clamped to1.0,
which tied every such match. A negative scoredis now2 - 1 / (1 - d),
in(1, 2); a distanced >= 0is still1 / (1 + d). -
The TypeScript SDK's WASM backend creates
pqandrabitqcollections.
WASM_CAPABILITIES.storageModeslists all five core storage modes: velesdb-wasm
keeps f32 vectors in every mode and returns core's results for it (#2544), so
the reason for refusingpqandrabitqwithNOT_SUPPORTED(the browser
stored both as SQ8) is gone. No mode saves memory in the browser. -
BREAKING (behavior) — the
velesdb-wasmVelesQL executor validates queries
with core'sQueryValidator. A query core refuses is now refused in the
browser instead of answered. In particularUSING FUSIONover a single
NEAR(one fusable branch) fails with V012 for every strategy:rrf,
maximumandaverageused to be accepted as a documented no-op, butrrf
re-ranked the hits by RRF against a constant metadata branch, so order and
scores changed.NEAR_FUSED ... USING FUSION 'weighted'or'rsf'also fails
with V012 instead of silently running asrrf. Migration: drop theUSING FUSIONclause from a single-NEARquery (the metadata predicate already
filters), and giveNEAR_FUSEDrrf,averageormaximum. -
BREAKING (Tauri plugin) — every rejection carries core's
VELES-###code
and a newrecoverablefield. The plugin coded the conditions it detects
itself apart from core, so one app could receive two codes for a dimension
mismatch depending on which layer caught it. The plugin-local codes are
replaced by the code core gives the same condition, read from the core
variant rather than copied:NOT_FOUNDbecomesVELES-003,
DIMENSION_MISMATCHVELES-004,INVALID_CONFIGVELES-009and
SERIALIZATION_ERRORVELES-012. A VelesQL parse error is core's query
error,VELES-010, instead ofINVALID_CONFIG, as on WASM and mobile, and a
core failure duringtrain_pqor an aggregation query keeps its own code
instead of being flattened intoINVALID_CONFIG; their messages are now core's
(Database error: [VELES-0xx] ...) rather thanVelesQL parse error: ...,
PQ training failed: ...orAggregation error: ....recoverableis core's
Error::is_recoverablefor the condition.TASK_JOIN, a blocking task that
panicked or was cancelled, has no core equivalent: it keeps its code and
reportsrecoverable: false. Migration: matchVELES-009where you matched
INVALID_CONFIG, and so on for the other three; the guest-jsCommandError
type gainsrecoverable: boolean. -
BREAKING (mobile) —
MobileGraphStoreis a graph collection, stored and
traversed by core. The store was a second graph engine of its own: hash
maps in RAM, its own BFS and DFS, and a JSON file written bysaveand read
byload. Its multi-source BFS kept one result per node where core keeps one
per path, and it accepted edges to nodes it had never been given. It is now a
handle on a coreGraphCollection, so every traversal is core's and the graph
is durable with no save step: it survives an app restart, and its nodes are
the ones VelesQLMATCHsees throughexecuteQuery. Changes for Swift and
Kotlin:- The
MobileGraphStore()constructor,save(path)and
MobileGraphStore.load(path)are gone. Create the collection with
createGraphCollection(name)(orcreateGraphCollectionWithEmbeddingsfor
nodes with vectors) and get its store with the new
VelesDatabase.getGraphStore(name), which returns nil/null for a missing
collection and throws for one that is not a graph collection.flush()is
new. - Every method now throws. Reads pass the database's read gate as
GraphTraversal, so an observer registered withopenWithObservercan
deny them, and a scope filter is refused since no graph read can apply
one; writes and storage can fail now that they reach disk. removeEdge(edgeId)returns aBool:truewhen it removed the edge,
falsewhen there was none, which stays a no-op. A removal the edge
write-ahead log refuses leaves the edge in place and throws core's storage
error (Database, codeVELES-006) instead of passing silently.clear()
runs core'sTRUNCATErule (the newGraphCollection::truncate): every
edge is attempted, then the nodes are removed only if none was refused, so
a refusal throws with the nodes left in place and no edge is left pointing
at a deleted node.- Core's rules apply.
addEdgerefuses an edge whose source or target was
never added, and an empty edge label; edge labels are trimmed.
properties_jsonmust be a JSON object and must not contain the reserved
_labelskey, which is where core keeps the node's label.
bfsTraverseParallelreports a node reached from two sources once per
source, ordered by depth across sources. A node'svectorneeds a
collection created with embeddings. addNodeon an existing ID replaces its label and properties; avector
replaces its embedding andnilkeeps it, as core'supsert_nodedoes
(the old store replaced the whole record). A node read back carries one
label, its first: one with several labels, written through VelesQL, keeps
only the labeladdNodeis given if it is rewritten.propertiesJsonreads
back as core re-serializes it (key order and whitespace may differ, and
"{}"reads back as nil), andgetOutgoing/getIncomingreturn edges in
core's order, not insertion order.
Migration: an app holding a graph saved by the old
save(path)creates the
graph collection, gets its store, and callsimportSnapshot(path)once. It
reads that JSON file as written, upserts its nodes and adds its edges, and
refuses, before writing anything, an edge ID the store already holds or an
edge whose endpoint is in neither the file nor the store; add any such node
withaddNode, then import again. An edge whose ID another writer takes
while the import runs is reported as an error rather than skipped. The old
file can be deleted afterwards. - The
-
BREAKING (mobile) — the
CollectionandDimensionMismatcherrors carry
core'scodeandrecoverable, asDatabasedoes. A missing or
duplicate collection and a dimension mismatch reached Swift and Kotlin as
typed variants with no code, so a retry policy or a log keyed onVELES-###
had to special-case them. Both variants gaincodeandrecoverable, read
from core's error for the condition:VELES-002for a missing collection,
VELES-001for a duplicate,VELES-004for a dimension mismatch, all
recoverable.getCollectionandgetGraphStoreon a collection of the wrong
kind answerVELES-002, the code core's own lookups give it, with the kind in
the message. Their messages now lead with the code
([VELES-004] Dimension mismatch: expected 384, got 2). Migration: a Swift
pattern or a Kotlin destructuring that binds every field of either variant
binds the two new ones too; code reading the fields by name is unaffected. -
BREAKING (behavior) —
velesdb-wasmstorage modes return core's results,
and no longer save memory. AVectorStorecreated withsq8,pqor
rabitqkept only a per-vector SQ8 encoding, and one created withbinary
only sign bits decoded to 0/1; searches scored that reconstruction, so the
same mode name returned different scores and a different order than a
velesdb-corecollection, andget()returned the reconstruction. Core keeps
the f32 vectors in every mode and re-ranks its quantized paths with exact f32
(itsBinarybehaves asFull). The browser store now keeps the f32 vectors
in every mode and scores them exactly: same top-k and scores as core for the
same mode, pinned per mode and metric against a real core collection by
quantization_parity_tests. The cost is memory: every mode now takes 4 bytes
per dimension plus 8 per id (3080 bytes per 768-D vector, wheresq8took
~784 andbinary~104).batch_searchnow works in every mode, and
hybrid_searchscores every mode in one exact pass. Thepq/rabitq
console.warnabout an SQ8 fallback is gone, since there is no fallback.
One core-side split is documented rather than copied: after a PQ collection
trains (128 inserts), core's defaultsearch()returns ADC-approximated
scores whilesearch_with_qualitystays exact; the browser store returns the
exact ones. -
velesdb-wasmbinary format v3.export_to_bytes/savewriteVELSv3:
the v2 layout without its four quantized blobs, the f32 vectors being kept in
every mode. v2 and v1 blobs still load; a v2 quantized store loads as the
reconstruction it was searching over (SQ8 decoded through core's
QuantizedVector::to_f32, sign bits as 0/1), since its f32 values were never
saved. An older build refuses a v3 blob withUnsupported version: 3. -
velesdb_commoncalls the binding instead of re-implementing core. Its
FNV-1a copy and its search-quality grammar (preset set,custom:/adaptive:
regexes, a mirrored 16–4096 ef range) are gone.stable_hash_id(value, algorithm="fnv1a")isvelesdb.hash_idand raisesImportErrornaming
velesdb 8.0.0 with an older wheel; its SHA-256 default is unchanged, needs no
binding, and still differs from core's FNV-1a by design (KNOWN_LIMITATIONS
#12).validate_search_qualityaccepts exactly what core accepts (now
custom:+256too) and itsSecurityErrorcarries core's message; with a
wheel older than 8.0.0 it checks only the type and leaves the mode to the
binding, which refuses an invalid one at search time. The connectors' declared
velesdbfloors stay where they are, since every new call is
feature-detected. -
SearchConfigOptions.default_modereads core's own mode names. The
binding parsed and renderedfast/balanced/accurate/perfectfrom a
hand-written table; it now goes throughSearchMode's serde names, the ones
the[search]TOML section is read with, so a mode core adds is accepted
without a binding change. An unknown mode still raisesValueErrornaming
default_mode, now wordedinvalid search.default_mode: unknown variant ..., expected one of .... -
Every surface that takes a fusion strategy by name now goes through
FusionStrategy::from_name. Before, sevenmatchblocks had drifted apart:- CLI (
multi-search --strategy weightedand the REPL's
.hybrid-sparse --strategy weighted):weightednow uses core's
0.6 / 0.3 / 0.1 instead of the pre-#1545 literals 0.5 / 0.3 / 0.2, so it
ranks the same as the server, Python, WASM and Tauri. This reorders
weightedresults from the CLI. The REPL now reads names
case-insensitively and acceptsavg; - Tauri:
parse_fusion_strategyacceptsavgandmax, builds the
weighted strategies through the validating constructors (weights it would
have refused only when the fusion ran are refused when the strategy is
built), and a singledenseWeight/sparseWeightimplies the other. Ak
pastu32::MAXis refused whatever the strategy; - REST (
/search/multiand the hybridfusionobject): the accepted
names are unchanged. The two endpoints now report core's message for an
unknown strategy (Unknown fusion strategy '...'. Valid values: ...) and
for refused weights, instead of their own wording. The multi-query
endpoint used to sayInvalid strategy: ...; - WASM
fuse_results: the name grammar is core's. The N-branchrsf
aggregation stays WASM-local by design (fixture divergence D005). - MIT integrations (
velesdb_common.fusion.build_fusion_strategy, used
by LangChain and LlamaIndex): the name goes to the binding's
FusionStrategy.from_name, so they accept core's aliases and casing and
report core's errors. A wheel older than the export (velesdb < 8.0.0)
keeps the previous literal dispatch, as the othervelesdb_commonhelpers
do.
- CLI (
-
velesdb-wasmvalidates search modes with core'sparse_search_mode.
search_with_qualityaccepts and refuses exactly what every other surface
does, and its refusals now carry core's messages (Unknown search mode '...'. Valid values: ..., which names theauto_tuneandautoaliases),
replacingUnknown search quality,Invalid custom ef_search value,
Invalid adaptive ...andAdaptive min_ef ... must be <= max_ef. -
The metric aliases are listed once in core.
DistanceMetric::parse_alias
reads them from one table, and VelesQL DDL validation calls it instead of
repeating the list. AWITH (metric = ' L2 ')is therefore trimmed at
validation as it already was at execution. The DDL executor's unknown-metric
message now lists every alias (it named five of the nine). -
One
LIMITceiling in the query pipeline. The union query used its own
MAX_LIMITcopy, andJOIN_ROW_CEILINGis now defined from the same
constant instead of mirroring it by hand. Both values are unchanged (100 000). -
The core rustdoc no longer names a private crate. The observer port is
described by what it is, the hook an out-of-tree policy layer implements. -
The REST
FusionRequestandMultiQuerySearchRequestdocs (and so the
OpenAPI spec) list the strategy aliases and core's weighted defaults. The
FusionRequestfield docs had kept the pre-#1545 0.5 / 0.3 / 0.2, and the
MultiQuerySearchRequestexamples 0.5 and 0.2. -
BREAKING —
AsyncIndexBuilder::enqueuetakes ids, andflush_synca
callback that indexes them.enqueuetook(id, vector)copies and a
flush inserted them, so a point overwritten or deleted between the two
was indexed on its old vector or put back in the graph (see Fixed).
enqueuenow takes the ids (impl IntoIterator<Item = u64>), and
flush_sync(index, index_current)hands the buffered ids, each once, to
index_current, which indexes what storage holds for them and returns
the ids it settled: inserted, already held by the graph as storage holds
them, or no longer stored. The builder drops only those and keeps the
rest queued. It returns aFlushOutcome:indexed, the ids settled plus
the placed ids linked, andkept, the distinct ids it kept queued — a
state the next flush retries, logged as a warning, not an error. Only a
failed link is an error.
Migration: enqueue the ids; pass a closure that inserts what your
vector storage holds for them into the index and returns the ids it
inserted, found indexed already, or found deleted; read the count from
outcome.indexed. -
BREAKING (behaviour):
velesdb-serversends no CORS header unless origins
are configured. 7.1.0 answered every origin with
Access-Control-Allow-Origin: *by default, so any web page open in a
browser that could reach the server — every site a local-first user visits,
for a server on127.0.0.1— could read and, without API keys, write
through it. The default is now same-origin only. A browser front end on
another origin needs its exact origin in[server] cors_origins(or
[cors] allowed_origins);cors_origins = ["*"], written explicitly,
restores allow-all, with the startup warning it always had. Non-browser
clients are unaffected. What the closed default protects is spelt out in
docs/guides/SERVER_SECURITY.md§6: no cross-origin read, and no
cross-origin request with a body — which needed the next entry to be true.
Migration:docs/guides/MIGRATION_v8.0.0.md, section 2. -
BREAKING (behaviour):
POST /collections/{name}/points/streamand
.../points/rawrefuse aContent-Typeother than the one they document
(application/x-ndjson,application/octet-stream) with415. In 7.1.0
they read any body, so a page on another site could write points with a
text/plainrequest a browser sends without a CORS preflight, whatever the
CORS policy. One shared extractor (handlers::content_type) guards both;
every other route that takes a body is a JSON route, which axum already
restricts toapplication/json. The TypeScript SDK sends both types. -
BREAKING (behaviour): a request that changes state from a browser origin
the CORS policy does not allow is refused with403. CORS only hides
answers; in 7.1.0 a page on any site could still fire the body-less
POSTs (flush, analyze, index rebuild, vacuum, compact, locality reorder)
without a preflight. An origin guard now covers every route at once: a
method other thanGET/HEAD/OPTIONSwith anOriginheader passes only
if that origin is the server's own (Origin's authority equalsHost) or
allowed bycors_origins(["*"]: any). Requests withoutOrigin— curl,
the SDKs, backends — are not affected. -
VelesConfig::load_from_path_engine_onlyreads only the environment
variables of engine sections (and of removed sections, so their warning
still fires). In 7.1.0 aVELESDB_SERVER_*orVELESDB_LOGGING_*variable
reachedVelesConfig's ownserver/loggingfields and was validated
there, so an unrelated variable could fail every caller of this loader —
velesdb-serverand thevelesdbCLI, and any Rust caller that loads a
shared config file this way:VELESDB_SERVER_PORT=80stopped
velesdb-server(whose port isVELESDB_PORT) with `server.port must be= 1024
. A caller that relied on such a variable reaching those fields through this loader no longer gets it;load_from_pathis unchanged. The newVelesConfig::load_engine_only_from_envis the same layer with no file, and both binaries use it when they have no config file: in 7.1.0velesdb-serverand thevelesdbCLI then opened on bare defaults and ignoredVELESDB_LIMITS_MAX_COLLECTIONSand every other engine variable.velesdb_core::config::WORKERS_CAPandLOG_LEVELSare now public, so a hosting shell validates[server] workersand log levels by the engine's own rules instead of a copy, and so isVelesConfig::is_engine_table`, which
tells such a shell which top-level tables of a shared file the engine reads. -
BREAKING (behaviour):
velesdb-serveranswers only theHostnames it
is configured for, with421 Misdirected Requestotherwise. A
DNS-rebinding page (a name that resolves to127.0.0.1after the page
loads) is same-origin for the browser, so neither CORS nor the origin guard
stopped it reading and writing a loopback server — 7.1.0's default
deployment. Answered with no configuration:localhost,127.0.0.0/8,
[::1]and the bind host unless it is0.0.0.0/::; new
[server] allowed_hosts(--allowed-hosts,VELESDB_ALLOWED_HOSTS) adds
names,["*"]turns the check off. All methods are checked;/healthand
/readyare exempt for orchestrator probes;X-Forwarded-Hostis never
read. The origin guard's same-origin comparison now normalises default
ports and is trusted only after theHostcheck. One function,
config::apply_browser_policy, installs theHostcheck, the origin guard
and the CORS layer — the only public way to install the origin guard, whose
same-origin rule relies on theHostcheck. Anallowed_hostsentry must
be a bare name or address (a port, scheme, path,*.pattern or trailing
dot is refused at startup) and IP entries are compared as addresses;
--allowed-hosts/VELESDB_ALLOWED_HOSTStrim items, drop empty ones and
treat an empty value as unset; a malformed or unreadableHostis refused;
allowed_hosts = ["*"]is warned about at startup. The421body names the refused host and the three
ways to allow it, and a server bound to0.0.0.0/::with no
allowed_hostssays at startup that it answers only loopback names. A
container reached by its service name needs that name listed:
examples/node-express-rag/docker-compose.ymlnow sets
VELESDB_ALLOWED_HOSTS: velesdb, and the rootdocker-compose.yml, the
Dockerfileand the deployment guides show how. -
BREAKING (Rust API) —
Error::GuardRailsays which limit was crossed.
The variant isGuardRail { kind: GuardRailKind, message: String }, where it
wasGuardRail(String);GuardRailKind(Timeout(TimeoutBudget),Depth,
Cardinality,Memory,Capacity,RateLimit,CircuitOpen) and
TimeoutBudget(Client,Configured) are exported at the crate root,
GuardRailViolation::Timeoutcarries abudget,GuardRailViolation::kind
maps a violation to its kind, andQueryContext::set_client_timeout_ms
applies a request's own budget. The code staysVELES-027and the message
is unchanged. MatchError::GuardRail { .. }, or onkind, where you
matchedError::GuardRail(_). -
REST statuses follow core's caller/engine classification. Every REST
error built from a core error now takes its status from the one mapping
(http_status_for_error, which readsError::is_caller_error):4xxfor
the client's errors,5xxfor the engine's, with itsVELES-NNNcode. A
request the server refuses while reading it, before core is called (a
missing parameter, a malformed filter, …), stays400with no code;
/query,/aggregateand/query/explainkeep422for a client's
failure. Changes from 7.1.0:- routes answering with the canonical mapping (
/collections/{name}/match,
collection, point and graph routes): a depth, cardinality or capacity
ceiling (max_collections,max_dimensions,
max_vectors_per_collection,max_payload_size,
max_perfect_mode_vectors)503→400; the rate limit on/match
503→429. An open breaker, the memory budget, a timeout under the
server's limit and a locked database stay503; /search,/search/ids,/search/hybrid,/search/batch,
/search/multiand/search/multi/idsanswered every failure during
the search400;/search/textanswered it500. Each now answers the
error's canonical status (an engine fault500, a guard rail the engine
could not carry503, a collection deleted while the search ran404,
the client's errors400). A missing collection or an open breaker
found before the search starts was already404/503;- a read the access gate denied answered
400(/search,/search/ids,
/search/hybrid),500(/search/text) or403with no code
(/search/batch,/search/multi,/search/multi/idsand every graph
read,/graph/searchincluded); it now answers the denial error's
canonical status and code on all of them. A scope narrowing an endpoint
cannot apply (batch, multi-query, graph reads) is400VELES-009
(was403); /graph/searchandGET/PUT /graph/nodes/{id}/payloadanswered every
core error500with no code (a query vector of the wrong dimension
included, now400VELES-004); an invalid edge on/graph/edges,
/graph/edges/batchand/relationsgains its code; a core error
setting a point's TTL is no longer always500;- a blocking worker that panicked answered
500with no code, or
VELES-INTERNAL-WORKER-FAILUREoutside theVELES-NNNnamespace on
search; it is now500VELES-013, and on a search it is counted by the
circuit breaker as the engine failure it is; - a REST search's
timeout_msat or above the server'stimeout_ms
limit answers503(was408): the search ran as long as the server's
own limit. A budget strictly tighter stays408; /query,/aggregateand/query/explainanswered every execution
failure422: a storage, I/O or internal fault is now500; a timeout
under the server's limit, the memory budget, an open breaker and a
locked database are503. The client's own failures stay422, its
WITH (timeout_ms)running out included, keeping theVELESQL_*code.
- routes answering with the canonical mapping (
-
A request's own
timeout_mscan shorten the server's time limit, never
lengthen it.VelesQLWITH (timeout_ms = N)and a REST search's
timeout_msresolve the same way, throughQueryLimits::client_timeout_ms:
the budget ismin(N, QueryLimits::timeout_ms)when the server sets a
limit, and0keeps that limit. 7.1.0 letWITH (timeout_ms = N)replace
the limit, longer or0(disabled) included, and let a REST search's
timeout_msrun past the limit, or time out at once (408) with0.
With no server limit,Napplies as before. -
An unsupported MATCH
ORDER BYis refused at validation, withV015
instead ofVELES-018.ORDER BY COUNT(*),ORDER BY MAX(year), a bare
pattern alias (ORDER BY d), aRETURNalias (RETURN d.year AS y ORDER BY y) and an unqualified property (ORDER BY year) passed
QueryValidator::validateand failed after traversal withVELES-018
(GraphNotSupported; REST/query422 VELESQL_EXECUTION_ERROR,
/collections/{name}/match400 VELES-018). They are now refused before
traversal withV015(UnsupportedMatchOrderBy):VELES-010from the
Rust API and/collections/{name}/match(400),422 VELESQL_VALIDATION_ERRORfrom/query, and an error from the WASM
executor.QueryValidator::validate_match_clauseholds the one rule set,
andmatch_query_ordered,execute_matchand
execute_match_with_similarityrun it on the clause they receive, so the
same MATCH gets the same answer on every surface. A test runs every form
through the validator and the executor and fails if their verdicts differ;
rejected_casesR005–R009 lock the refusal in both engines. -
BREAKING (behavior) — the TypeScript SDK refuses a filter that is not
core-shaped.normalizeFilter, now called on every filter before it
leaves the SDK (REST search, batch, text, hybrid, multi-query, sparse,
ids and scroll; the WASM filtered search and its batch pre-loop), throws
ValidationErrorunless the filter is{ condition: {...} }, the shape of
velesdb_core::filter::Filter. A flat map such as{ category: 'tech' }
used to reach velesdb-server, which answered400 missing field condition,
or velesdb-wasm, whosematches_filterread a filter withoutcondition
as "no filter" in 7.1.0 and returned unfiltered neighbours.nullstill means no
filter. Migration: build filters withf.*. -
BREAKING (behavior) —
recallRecent(collection, since)is exclusive in
the TypeScript SDK, returning events withtimestamp > sinceas core's
TemporalIndex::recentdoes (it skipsts <= since); it was>=. The
shared fixture'ssincecase now pinsgt.recallOlderThanalready
matched core's strictolder_than. Migration: passsince - 1to keep an
event stamped exactlysince. -
BREAKING (behavior) —
createGraphCollection({ schemaMode: 'strict' })
creates a strict graph (TypeScript SDK). The SDK sentschema_mode, a
field velesdb-server never read, so every graph collection came out
schemaless; it now sends core'sgraph_schema. The newnodeTypesand
edgeTypesoptions mirror core'sNodeTypeandEdgeType(properties
typed byValueType), so a strict graph can declare what it accepts;
declaring types on a schemaless graph, where core ignores them, throws
ValidationError. The wire shape is pinned by
conformance/graph_schema_wire_cases.json:
crates/velesdb-core/tests/graph_schema_wire_conformance.rsparses each case
asGraphSchema, round-trips it and checks the node types and edges it
accepts and refuses, and the SDK test checks it sends each case's
graph_schemafor its config. The WASM backend still refuses graph
collections withNOT_SUPPORTED. Migration: declare the types, or drop
schemaMode: 'strict'to keep the old schemaless behaviour. -
BREAKING (behavior) — velesdb-wasm filters with core.
VectorStore.search_with_filterparses the filter as
velesdb_core::Filter(filter::parse_filter) and evaluates it with
Filter::matches; the crate's own evaluator, which knew onlyeq,neq,
the four comparisons,and,orandnot, is deleted. Every other core
condition kind (in,contains,is_null,is_not_null,like,
ilike, the threearray_contains*,geo_distance,geo_bbox) matched
nothing in the browser in 7.1.0 while REST returned hits; they now select what
core selects, checked against the shared
conformance/filter_wire_cases.json, which now holds one case per kind and
is run by both velesdb-core and velesdb-wasm. A filter core cannot parse
throws core's message (Invalid filter: missing field \condition`, for one): a flat{ "category": "tech" }read as "no filter" in 7.1.0 and returned every point, and a rawsearch_with_filter(query, k, null)now throws instead of searching unfiltered — callsearch` for an unfiltered
search. -
BREAKING — the TypeScript SDK requires Node.js 22 or later
(engines.node, was>=18.0.0). CI builds and tests the SDK on Node 22
only, its test toolchain needs 22.12+, and Node 18 and 20 are end-of-life.
The SDK README,docs/guides/INSTALL_OPTIONS.md,CONTRIBUTING.mdand
examples/node-express-rag(engines, Dockerfile) now state it. The
velesdb-wasm README states the floor of its own binary instead, Node 18+
(WebAssembly SIMD, bulk memory, non-trapping float-to-int and reference
types), where it claimed 20+. -
BREAKING (behavior) —
velesdb-wasmreturnsnullfor an absent value,
where 7.1.0 returnedundefined. It follows from the fix below: the
binding now serializes asJSON.parsewould, andJSON.stringifykeeps a
nullwhere it drops anundefined. Affected: a missingpayloadin
hybrid_searchandsearch_with_filterrecords,query()'sgraphScore
andcolumnData,GraphNode.to_json()'svector, and a missing alias in
ParsedQuery.matchReturnItems. A check written=== undefinedmust become
== null. See the migration guide. -
BREAKING (TypeScript SDK types) —
textSearchreturns
TextSearchResult[], whosescoreisnumber | null. velesdb-wasm's
text search is an unranked, case-insensitive substring match, so its
records carryscore: null(in 7.1.0 the SDK'stextSearchthrew on any
match instead, see Fixed). REST still
returns the BM25 score. A caller that sorts or thresholds on it must handle
null. -
BREAKING (behavior) — LlamaIndex
GraphLoaderwith a graph collection
writes its nodes there, at VelesDB point ids. In 7.1.0add_nodeand
load_from_nodeswrote every node into the vector store's collection — a
metadata-only point, or a vector point whenadd_nodegot a vector — at the
given id, or atstable_hash_id(f"{node_label}:{node.node_id}")for
load_from_nodes, while its edges never persisted (see Fixed). In 8.0.0,
with a graph collection, a node is upserted into that graph collection only,
labelled through_labels(fromlabel, which a_labelskey in the
node's metadata no longer overrides); a vector is dropped when the graph
collection has no embeddings.load_from_nodesgives it the id
VelesDBVectorStorestores the node under,stable_hash_id(node.node_id),
soGraphRetrieverexpands from a vector hit straight into the loaded
graph, as LangChain's retriever expands from the point id (_int_id).
Without a graph collection nothing changes. Migration: the points the
7.1.0 loader wrote into the vector store's collection are no longer read.
They are metadata-only, so search andgetnever return them, but they
stay stored and counted; delete them with the snippet in
MIGRATION_v8.0.0.md(section 4). -
BREAKING (behavior) — LangChain and LlamaIndex
GraphRetrieversearch
seed_kseeds. In 7.1.0 they searchedexpand_k(default 10); in 8.0.0
they searchseed_k(default 3), as documented, and fill the rest of
expand_kwith graph neighbours (see Fixed). A query whose seeds have no
neighbour now returns at mostseed_kdocuments where it returned up to
expand_k. Migration: raiseseed_kto get more vector hits. -
BREAKING (behavior) — LlamaIndex
GraphRetrieverneighbours carry
score=None. In 7.1.0 a node reached through the graph hadscore=0.5,
a value no search produced, which a downstream score threshold or re-ranker
read as a similarity.NodeWithScore.scoreis optional, andNonenow says
the node was not scored, as LangChain's neighbours carry no
relevance_score.GraphQARetrieversorts unscored nodes after the scored
ones of the same depth. Migration: a postprocessor that compares scores
must handleNone(keep or drop graph-reached nodes on purpose). -
BREAKING (behavior) — LlamaIndex
GraphRetrieverexpands a seed from its
point id only. In 7.1.0 it took the seed's id from anid,doc_idor
node_idkey of its metadata when one held an integer, elseint()of a
numeric node id, else the hash — whileVelesDBVectorStorealways stores a
node under the hash, so a node id like"1"(or such a metadata key)
expanded from a point that does not exist and found nothing. Store, loader
and retriever now share one derivation,point_id(node_id)=
stable_hash_id(node_id). Migration: edges added by hand must use
stable_hash_id(node.node_id)assource/target. -
BREAKING (behavior) — a
GROUP BYORDER BY that names no column of the
grouped rows is refused withV016(OrderByNotInResult). In 7.1.0 core
accepted it and sorted on nothing:... GROUP BY category ORDER BY category, yearignoredyear, and an aggregate the SELECT does not list
(ORDER BY SUM(year)besideCOUNT(*)),similarity()or an arithmetic
expression left the groups unsorted. WASM 7.1.0 did the same, except that
it sorted on a selected column outsideGROUP BY(SELECT category, title, COUNT(*) ... GROUP BY category ORDER BY title) by each group's first
title, a column core's rows never carried. 8.0.0 refuses all of these at
validation, in both engines. A vectorGROUP BY(NEARin WHERE) is
refused only for an ORDER BY aggregate it does not select; it still orders
bysimilarity(), arithmetic and its payload fields. Migration: order by a
GROUP BYcolumn or a SELECT aggregate — add the column toGROUP BYor
the aggregate to the SELECT list (see
MIGRATION_v8.0.0.md, section 2). -
BREAKING (behavior) — an aggregate query whose output columns share a
name is refused withV017(DuplicateOutputColumn). In 7.1.0 the
later column silently overwrote the earlier one:SELECT category, COUNT(*) FROM docs GROUP BY category, countreturned the count under
countand lost thecountgroup column, andCOUNT(*) AS n, SUM(year) AS nreturned one of the two. 8.0.0 refuses it at validation in every
engine, naming the column. A selected column counts even outsideGROUP BY(SELECT title, COUNT(*) AS title ... GROUP BY category), and in a
vectorGROUP BYan aggregate may not be namedidorscore, which
would shadow the group's own id and score underORDER BY; in a SQL
GROUP BYAVG(rating) AS scorekeeps working. Migration: alias the
aggregate whose name repeats another column. -
BREAKING (behavior) — a HAVING the query cannot evaluate is refused with
V018(InvalidHaving), in every engine. A threshold that is neither a
number norNULL(HAVING COUNT(*) > 'many',> TRUE, or a parameter
bound to a string) matched no group in 7.1.0, silently; it is refused at
validation, or, for a bound parameter or scalar subquery, when the query
runs, and the error shows the value as JSON ("many"). ANULL
threshold —= NULL, a parameter bound tonull, a subquery returning no
row — is SQL's UNKNOWN: it matches no group and is no error.HAVING
withoutGROUP BYwas refused by core with an unnumbered error and
filtered the single implicit group in WASM; both now refuse it with
V018. Migration: compare aggregates with numbers, and addGROUP BY(or
move the test to WHERE). -
BREAKING (behavior) — SQL
GROUP BYresults no longer depend on scan
order. In 7.1.0 core grouped in hash order and WASM in scan order, so the
same query could return different rows:- without ORDER BY, groups come by
GROUP BYkey, ascending,NULLlast
(core returned hash order, WASM first-seen order); - groups an ORDER BY ties come by
GROUP BYkey too, soORDER BY n DESC LIMIT 3keeps the same three groups everywhere (core used an unstable
sort with no tie-break); FIRST(col)reads the group's row with the smallest point id (core
returnednull, WASM the first row scanned); in a vectorGROUP BYit
reads the most similar row, the smallest id among equally similar ones
(the first one scanned in 7.1.0);- numbers equal by exact value are one group:
1and1.0,0.0and
-0.0,2^53as an integer and as a float (separate groups in 7.1.0); - groups whose keys compare equal (two arrays) come by their best row's
id; - rows are aggregated in point id order, so a float
SUMorAVGis the
same on every run (core followed storage order); above 10,000 rows an
aggregate withoutGROUP BYsums per 1,000-row chunk in core, so its
last digits can differ from WASM's.
Migration: add an ORDER BY to pin another order (see
MIGRATION_v8.0.0.md, section 9).
- without ORDER BY, groups come by
-
BREAKING (behavior) —
idis the point id in every VelesQL clause, in
core as in WASM. Core 7.1.0 read a payload field namedidin a
SELECT's WHERE (plain or aggregate), ORDER BY (plain or arithmetic),
GROUP BY and aggregates, a window function's ORDER BY and PARTITION BY,
in a JOIN key read from the payload first, and in a MATCH's WHERE (=andIN), ORDER BY and RETURN (n.id,r.id),
while its projection, itsUPDATE/DELETEWHERE, and every WASM
clause, read the point id. For a payload carrying its ownid, all of
these now read the point (node, relationship) id;RETURN nprojects the
node id undern.id, a node without payload included; a property index on(Label, "id")no longer
answers a MATCHn.idpredicate. Without such a payload field,GROUP BY idgrouped every row into oneNULLgroup andWHERE id < 100matched
nothing.id = nandid IN (...)now visit those points directly
(a SELECT, an aggregate,NEAR … AND id IN (...), UPDATE), and EXPLAIN
reports aPointIdLookup.
VelesQL cannot reach a payload field namedid:SELECT *already
returned the point id under that name. Migration: rename the payload field
(doc_id) to filter or sort on it in VelesQL; a JSON filter (REST search
and scrollfilter,
{"condition": {"type": "eq", "field": "id", "value": 3}}) still reads the
payload field (see
MIGRATION_v8.0.0.md, section 9). -
Security — an observer scope on
idnarrows Python reads. A scope
such asid != 2was tested against the payload alone in the Python
binding's post-filtered reads (get, batch, sparse and hybrid-sparse
search, graph embedding search): with no payloadid,id != 2,NOT INorIS NULLmatched every row, and the scope widened instead of
narrowing. Every filter is now tested against the point, its id included. -
BREAKING (behavior) — numbers compare by exact value, with no
tolerance. 7.1.0 compared numbers several ways:- WHERE
=/!=/IN, a JSON filter (REST search and scroll,
Python, every binding) and the WASM executor took two numbers less than
f64::EPSILONapart for equal, and compared an integer as anf64, so
id = 9007199254740993matched9007199254740992andn > 9007199254740992.0missedn = 9007199254740993; - a MATCH WHERE took two floats less than
0.001apart for equal, and
compared an integer literal with a float property not at all (d.x > 3
missed3.5); - HAVING allowed a relative
f64::EPSILON; - a secondary index and the payload mirror compared exactly, so a query
could return different rows once an index or the mirror served it, and
anINlist past 16 values another result than a shorter one.
Every clause, path and engine now compares numbers by their mathematical
value: an integer and a float exactly, beyond 2^53 included;-0.0
equal to0; no tolerance, so0.30000000000000004(0.1 + 0.2) is not
0.3.=,<and>are a trichotomy and=is transitive, so a scan,
an index, the mirror and anyINlist return the same rows. Migration:
compare a computed float with a range,x BETWEEN 0.29999 AND 0.30001
(seeMIGRATION_v8.0.0.md, section 9). - WHERE
-
SELECT DISTINCT id, a window'sPARTITION BYandDISTINCTkey
numbers asGROUP BYdoes.DISTINCT idkeyed on a payload field named
idand collapsed different points; it reads the point id.PARTITION BY xandDISTINCT xkeyed numbers by their JSON text, so1and1.0, or
0and-0.0, fell in different partitions or rows; they are one, as in
GROUP BY. In core and WASM. -
BREAKING (Rust API):
Condition::In'svaluesis anInValues, not
aVec<serde_json::Value>. The newtype sorts and de-duplicates at
construction — everyCondition::In, however built (Condition::is_in,
VelesQLconversion,Deserialize, a literal), carries a sorted list,
so the >16-element binary search can never see an unsorted one. The wire
format is unchanged. A struct literal builds it withvalues: vec.into();
reads go through the[Value]deref.id = 5.0andid IN (5.0)now
take the point-id lookup, likeid = 5. -
Aggregates see every point of a metadata collection.
COUNT(*),
every other aggregate andGROUP BYover a metadata-only collection
scanned vector storage only and found no row; they scan the points a
SELECT does. -
BREAKING (behavior) — a point without payload is tested with a
null
payload on every filtered path. REST search and scroll filters, text
and hybrid search, batch search and MATCH dropped it outright, while a
SELECT kept it when the filter matched anullpayload (IS NULL,
!=):WHERE id = 5and{"type": "is_null", ...}now match it
everywhere. -
BREAKING (behavior) — UPDATE and JOIN reach a point without payload.
AnUPDATEwhose WHERE was notid = n/id IN (...), and a JOIN
scanning its left collection, visited only the points that carry a
payload:UPDATE docs SET kind = 'x' WHERE kind IS NULLskipped every
point without one. They now visit every point, as a SELECT does. -
BREAKING (behavior) — a payload index never answers past 2^53. A
secondary index keys a number by itsf64, which cannot tell2^53from
2^53 + 1. An ORDER BY over an indexed field, an indexed JOIN key and an
indexed range or equality now fall back to the exact path when a key or
literal reaches 2^53, so the index no longer ties, drops or adds such a
row. An index named@velesql.point_id, the name VelesQL readsid
through, is refused withError::Config; it would have answeredid
ranges from payloads. -
BREAKING (behavior) —
MINandMAXkeep an integer exact. They
returned anf64:MAX(id)was33.0, andMAX(n)over2^53 + 1was
2^53. A minimum or maximum of integers is now that integer (33), in
the result and inHAVING. A float column still returns a float. Among
values equal by value the integer wins (3over3.0,0over-0.0),
then0.0over-0.0, whichever row comes first. -
BREAKING (behavior) —
Collection::execute_aggregateand
Database::execute_aggregaterefuse a vectorGROUP BY(NEAR+
GROUP BY) withError::Query. It grouped every point of the collection,
with no search score and no group budget; run it withexecute_query,
which groups the search results. -
BREAKING (behavior) — ORDER BY compares an integer and a float
exactly. 7.1.0 compared them asf64, so2^53 + 1equalled the float
2^53, which is less than2^53 + 1as an integer: the order was not
transitive and a sort over such values could come out inconsistent.
Numbers now compare by exact value (-0.0equal to0), a total order. -
BREAKING (behavior) — aggregates count and read columns as SQL does,
with or withoutGROUP BY.COUNT(col)counts non-null values of any type: core 7.1.0 counted
numeric values only, soCOUNT(category)over a string column returned
0.SUM,AVG,MINandMAXstill read numeric values only.HAVING COUNT(col)over a column no row of a group holds compares0;
7.1.0 compared nothing and dropped the group.HAVING AVG(score)in a SQLGROUP BYaverages the payload field
score, as the same aggregate in the SELECT does; 7.1.0 dropped every
group.HAVING a OR b AND creads asa OR (b AND c), SQL's precedence, in
every engine; 7.1.0 read it left to right,(a OR b) AND c(write that
asa AND c OR b AND c).
Migration: none for a query that did not depend on the old reading (see
MIGRATION_v8.0.0.md, section 9).
-
BREAKING (behavior) —
velesdb-wasmcaps a SQLGROUP BYat
WITH (max_groups = N)groups, 10,000 by default, as core does; 7.1.0
WASM grouped without bound. A query past the budget fails withToo many groups (limit: N); raisemax_groups(1,000,000 at most) to allow more. -
BREAKING (API) —
AggregateResultgainsmin_valuesand
max_values(HashMap<String, serde_json::Number>), the exactMIN
andMAXof each column;minsandmaxsstill hold theirf64. A
struct literal must now set them (..Default::default()); a serialized
result without them reads them as empty.Aggregator::process_batch
keeps an infinite extreme inmins/maxs(no JSON number holds it, so
min_values/max_valueshave no entry) and skips a NaN as no value. -
BREAKING (API) —
all_ids()lists every point.Collection,
VectorCollectionandMetadataCollection::all_ids,
GraphCollection::all_node_ids, Python'sCollection.all_ids()and the
mobile binding'sall_ids()listed only the points carrying a payload,
in storage order; they list every point, ascending, asall_point_ids
does. The CLI REPL's.export,.sampleand.browseread it, so
.exportno longer writes a file without the points that have no
payload. -
BREAKING (API) —
AggregateResult::countsandAggregator::process_value
/process_batchcount non-null values of any type forCOUNT(column),
where they counted numeric values only;sums,avgs,minsandmaxs
still hold numeric values only. -
BREAKING (behavior) —
velesdb-wasmnames an unaliased aggregate as
core does. 7.1.0 WASM returnedCOUNT(*)ascount(*)andSUM(price)
assum(price); 8.0.0 returnscountandsum_price, the names core and
every other binding return (<function>_<column>,<function>_score),
through one derivation,AggregateFunction::result_key. A core vector
GROUP BY(server, Python, Node, every binding) uses it too: an unaliased
COUNT(*)there iscount, notCount_*, andCOUNT(score),
SUM(score),MIN(score)andFIRST(score)arecount_score,
sum_score,min_scoreandfirst_score, notagg_score. The old keys
heldnull(7.1.0 computed none of those aggregates there). Migration:
read the new key, or alias the aggregate (COUNT(*) AS n), the key in
every version (see
MIGRATION_v8.0.0.md, sections 7 and
9). -
BREAKING (behavior) —
velesdb-wasmgroups as core does. Its GROUP BY
and aggregates now run through core's engine-shared grouping
(velesql::Grouper), so three 7.1.0 WASM behaviours change: a selected
column outsideGROUP BYis no longer added to each group row (core never
output it);MINandMAXread numeric values, asSUMandAVGdo,
and returnnullfor a group of strings, asSUMnow does over no
numeric value (it returned0); andscoreas an aggregate
argument outside a vectorGROUP BYis the payload field of that name, as
in core, not the row's search score. Migration: add the column to
GROUP BY, and keepMIN/MAXfor numeric columns (see
MIGRATION_v8.0.0.md, section 7). -
BREAKING (behavior) — ORDER BY sorts
nulllast underASCand first
underDESC, in every engine and ORDER BY path. In 7.1.0 core sorted a
nullor missing key before every value underASC(and after under
DESC) while WASM sorted it last, soSELECT * FROM docs ORDER BY tag
returned rows without atagfirst in core and last in WASM, and aNULL
group moved the same way. WASM also compared a number with a string, or two
booleans, as equal. 8.0.0 uses one comparator,
velesql::compare_json_values, for rows,GROUP BYgroups, vectorGROUP BYresults,MATCHand window functions:nulllast ascending
(PostgreSQL's default), then booleans, numbers, strings, arrays, objects.
Migration: none for WASM row order; in core, a query that relied on
nullkeys coming first underASCreads them last (see
MIGRATION_v8.0.0.md, section 9).
Removed
-
BREAKING — Python: the deprecated search entry points. The native
Collection.search(...)keyword method (deprecated since 1.15.0, with a
DeprecationWarning) is gone; thevelesdb.Collection.search(vector, top_k, filter, sparse_vector, sparse_index_name)shorthand stays, no longer warns,
and goes throughsearch_request.SearchOptions(top_k=...), itstop_k
property and.with_top_k()(aliases ofksince 5.1.0, #1921) are
removed. Migration:top_k=becomesk=and.with_top_k(becomes
.with_k(. -
BREAKING — Python:
Collection.get_graph_store(). It returned a
standalone in-memory graph unrelated to the collection, with a
DeprecationWarningsaying so. Migration:Database.get_graph_collection()
for a persistent graph, orvelesdb.GraphStore()for an in-memory one. -
scripts/release.sh, an unreferenced script that bumped every
Cargo.tomland pushedmainand a tag itself, outsidebump_version.py
and thetag-release.ymlworkflow that releases go through. -
BREAKING (Rust API) —
velesdb-coreitems deprecated one major or more
ago. Each replacement already exists and does what the removed item did:GraphCollection::store_node_payload(deprecated in 1.6.0): call
GraphCollection::upsert_node_payload, same arguments.OperationalMetrics::shared(1.13.2): callOperationalMetrics::new_arc;
both returned a freshArc, never a shared one.VelesConfig::effective_ef_search(5.2.0): the engine resolves[search]
throughSearchConfig::resolved_quality(#2087,persistencefeature),
which returns theSearchQualityto read rather than ausize; the
removed helper returnedusize::MAXforperfect(#2238). Without the
persistencefeature, the ef_search it computed is
search.ef_search.unwrap_or_else(|| search.default_mode.ef_search()).velesql::match_planner::CollectionStats(5.2.0): the type is
velesql::match_planner::MatchGraphStats, same fields.- The update-check entry points deprecated in 7.0.0:
check_for_updates
andspawn_update_checkbecomecheck_for_updates_withand
spawn_update_check_with, which take an
UpdateCheckContext::new(entry, edition)(add.with_data_dir(dir)to
keep a data directory) and report the calling binary instead of
"library".compute_instance_hashhas no replacement: the update check
derives its install hash itself, and the helper only ever read a stored
id, returning a fresh random hash when there was none.
Database::notify_upsertandDatabase::notify_query(deprecated in
3.9.0) stay in 8.0: the server's REST bulk upsert, NDJSON stream, search
pipeline, batch search and aggregation routes do not run through the core
layer that fireson_upsert/on_query, so these calls are their only
observer telemetry. -
BREAKING (Rust API) —
velesdb_core::TraversalMetrics(also
velesdb_core::metrics::TraversalMetrics). Nothing in the engine fed it,
and itsexport_prometheusstill wrote the process-wide
velesdb_traversal_*families the server stopped writing in #2548. Graph
traversal counts are per collection and fed by core:
GraphCollection::metrics(), andDatabase::graph_metrics_prometheus()
for thevelesdb_graph_traversal*families. -
BREAKING (velesdb-migrate) —
PineconeConfig::environment(deprecated
in 1.12.0). Pinecone serverless discovers the index host through
GET /indexes/{name}, so the field was never read. A YAML config that still
setsenvironment:keeps loading, the key ignored like any unknown key;
Rust code that builds aPineconeConfigdrops the field. The shipped
template and example no longer write it. -
Dead internal code, with no public API change. In
velesdb-core: the
graph index advisor (IndexAdvisor,QueryPatternTracker) that nothing
read, so aMATCHno longer takes a write lock to record its pattern; a
second BFS / DFS implementation only tests called (the tests now run the
traverse_*_configpath every binding uses); the never-called
deferred-sync bulk upsert and thefsyncflag it alone set tofalse; the
Collectionsparse-search wrappers,estimate_column_selectivity, the trigram and generic GPU
backend selectors, two never-incremented HNSW safety counters, and other
unreachable helpers. Invelesdb-server, an unused query-type detector and
thereqwestdev-dependency; invelesdb-cli,LicenseInfo::has_feature;
invelesdb-wasm, a test-only accessor; invelesdb-python, the
pyo3-build-configbuild-dependency with no build script to use it. -
BREAKING —
Filter::matchesandfilter::Condition::matches. They
tested a payload alone, so a filter converted from VelesQL could not read
itsidcolumn, and an observer scope such asid != 2matched every row
(fail-open). Callmatches_point(id, payload). A JSON filter naming the
internal point-id field@velesql.point_idis refused when deserialized. -
BREAKING —
AggregateResult::to_json. It named columns its own way
(countonly when non-zero, noCOUNT(column)key), which no engine
returned. Read thecount,counts,sums,avgs,minsandmaxs
fields, or run the query throughDatabase::execute_aggregate. -
BREAKING — the
Parallelexecution strategy, on SELECT and on MATCH.
It ran the GraphFirst pass and the VectorFirst search and merged them, but
GraphFirst is exact over every row its filter or pattern admits: the
VectorFirst rows the union added were rows it already returned, so the
plan only cost both legs (on 20 000 768-d points, aParallelSELECT's
p50 was 40–48 ms against 25–31 ms for GraphFirst). The planners now pick
GraphFirstin the band where they pickedParallel, and EXPLAIN shows
noParallelplan.ExecutionStrategy::Parallel,
MatchExecutionStrategy::Paralleland
QueryPlanner::with_thresholds(graph_first, vector_first)are removed.
One visible difference remains onMATCH: the union gave a row the
VectorFirst leg's similarity score (andRETURN similarity()its
projection); aGraphFirstrow carries none — itsscoreis the
depth-derived value every graph-reached row gets.VectorFirstrows
still carry theirs. The union also ransort_match_results_by_score
over its merged rows, and that ordering leaves with it: aParallel
MATCH withoutORDER BYreturned its first N best-scored, and the same
MATCH now returns GraphFirst's first N in traversal order.ORDER BY
sorts as before.
Migration: drop theParallelarm of amatchon either enum; build a
planner withQueryPlanner::with_vector_first_threshold(vector_first). -
BREAKING —
AsyncIndexBuilder's test-only calls.search_buffer,
drain_buffer,buffer_len,is_buildingandtrigger_build_asynchad
no caller in core, the server or any binding: only tests reached them.
trigger_build_asyncspawned a thread that drained one of the builder's
two queues and could report a build done with slots still unlinked;
drain_buffer,buffer_lenandis_buildingserved it or its tests, and
search_bufferwas a scan no search called: searches now score the
builder's pending points themselves (see Fixed).enqueue,flush_sync
(both changed, see Changed) andmerge_thresholdstay.
Migration:VectorCollection::flushindexes what the builder holds;
searching the collection already finds it (see Fixed). -
BREAKING —
velesql::match_clause, a second MATCH parser no query used.
parse_match_clause,parse_node_patternandparse_relationship_pattern
scanned strings beside the pest grammar that every query goes through; only
their own tests and a benchmark called them, and they had drifted from it
(they read a named pathp = (a)-->(b), which the grammar refuses, and an
unbounded[*], which validation refuses). The behaviours their tests
checked that the grammar shares are now tested againstParser::parse, and
match_parser_benchmarktimes whole statements under new ids.
Migration:velesql::Parser::parse("MATCH ...")?.match_clause. -
The composite graph index (
CompositeGraphIndex,
CompositeIndexManager). No path created one, so the manager every
collection carried stayed empty: every labelled node payload write took its
write lock to find that out, and every MATCH with two equality predicates on one alias
asked it for a lookup it could never answer. Both calls are gone, and the
MATCH pre-filter now intersects every indexed predicate rather than skipping
the equality ones a composite lookup was assumed to cover. Crate-private;
no public API changes. -
BREAKING — Python:
velesdb.SearchResult. The class had no methods and
no call returned or built one: every search returns dicts (SearchResultDict
in the stub). It is no longer importable, listed in__all__, or stubbed.
Migration: drop the import; read results as dicts. -
Crate-private code that
#[allow(dead_code)]kept alive. With the
allows lifted, the compiler named what nothing called:
NativeHnswInner::{search_neighbours, is_empty, metric},
ShardedIndex::{clear, reserve}andNativeHnswIndex's unreadparams
field are removed; helpers only tests call (Collection::create,
create_with_hnsw_params,sparse_indexes, the range index's accessors, the HNSW
safety-counter snapshot,BitVecVisited::contains,ShardedIndex::is_empty,
the trigram SIMD level'sname, andLayer::get_neighborsoutsidegpu)
are now#[cfg(test)], so a production build no longer compiles them. Two allows
were stale —SnapshotBuilder::emptyandmerge_batch_postingshave
production callers — and are dropped.RaBitQVectorStore, unused without
persistence, is now compiled only with it instead of carrying two
allows, andvelesdb-wasm'sdelete_edges_whereloses a stale one (the
memory store calls it). No public API changes. -
BREAKING — the delta buffer's and the deferred indexer's own searches
and merges.DeltaBuffer::search,DeferredIndexer::{search, merge_with_hnsw, swap_and_drain, drain_all}and
collection::streaming::{merge_with_delta, merge_with_delta_scored}had
no caller outside tests: a search merges what the deferred indexer holds
through the collection (see Fixed), and a flush indexes it. They scored
the copy each buffer took, where a search now scores what storage holds.
Migration: search the collection;VectorCollection::flushindexes
what the deferred indexer holds. -
BREAKING — the collection-level delta buffer, which nothing activated.
Every collection carried aDeltaBuffermeant to hold writes while an
HNSW rebuild ran, but no rebuild path ever activated it: writes checked
it, deletes purged it, every flush drained it and every search read it,
always empty. It is removed with what only fed or reported it:
VectorCollection::push_to_delta_if_activeand
VectorCollection::is_delta_active(alwaysfalse), Python's
Collection.is_delta_active(), and the stream ingester's and the NDJSON
stream route's copies into it.DeltaBufferstays, as the deferred
indexer's write buffer, without the activation state machine it kept for
rebuilds:activate,try_activate,deactivate_and_drain,
is_active,is_searchable,statsandActivateErrorare removed, and
it accepts every write.
Migration: drop the calls;is_delta_active()returnedfalseon every
collection. ADeltaBufferyou build yourself needs no activation; use
len/is_emptyfor its size. -
The edge property index (
EdgePropertyIndex). Every edge write fed it
(and cloned the edge's properties to do so), every open rebuilt it from the
edge snapshot and again from the edge WAL replay, and nothing ever read it
or removed an entry from it when an edge went away. It, its rebuilds and
the edge WAL replay's per-entry callback that only it used are removed.
Crate-private and never persisted; no public API or format changes. -
BREAKING —
CollectionConfig.description(TypeScript SDK). Core has no
collection description:CreateCollectionRequesthas no such field, so the
REST server dropped it silently, and the WASM backend stored it where
nothing read it. It is gone from the type, the REST request and
capabilities().collectionConfig. -
BREAKING —
HnswParams.storageMode(TypeScript SDK). Nothing read it;
the REST request takesCollectionConfig.storageModeonly, which is the
field to set. -
BREAKING —
SearchOptions.includeVectorsandSearchResult.vector
(TypeScript SDK). No search returns vectors on either backend: core's
SearchRequesthas noinclude_vectorsandSearchResultResponseis id,
score and payload. REST sent the flag to be dropped (while
REST_CAPABILITIES.includeVectorsclaimedtrue); WASM refused it. Read a
vector withget(). -
BREAKING —
QueryOptionsandquery()'soptionsargument (TypeScript
SDK). Core'sQueryRequest(/query,/aggregate) isquery,
paramsandcollection: REST senttimeout_msandstreamto be
dropped, WASM refused them. The server's query timeout is its guard-rails
timeoutMs(updateGuardRails). -
BREAKING — the
includeVectorsandqueryOptionscapabilities, which
described the two removed options, leaveCapabilityMap,
CAPABILITY_LIST_UNIVERSESandQUERY_OPTION_NAMESwith them. -
BREAKING —
TraverseRequest.cursorandTraverseResponse.nextCursor
(TypeScript SDK). velesdb-server'sTraverseRequestand
TraverseResponsehave no cursor (the WASM backend has no traversal), so
the request field was dropped and the response field alwaysundefined.
hasMoresays the traversal stopped atlimit.
Fixed
-
A huge
batch_sizeno longer aborts the process.
Collection::scroll_batchreservedbatch_sizepoints up front, and the
streaming ingester's drain task reserved its configuredbatch_size, so a
caller'susize::MAX(or any size past memory) overflowed the capacity and
aborted the process. The Pythonscrolliterator andscroll_batch, the
Tauriscroll_batchand the CLI pass a scroll size through (REST caps one
at 10 000); REST/stream/enableand the Python, Tauri and mobile
enable_streamingpass a streamingbatch_sizethrough, the mobile one
turning au64pastusizeintousize::MAX. Both now grow with the
points they actually hold. The sparse, hybrid and filtered search
resolvers and the GPU multi-probe merge are sized the same way, by the hits
found rather than bykorlimit, which the searches already bounded.
An HNSW index reserved its base layer and its id mappings at
max_elementsbefore its first insert; RESThnsw_max_elementsand the
PythonHnswOptions.max_elementspass it through, and on an SQ8 or RaBitQ
collection (whose own arena is capped)usize::MAXaborted the collection
create.max_elementsstays the capacity hint it is documented as: the
up-front reservation is capped at 2 000 000 slots, above every
for_dataset_sizepreset, and the index grows past it as nodes arrive. -
BREAKING (Rust API) —
enable_streamingrefuses a config the pipeline
cannot run with. A streamingbuffer_sizeof 0 or past
tokio::sync::Semaphore::MAX_PERMITS, or aflush_interval_msof 0, hit an
assertion in tokio's channel or interval and aborted the process (release
builds abort on panic), reachable from a REST request. The new
StreamingConfig::validaterefuses them, andCollection::enable_streaming
andVectorCollection::enable_streamingnow returnResult<()>, with
Error::Config(VELES-009) and the current ingester left running.
REST/stream/enableanswers400, Python raisesValueError, Tauri and
mobile reject withVELES-009. Migration: handle or?the result. -
Stored floats read back bit for bit. Payloads, edge-WAL records and
edge-snapshot properties are JSON, andserde_json's default parser returns
some doubles one ULP off (127.75015752618465reads back as
127.75015752618464): a float written to a payload or an edge property
could come back different after a reopen, and an exact-equality filter on
it stop matching.velesdb-corenow enablesserde_json's
float_roundtripfeature, which parses every float exactly. This covers
reads: a value an earlier version already read off by one ULP and wrote
back (a snapshot, a compaction, a read-modify-write) keeps that value.
Cargo unifies the feature across a build, so everyserde_jsonfloat parse
in a binary that links core pays it, the application's own and other
dependencies' included, REST request bodies too: measured on a
768-dimension query body (15.5 KB, release build), parsing went from
38-40 µs to 56-63 µs into a typedVec<f32>(about +20 µs per vector,
+200 ms per 10 000-vector batch) and from 44-47 µs to 57-61 µs into a
serde_json::Value(benches/json_float_parse_benchmark.rsruns the same
parse). Anf32is now parsed directly and correctly rounded
instead of throughf64, so a vector component can differ from before by
onef32ULP, toward the exact value. -
Python: a payload nested too deep raises instead of crashing the
interpreter. The conversion of a Python value to JSON recursed once per
level of lists and dicts with no bound, so a very deep list, or one that
contains itself (a = []; a.append(a)), overflowed the native stack: on
7.1.0 anupsertwith such a payload ends the process with a segmentation
fault. The conversion now stops atMAX_JSON_NESTING(127) levels, the
depthserde_jsonreads back, the payload dict (or an edge's properties
map) counting as the first, and raisesValueErrornaming the bound. A
payload within it but deeper than 124 levels still reaches core's own
refusal (VELES-017). -
Graph edges with properties survive a flush and a reopen.
flush()
writes every edge to theedge_store.binsnapshot, encoded with postcard,
and then truncates the edge WAL. An edge's properties are
serde_json::Values, which postcard can write but cannot read back, so a
snapshot holding any edge with properties could not be loaded: the
collection reopened with an empty edge store (a warning in the log), and
the next flush with a non-empty edge WAL overwrote the snapshot, making the
loss permanent. The snapshot now writes each edge's properties as one JSON
string;GraphEdge's own serde form, which the API and the edge WAL carry,
is unchanged. A snapshot of edges without properties keeps the same bytes
in both directions. Float properties read back exactly (see "Stored floats
read back bit for bit"). Anedge_store.binthat cannot be decoded is no longer
left in place to be overwritten: the open logs the decode error, moves the
file toedge_store.bin.unreadable-<unix milliseconds>next to it (never
replacing an earlier one), and opens the collection with the edges the WAL
still holds. A snapshot that cannot be read for another reason
(permissions, I/O), or cannot be moved aside (a read-only directory), fails
the collection's open withVELES-011;Database::openthen logs a
warning and leaves that collection out, as it does for any collection that
fails to open, where 7.1.0 opened it without its edges.Upgrading from 7.1.0 or earlier: 7.1.0 opened such a collection
without its edges and kept the snapshot until a flush found the edge WAL
non-empty and overwrote it; edges lost that way cannot be brought back. A
snapshot not yet overwritten is moved aside by the first 8.0.0 open, bytes
intact, for a manual recovery: postcard does not record the type of each
property value, so decoding them needs to know which types they had.
Do not downgrade a collection that has edges with properties to 7.x:
7.x cannot decode the 8.0 snapshot either, opens the collection without
its edges, and its next flush with a non-empty edge WAL overwrites them. -
A MATCH range predicate no longer hides nodes the graph range index
missed.MATCH (n:P) WHERE n.age > 10takes its candidates from the
in-memoryP.agerange index whenever that index exists, but only the
node-payload write (upsert_node_payload/upsert_node_payloads) fed it,
and every open started it empty. A node written through
upsert_node(id, payload, Some(vector)),upsert,upsert_metadata,
upsert_bulkorupsert_bulk_from_raw, or any node written before a
reopen, was then left out of the result, with no error, as soon as one
node-payload write had created the index — after a reopen, the query
returned only the nodes written since. Deletes also left their entries
behind. The range indexes are now derived from the payloads exactly like the
label index: every write and delete path that maintains the label index
maintains them too, through one shared remove-then-index step, and the open
rebuilds both from the stored payloads, WAL replay included, in the same
scan.The index lookup is also kept a superset of what WHERE accepts, which it
was not in four cases, each of which hid matching nodes:n.w = 0.5matched
exactly where WHERE accepts any value within 0.001 (the lookup is now that
tolerance, doubled, around the literal);n.age > 9007199254740992missed
2^53 + 1, which the index orders as thef642^53(a strict bound on an
integer is now looked up inclusive);n.w >= 0.0missed-0.0(the index
now orders-0.0and0.0equal, as WHERE does); and a top-level key
containing a dot,"a.b", was indexed under the key WHERE uses for the
nested patha→b(such keys are no longer indexed: WHERE cannot reach
them). Only scalar properties are indexed — null, booleans, numbers,
strings — since an array or an object never satisfies an index-eligible
comparison.Memory: every scalar property of every labelled node is now held in the
range indexes, strings included, on every write path and from the open on.
Measured on 100 000 labelled nodes, each with two numbers, a distinct 1 KB
string and a 20-element array, in a debug build (the open times are
only comparable with each other, not with a release build): the open took
23.8 s and 170 MiB of RSS, against 20.9 s and 46 MiB without the range
rebuild. The indexed strings dominate that cost: the indexes hold a copy
of every string property of every labelled node, and the 124 MiB they
added here is close to the 100 MB of strings the nodes carry. -
A bulk upsert of a point without a payload keeps that point's payload
indexed.upsert_bulkandupsert_bulk_from_rawstore nothing for a
point whose payload isNone, so the point keeps its stored payload, but
the index maintenance treated it as a delete: re-embedding a node with
upsert_bulk([Point::new(1, vector, None)])removed it from the label and
range indexes (MATCH (n:P)no longer returned it), from BM25 (its text no
longer matched), from the persisted histograms and from the columnar payload
mirror. Both bulk paths now derive every payload index from the points that
carry a payload, with a repeated id resolved to its previous payload in the
batch, and invalidate the mirror when a point keeps its payload.upsert
keeps its documented behaviour: a point without a payload deletes it.
upsert_bulkalso never maintained the secondary indexes, so an equality
lookup missed every point it wrote; it now does, asupsert_bulk_from_raw
did. The label and range indexes are now updated under the same payload
write lock as the write itself onupsertand both bulk paths, so a
concurrent write to the same ids can no longer leave them describing a
payload storage no longer holds. -
A payload write that fails part-way leaves the indexes describing what
was stored. A batch write to payload storage is not all-or-nothing: the
records written before the error, or all of them when only the sync fails,
are readable.upsert, both bulk paths andupsert_node_payloadsnow read
those ids back on the error and index what storage holds; the node-payload
write used to remove the old label, range and composite entries before the
write and index nothing on an error, so a node it had rewritten sat in
storage under no label at all. A bulk upsert whose vector write fails now
also updates BM25, the secondary indexes and the histograms for the
payloads it stored before returning the error, where it used to return
first and leave those payloads out of them. -
GraphCollection::traverse_bfs_parallelcounts only results against
limit. The traverser emitted each start node at depth 0, which took a
slot oflimitbefore themin_depthfilter dropped it, so
traverse_bfs_parallel(&[1], limit = 1)returned nothing where
traverse_bfs_config(1, limit = 1)returns one node, and two starts with
limit = 2returned none. The start entries now get slots of their own, and
the result is truncated tolimitafter the filter. REST
/graph/traverse/paralleland MobilebfsTraverseParallelcall it. -
Python:
GraphStore.traverse_dfsruns core's DFS. It ran a DFS of its
own (dfs_collectincrates/velesdb-python/src/graph_store.rs) that
emitted one result per edge pushed, so a node reached by two edges came back
twice, in an order neither depth-first nor core's. It now calls
dfs_traverse, astraverse_bfsalready called core'sbfs_stream: one
result per node reached, in depth-first order, describing the edge that
reached it, withlimit(StreamingConfig.max_visited) capping the results
as it does for BFS. TheTraversalResultshape is unchanged. An empty
relationship_typeslist now follows every edge, as BFS and core read it;
the old DFS followed none and returned no result. -
LlamaIndex no longer inverts Jaccard rankings. Core returns a Jaccard
similarity (higher is closer), butVelesDBVectorStorelistedjaccard
among the distances and mapped its scores through1 / (1 + s): the best
match got the lowest similarity, sosimilarity_cutoffand node
postprocessors dropped it first. The direction now comes from
velesdb.metric_higher_is_betterthrough the new
velesdb_common.scores.score_to_similarity, with a literal fallback, pinned to
the binding by a drift test, for a wheel that predates the function. Euclidean
and Hamming keep their1 / (1 + d)mapping. -
PUT /guardrailsandDatabase::update_guardrailsreach every limit they
set.rate_limit_qps,circuit_failure_thresholdand
circuit_recovery_secondswere returned but never applied: each collection's
rate limiter and circuit breaker kept the values they were built with. And a
collection created, loaded or opened after the update started from the
defaults. Every registration path now pushes the live limits, under a lock
that keeps a concurrent update from missing a collection being registered.
Python, Mobile and Tauri call the same core method and get the same reach.
Each refuses limits that failQueryLimits::validateand changes nothing:
PUT /guardrailsanswers400(VELES-009), Python raisesValueError,
Tauri rejects withVELES-009, and BREAKING (Mobile)
updateGuardrailsnow throwsVelesError. -
apply_advanced_configtakes effect, or says when it will. The deferred
indexer and the async index builder were built only when a collection was
opened, so setting either on an open collection, asPOST /collections
does right after creating one, only rewroteconfig.json. One is now
installed at once into a collection that runs none, shared by every handle
on it; a change to one already running is persisted for the next open and
reported bypending_reopen(a write uses the component with no lock
spanning its use, so swapping it live could strand what a writer had pushed). -
Every
/metricsfamily is fed.velesdb_limits_exceeded_totalnow counts
every limit a query context refuses,velesdb_rate_limit_requests_total
every rate-limiter decision,velesdb_cache_collision_fallback_totalevery
VelesQL query-cache insert a hash collision sends to an occupied bucket,
velesdb_match_guardrail_hits_totaltheMATCHqueries a guard rail stops
while running, andvelesdb_match_similarity_queries_totaltheMATCH
queries with asimilarity()condition.velesdb_match_latency_seconds
gains_sumand_count. The server counts open connections in
velesdb_active_connectionson the HTTP and HTTPS paths, and
velesdb_queries_by_type{type="graph"}(andvelesdb_queries_total) on
/graph/traverse,/graph/traverse/parallel,/graph/traverse/streamand
/match, once the read is admitted (collection found, request valid, read
authorized). All of them read0forever before. A scrape reads the
registries only to clone the collection handles, so walking the indexes for
velesdb_index_size_bytesholds no registry lock. -
list_indexesreports what each index is and holds.memory_byteswas
0for every index; it is now each index's heap estimate (key bytes plus each
value's serialized id bitmap). A secondary payload index was labelled
"hash"; it is a B-tree and is labelled"btree". A range index reported a
cardinalityof0; it reports its distinct values.indexes_memory_usage
left secondary indexes out; it now sums every indexlist_indexeslists. -
vector NEAR $v OR <metadata>keeps the NEAR hits. The shape was routed
to thesimilarity()union executor, which has no NEAR leg: every NEAR hit
was dropped and only the metadata side came back, each row scored 1.0
(... WHERE vector NEAR $v OR category = 'other' LIMIT 5returned two rows
instead of five). ANEARleaf is TRUE in a boolean context, as the WHERE
evaluator and the planner already read it, so the disjunction now runs as the
NEAR search with the rest of the WHERE read that way: the rows the NEAR
search ranks, with their NEAR scores, and(NEAR OR x) AND yfiltered byy.
Core now returns what the WASM executor did; the shared executor fixture pins
both (X021, X022). Asimilarity()threshold underORkeeps the union
semantics. The extractor now also sees a NEAR vector underOR, so
NEAR $d OR SPARSE_NEAR $sruns the same dense + sparse fusion as itsAND
form; it used to drop the dense vector and run a sparse-only search. -
A failing branch fails the hybrid dense + sparse search. The dense branch
dropped its error, so a dense vector of the wrong dimension answered with the
sparse hits alone instead ofDimensionMismatch(VELES-004), and an unknown
sparse index answered with the dense hits alone, or nothing, instead of the
Sparse index '...' not founderror (VELES-009) the sparse-only search
returns. Both now fail the query —NEAR ... AND SPARSE_NEAR ...and every
binding'shybrid_sparse_search. -
VectorCollection::hybrid_sparse_searchaligns distance polarity. It
carried its own copy of the dense + sparse fusion, without the score-direction
alignment of #2102: on a Euclidean or Hamming collection a score-level
strategy (RSF, weighted, average, maximum) ranked the farthest dense point
first. It now delegates to the collection's fusion, so the server, CLI,
Python, mobile and Tauri hybrid searches rank like the VelesQL path. RRF was
never affected. -
velesdb-wasmranks asimilarity()threshold like core.WHERE similarity(vector, $q) > twithoutNEARreturned the passing rows in
insertion order with score 0.0, soORDER BY similarity()had nothing to sort
on. Each row now carries its similarity as its score and the rows come back
best first, as core returns them (conformancenear_casesN009-N012). -
velesdb-wasmDISTINCT honours ORDER BY and LIMIT.SELECT DISTINCTran
in the aggregation pipeline, which ignoredORDER BY, applied no default
LIMITand keyedDISTINCT *on id and score, so no row ever collapsed. It
now runs in the plain SELECT pipeline through core'sapply_distinct:
DISTINCT, then ORDER BY, OFFSET and LIMIT (10 by default, as in core), and
each row keeps its point id instead of a synthetic 0. -
velesdb query execute --format jsonprints[]for a SELECT that matches
nothing. It printed the prose lineNo results., which no JSON parser
accepts; the table format keeps the message. Found by the new executor
conformance case X020 (OFFSETpast the end), which the CLI runs. -
velesdb-wasmrejects a corrupt store at load instead of aborting at
search. AVELSblob whose vector or quantized buffers do not hold exactly
one row per id now failsimport_from_bytes/loadwithInvalid data: ....
This includes a v2 quantized store filled through the oldinsert_batch,
which appended to the f32 buffer whatever the mode and left the codes short,
so its first search indexed out of bounds. -
LlamaIndex no longer reverses fused, hybrid and BM25 rankings on a distance
metric. Every search path mapped its scores through1 / (1 + s)on a
euclideanorhammingstore, including scores that are already
higher-is-better:multi_query_searchwithrrforrelative_score,
hybrid_query(RRF of vector and BM25 ranks),text_query(BM25), a dense +
sparsequery(..., sparse_vector=...)andcontains_text_search(a match
scores 1.0). The best match came out least similar, the first one
similarity_cutoffdropped. Each path now states the direction of its
scores: those pass through unchanged, a raw single-metric score is mapped as
before, andaverage,maximumandweightedfusion, which keep the
metric's units, take their direction from core through
velesdb_common.scores.fused_higher_is_better. -
A dense + sparse search whose sparse (or dense) branch comes back empty
still returns fused scores.fuse_dense_sparseskipped fusion and returned
the other branch's raw scores, so on aeuclideanorhammingcollection a
sparse term no point carries returned ascending distances where every other
hybrid search returns higher-is-better fused scores; a caller reading them as
fused, as LangChain and LlamaIndex do, ranked the closest documents last. The
empty branch is now fused as an empty list, so the scores always have the
directionFusionStrategy::fused_directionstates for a dense and a sparse
branch (higher-is-better). The ranking of the non-empty branch is unchanged. -
Mobile: a core error keeps its code and recoverability on every VelesQL,
training and sparse or multi-query call.executeQuery,query,
trainPq,sparseSearch,hybridSparseSearchand the three
multiQuerySearchvariants wrapped core's error in aDatabaseerror with an
emptycodeandrecoverable: true, so a non-recoverableIndexCorrupted
(VELES-008) reached Swift and Kotlin as retryable. They now map it as every
other call does, keeping the call's context in the message: a VelesQL parse
error isVELES-010, a missing collection is theCollectionvariant and a
dimension mismatch theDimensionMismatchvariant, where these calls used to
answerDatabasefor both. -
Mobile:
SELECT EDGESreports kindRows, notMutation.
executeQueryanswered"N row(s) affected"for a read; it now answers
"N row(s) returned", as WASM always did. -
/metricsno longer writesvelesdb_traversal_nodes_visited_total,
velesdb_traversal_edges_scanned_totalandvelesdb_traversal_max_depth.
The server declared them process-wide and never fed them; the per-collection
velesdb_graph_traversals_totalandvelesdb_graph_traversal_nodes_visited_total
core already feeds are the traversal figures. The bundled Grafana dashboard
reads those instead and drops its max-depth panel.velesdb-server's
AppStateloses itstraversal_metricsandquery_limitsfields; the
limits live in theDatabase. The HELP texts ofvelesdb_documents_total,
velesdb_index_size_bytesandvelesdb_active_connectionsnow say what
each gauge counts. -
A secondary index's
index_typeis"btree"inlist_indexes, REST
GET /collections/{name}/indexes, the CLI, Python, Mobile and Tauri, which all
relay it. A consumer that matched"hash"to find payload indexes should
matchlabel == "secondary"or"btree". -
An
INlist longer than 16 no longer misses members. The binary
search ranked every array (every object)Equal, so a member past the
first array position was reported absent; and aCondition::Inbuilt
outside theVelesQLconversion —Condition::is_in,Deserialize, a
literal — reached it unsorted.InValuessorts at construction, and an
array or object probe scans the list linearly. -
A MATCH
WHERE n.prop != xagrees with SELECT's!=. A missing or
cross-typed property wasfalseunder MATCH while the SELECT filter
(andvalues_equal) calls it "not equal": the row is now kept, on node
and relationship properties alike. -
A candidate scan can no longer return a point outside the scan
universe.id IN (...)literals and bitmap supersets reaching
collect_filtered_scancould hydrate an id that owns no vector in a
mixed collection — a payload-only graph node, a torn-delete residue or
a nonexistent id — while the sequential scan excluded it. The row
universe is one rule now: vector-backed, or payload-backed when the
collection stores no vector at all, whichall_ids' doc finally
describes (it lists a superset — the storage set — not the scan's). -
range_index.binfiles are version-checked on load.OrderedValue's
Ordchanged with the exact-number epic, so a v1 B-tree could lose
folded keys on deserialize;RANGE_INDEX_VERSIONis 2 and
RangeIndex::from_bytesrefuses a stale version instead of adopting it. -
A collection lock and a rayon pool could deadlock the process. The
dense and sparse legs of a hybrid search, both legs of aParallel
SELECT and MATCH, and the vector and payload stores ofupsert_bulkran as
the two halves of arayon::joinon the global pool, each taking
collection locks there; a batch search's resolve, an aggregation's scan
and the lazy PQ training held storage or quantizer guards across a join on
that pool; and a merge into the HNSW index held the storage guard across
the graph pool'sinstall, which, called from an application's own rayon
job, let that worker run the application's other jobs meanwhile. Once a
writer queued behind such a guard,parking_lotparked every new reader,
among them the jobs the guard holder was waiting for. The legs and the
stores now run one after the other (theParallelstrategies are removed,
see Removed); the resolve, the scan and the training run on a dedicated
pool whose jobs take no lock; and a caller that is itself a rayon worker
waits on no pool at all — it runs that work, and the HNSW place, link and
vacuum-settle phases, on its own thread. -
A failed flush step stopped every step after it.
flushand
flush_fullreturned at the first failing step, so a failure of the
config, the vector or payload storage or the async builder's drain left
the HNSW save, the quantizers and the sparse, BM25 and secondary indexes
unwritten; andupsert_bulkreturned a failed builder flush before
refreshing the point count, applying the batch's sparse vectors and moving
the write generation, for points it had stored. Every step now runs and
the first error is returned at the end, with two exceptions that keep a
file from describing what is not on disk: the HNSW save waits on a drain
whose link succeeded, andvectors.idxon a vector storage flush that
succeeded.upsert_bulkalways finishes its bookkeeping. Ids the builder
could not index stay queued, and searchable, for the next flush, and are
logged rather than failing it. -
upsert_bulkwith an async index builder returned before its points
could be found. Its V2 path places each vector in the graph's arena and
leaves linking it to the builder untilmerge_thresholdpoints wait or a
flush runs. A graph walk cannot reach an unlinked slot, and no search read
what the builder held — the module doc said its buffer was "searchable via
brute-force scan", but that scan had no caller — so dense, filtered, VelesQL
NEARand hybrid search all missed such a point (on a collection over the
100-point exhaustive shortcut) until the next flush. Every vector search now
reads the builder's pending ids before its index search, scores them
exactly on what the collection holds for each now (a point deleted or
overwritten since is skipped or scored on its new vector, an id queued
twice is scored once) and merges them with the index's results. The
search's own restriction applies to them before they compete for itsk:
a filter's bitmap or a graph anchor set admits only its ids, and a search
that post-filters keeps every admitted pending point, so pending points
outside a filter or anchor set neither appear nor take the place of a
match. A flush drops what it indexes from its queue only once indexed, so a
search racing it finds every point. The cost is an exact scan of the
pending points on each search, up tomerge_threshold(default 10 000) of
them, more while a flush that failed or found another running leaves its
queue behind (the nextupsert_bulkover the threshold flushes again and
returns a failing link's error). A post-filtered search scans and keeps
all admitted pending points rather than itsk, and a batch or
multi-query search pays the scan once per query. Measured at 10 000
pending 768-d points (bulk_insert_v2_benchmark's
search_with_10k_pending_768d, on a shared 4-vCPU cloud sandbox, not the
reference machine): seedocs/BENCHMARKS.md. It lasts only until a flush
or the threshold links them. -
A restricted search could return the deferred indexer's points outside
its restriction, or lose its matches to them. It merged its
unrestricted topkinto every search: a filter applied
after the merge could find its places taken by buffered points it rejects
and return fewer matches or none, and a graph-anchoredNEARover more
than 10 000 anchors could return buffered points outside the anchor set.
They now go through the same merge as the async index builder's points,
under the same restriction: only the ids the search's bitmap admits, and
every admitted point when the search post-filters. -
A search could score a point on a vector it no longer has. An
unindexed point was scored on the copy its source took: the async index
builder's arena slot, the deferred indexer's buffer.
A point bulk-loaded and then overwritten before a flush was scored on its
first vector, and the index's result for a point overwritten since it was
indexed could outrank the newer one, since the newer one won only within
each source's own topk. Every unindexed point is now scored on what
vector storage holds for it, and the index's result for an unindexed id
is dropped before ranking, so only the current vector competes; when such
ids crowd the index'sk, it is asked for more — at least twice as many
each time, never more thankplus the pending count, so at most
log2((k + pending) / k)more index searches, run in parallel across a
batch's queries — so the next point it ranks keeps its place. -
A point deleted and then written again while the deferred indexer
buffered it stayed out of search until the next merge: the delete
recorded the id in a set that filtered the buffer's results, and the
rewrite did not clear it. The set is gone: a delete removes the point from
the buffer, and a search skips a point storage no longer holds. -
An async index builder flush indexed the copy
upsert_bulkqueued. A
point overwritten since was indexed on its old vector, and a point deleted
since was put back in the graph, where it could take a search's places
only to be dropped when results were read. A flush now inserts what
storage holds for each queued id when it runs, and leaves out an id it no
longer holds. It also leaves out an id the graph already holds as storage
does — a point upserted straight into the graph afterupsert_bulk
queued it — which 7.1.0 inserted a second time, leaving its first node in
the graph, unmapped, untilvacuum. The deferred indexer's merge skips
such an id the same way. -
A search could return fewer than
kresults, or none, near a point
deleted or overwritten since the lastvacuum. A delete or an upsert
leaves the point's old node in the graph, mapped to no id. The graph walk
ranked those nodes with the rest and was cut tokbefore they were
dropped, so ak = 1search at a deleted or moved point's old vector
could come back empty; most visibly without reranking (Fastquality, or
an index built without exact-distance features). Present in 7.1.0.
When such nodes cut a search short,HnswIndexandNativeHnswIndexnow
walk again for the walk's wholeefpool and keep its firstkmapped
nodes; if that pool was all used and still falls short, once more at
twice theef(at most 10 000). A search that fetches more candidates
than it keeps (reranked, PQ-rescored) walks again only when such nodes
leave fewer than it keeps, and a walk already run is not repeated. A
search near no such node runs one walk, exactly as before. A neighbourhood so dense with deleted points
that this still falls short needsvacuum. -
A search racing a deferred-indexer merge could miss its points. The
merge emptied its buffer before inserting what it held. It now inserts
first and drops only what it inserted, as the async index builder does,
and keeps an entry written again during the merge for the next one. -
A merge could undo a delete or an upsert that raced it. The async
index builder's flush and the deferred indexer's merge read the vectors
to insert, let go of storage, then inserted them: a delete landing in
between was undone (the point put back in the graph), and an upsert
landing in between was overwritten by the vector read before it. Each
chunk of 256 ids is now read and inserted under onevector_storageread
guard, which a delete or an upsert must wait for before it writes; a
writer waits at most one chunk's insert. -
A merge dropped ids it had failed to insert, and a panicking merge
stopped every later one. The builder and the deferred indexer drop only
the ids the merge settled — inserted, held already, or deleted — and
keep the rest queued, and the flags that serialize