Skip to content

v0.7.1 — Atomic semantic graph rebuilds and correct scoping

Choose a tag to compare

@jpmanson jpmanson released this 06 Aug 14:00
· 30 commits to main since this release
fd8bec1

Bug-fix release for create_semantic_graph, introduced in v0.7.0. Two of these lose data — upgrade if you use the semantic graph.

Rebuilds are now atomic

create_semantic_graph(replace=True) deleted the old edges and committed, then ran the neighbour search — minutes on a large corpus — and only then inserted. A failure anywhere in between destroyed the semantic graph, and concurrent readers saw it empty for the whole build, not just on failure.

Neighbours are now computed first, and the old graph is swapped for the new one inside a single short transaction. A failed build leaves the previous graph intact. Constraint validation moved ahead of the delete as well.

approximate no longer skips nodes it should process

It treated any relationship of the type as evidence a node was already linked. A single hand-made SEMANTIC_SIMILAR edge therefore excluded both of its endpoints from every future refresh_semantic_graph() — permanently — as did edges generated from a different vector index.

It now counts only edges this method generated for the same index, which is what the generated_by marker was introduced for.

replace and drop are scoped by index

Rebuilding the semantic graph for one vector index deleted edges generated from another. Both now scope by index, and drop_semantic_graph(index=...) can target one:

db.drop_semantic_graph(index="papers_vec")

min_score has a sane default

0.0 was wrong in both directions: on cosine it admitted orthogonal neighbours, and on l2 — where scores are negated distances (<= 0) — it admitted almost nothing.

It now defaults to 0.1, and refuses to guess on an l2 index rather than silently building an empty graph:

db.create_semantic_graph(k=15)                  # DatabaseError on l2
db.create_semantic_graph(k=15, min_score=-0.5)  # explicit and meaningful

This mirrors the rule SIMILAR() already followed; the inconsistency between the two APIs was the underlying bug. Passing min_score=0.0 explicitly still means 0.0.

Behaviour change: builds that relied on the implicit 0.0 will now produce fewer edges. Pass min_score=0.0 to keep the old result.

Also

  • index_documents(configure_fts=True) registers the full-text index in the same call, so hybrid retrieval works without a second one. Off by default.
  • Fix count(DISTINCT n) raising TypeError: unhashable type: 'Node'. Graph entities are mutable dataclasses; DISTINCT now keys them by identity, which also fixes collect(DISTINCT n) and UNION over nodes. This predates v0.7.0.

Full changelog: v0.7.0...v0.7.1