HOLOS 0.14.0

0.14.0 — 2026-09-24

Spilling has a disk ceiling, not only a memory budget

Caught while deciding whether this release was ready, and it is the kind of thing a release
should not carry. --spill-bytes bounds what a spilling operator holds; nothing bounded
what it writes, and an operator that spills writes in proportion to its input. That was a
documented hazard while it was reachable only by a DISTINCT under --reorder. Making
ORDER BY spill put it on by default for every unbounded sort — and the behaviour it
replaced was a clean refusal, the memory ceiling stopping the query in seconds. A sort over
a 654-million-quad store would have written on the order of a hundred gigabytes to the
scratch directory, which is very often the system volume. Trading a refusal for a full disk
is not an improvement.

--max-spill-disk <GiB>, default 16, bounds it. Past that the query is refused with a
spill-ceiling problem document carrying what it wrote, the ceiling, and the flag — the way
the memory ceiling already did. 0 removes it. The ceiling is per query, so two spilling at
once can write twice it, and it is enforced in the collectors themselves, so a library
caller gets it without asking. The 48.4-million-row sort this was sized against writes about
7 GB, well inside the default.

The top-k query was half heap allocation, and the heap was free

Profiled rather than guessed at, after the last release left the question open. holos-bench
gains topkprofile, which takes the query apart by ablation — the same join over the same
48.4 million rows, four times, changing only what the sink does with each row, with no timing
in the hot loop to perturb what it measures.

Two findings, both against expectation. The heap is free: under a second over 48.4
million rows, so the operator the query is named for was never worth tuning. And the join
cost as much again as the RocksDB scan beneath it — 10.5 s on top of 10.7 s — which is
where the time had been all along.

It was two heap allocations per row:

• a Vec of the variables each candidate quad bound, allocated inside the scan loop — four
  pointers, 48.4 million times. Now four inline slots, with a Vec behind them that a quad
  never reaches and only a wide VALUES row can.
• a Vec for each projected row. bindjoin::Sink now takes &[Option<TermId>] rather than
  an owned row, so the join fills one buffer and reuses it, and the top-k heap copies only
  the k rows it keeps out of millions.

	before	after
the join, over 48.4M rows	10.4–10.7 s	5.4–6.2 s
the whole query, in the profiler	29.0 s	25.0 s
the whole query, through the server	42 s	29.6 s

The server gains more than the profiler because it installs a counting allocator so the
memory ceiling has something to read, and all 96.8 million allocations were paying it.
BENCHMARKS.md §3e has the phase table and the method.

Thirty-eight W3C tests were failing because the harness told the parser the wrong base

A conformance baseline is only worth what its attributions are worth, and thirty-eight
entries in this one were filed under upstream: — a parser defect — when the defect was the
harness's. It parsed every test file against <assumed base>/<file name>, which is right
for a suite whose files sit beside their manifest and wrong for one that groups them into
subdirectories. The RDF/XML suites do: rdf-xml/manifest.ttl lists
rdf-ns-prefix-confusion/test0004.rdf, and parsing it against <base>/test0004.rdf resolves
every relative IRI in the file one directory too high, so the subject came out as
…/rdf-xml/test0004.rdf#foo where the fixture says
…/rdf-xml/rdf-ns-prefix-confusion/test0004.rdf#foo. The parser was told the wrong base and
did as it was told.

A test file's IRI is now the base directory plus its path relative to its manifest.

Suite	Was	Now
RDF 1.1	1019 / 1040	1038 / 1040
RDF 1.2	1382 / 1405	1401 / 1405

That is 4,013 of 4,021 across every suite, and all eight remaining failures are upstream
for real: six are oxrdfxml writing an rdf:XMLLiteral with every in-scope namespace
declared on it where RDF 1.1 asks for exclusive canonical XML, and two are spargebra —
a SPARQL 1.0 test of case-insensitive keywords, and SPARQL 1.2's relaxed rule on reusing a
SELECT variable in a later expression of the same SELECT. DESIGN.md §15's table had
drifted badly out of date in the other direction, understating every row; it and the
README now agree with what the suites report.

A skip is not a pass, so the skips are auditable. A suite that quietly skips a third of
its tests reports a compliance it has not demonstrated, and until now the reasons were
visible nowhere. HOLOS_CONFORMANCE_SKIPS=1 prints each suite's skips tallied by reason.
SPARQL 1.1's 113 are 47 protocol tests run by the dedicated protocol suites, 34 needing an
entailment regime this engine does not implement, 6 test types not implemented, and 26 where
HOLOS and a direct spareval run agree with each other and differ from the fixture.

ORDER BY without a LIMIT finishes, and is faster than not finishing

0.13.0 gave SELECT … ORDER BY … LIMIT a heap of the rows it returns. The sorts the heap
declines — no LIMIT, or one too large to hold — still went to the evaluator, which
collects every row and sorts the lot, so on a large store they were refused by
--max-blocking-rows or ran until the memory ceiling stopped them. This is the external
merge sort the design has named as the next piece since that release.

• A sorting collector beside the deduplicating one. holos_engine::spill::Sorted holds
  rows with their sort keys, writes a sorted run when it passes its budget, and merges the
  runs into one ordered stream. What an ORDER BY costs is the budget, not the answer, and
  the answer is never resident: the merge yields one row at a time and the serialiser writes
  it out. It reuses the run files, the encoding and the readers that DISTINCT already had.
• Faster, not merely bounded. Two million dates sorted in 8.5 s spilling against a
  budget of 8 MiB, against 20.0 s for the evaluator holding all of them in memory
  (holos-bench's topk). Writing runs to disk and reading them back still wins, because
  the evaluator's comparator decodes both sides of every comparison from the dictionary —
  2·n·log₂n decodes — and the collector decodes each key once, as the row arrives.
• One order throughout. The heap and the collector share topk::cmp_keys, so a query
  answered by either comes back the same way. Both are stable: rows their conditions do
  not separate keep the order they arrived in, within a run and across the merge. The
  evaluator sorts with sort_unstable_by and does not, which is conformant — SPARQL fixes
  no order over tied rows — but is not repeatable.
• --spill-bytes, was --spill-distinct. The budget now bounds a sort as well as a
  DISTINCT, so it is no longer named for one. The old name is still accepted.
• --max-blocking-rows follows. A sort that will be answered in bounded memory is
  measured by what is under it, so it is no longer refused for the size of its input.

Worth knowing before relying on a sort: SPARQL defines no order between terms of unlike
kinds — an integer and a string — and says an implementation may extend it. Both this engine
and the evaluator extend it by comparing lexical forms, and that extension is not
transitive: 2 < 10 by value, "1abc" < 2 and 10 < "1abc" by lexical form, so those
three form a cycle and which comes first depends on the order they arrived in. Both paths
were measured doing exactly that, and disagreeing with each other. A sort key of one
datatype, which is what real data has, is a total order and is fully determined.

---

Python bindings for this release are on PyPI: pip install holosdb==0.14.0

Each archive holds holos (the CLI) and holos-server, built with RocksDB.
Verify with sha256sum -c SHA256SUMS.

HOLOS 0.13.0

0.13.0 — 2026-09-21

A running server says which build it is

Asked of a server that had just been restarted: was it the binary with the fix, or the one
from before? Nothing said. /health answered ok, /stats gave the counts, every
response carried the HTTP library's own Server: tiny-http (Rust), the banner had no
version line, and neither binary took --version. Now, from one stamp:

• Server: holos/0.13.0 (d4b4fdd) on every response — the place HTTP gives a server
  to say what it is (RFC 9110 §10.2.4), replacing the library's name.
• /stats carries "version", "commit" and "modified" next to the counts, for a
  monitor that already reads it.
• holos-server --version and holos --version, and the same line in the
  startup banner.
• The commit, and whether the tree was clean. A binary built between releases says
  the same version number as the release; the parenthesis says which commit it was built
  at, and modified that the tree had uncommitted changes. The stamp is written by a
  build script that re-runs whenever anything under crates/ changes, so it cannot go
  stale between edit and rebuild. A build without a repository says only its version.

New crate holos-build holds both halves — the stamp and its display — so the two agree.
Also in this release: scripts/third-party.py again reproduces THIRD-PARTY.md — the
console's "loaded by the browser" section, hand-written at 0.11.0, is generated now, with
the versions read from the pins in ui.rs — and the dependency counts are current.

ORDER BY … LIMIT from a heap of the rows it returns, and a timeout that says which limit

Prompted by SELECT ?o WHERE { ?s schema:deathDate ?o } ORDER BY DESC(?o) LIMIT 4 on the
653.8-million-triple store, which had run for ninety minutes at one core when it was looked
at, holding gigabytes, and could not be stopped. The evaluator answers ORDER BY by
collecting every row, sorting the lot, and applying the LIMIT afterwards; its comparator
decodes both sides of every comparison from the dictionary, so a sort of n rows costs
2·n·log₂n point reads and literal parses. And once the rows were collected the query had
stopped reading, which is where both the timeout and the memory ceiling are checked — so
neither could reach it.

• A top-k operator. SELECT … ORDER BY … LIMIT, with or without OFFSET, is now
  answered from a heap of OFFSET + LIMIT rows: the body is evaluated as a stream, each
  row's sort key is decoded once as it arrives, and a row that sorts after the heap's worst
  is dropped on arrival. The cost is n decodes of the key, n·log₂k comparisons in
  memory, and k rows held. Where the bind join accepts the body — a scan, a star, an
  optional, a filter — its rows are streamed as ids and only the key is decoded per row;
  the columns of the rows that survive are decoded at the end, so a wide projection costs
  what a narrow one does. The body is a streaming evaluation, so the deadline and the
  ceiling apply the whole way through. The shape is recognised exactly and everything else
  — a DISTINCT between the sort and the slice, no LIMIT, a slice past a million rows, a
  sort key with EXISTS or RAND() — goes to the evaluator unchanged. The order is
  SPARQL's, held to the evaluator's own by a test that sorts a mixed bag of every kind of
  term both ways. The query above, 48.4 million rows on the predicate, answers in 42 s,
  of which the scan alone is 21; SELECT ?s ?o … ORDER BY ?o LIMIT 3 in 41 s. On an
  in-memory store of two million dates the heap is 17.6× faster than the evaluator's
  sort (holos-bench's topk).
• A decode cache on the view. Every path that turns an id into a term — the
  evaluator's rows, the bind join's, a filter's variable, a sort key — now goes through one
  cache per query, bounded at a quarter of a million terms and started again when full. A
  scan grouped by object decodes each date once rather than once per row: the heap's
  evaluator-side path went from 264 s to 43 s on the query above from this alone, and an
  ORDER BY the heap declines, whose comparator decodes both sides of every comparison,
  is helped in the same proportion.
• Admission control follows. --max-blocking-rows measures a sort in that shape by
  what is under it rather than by its input, and the refusal of a sort it still declines
  now says which LIMIT would help, since one does.
• A timeout is answered with its number. A query stopped by --timeout was answered
  with the evaluator's own words, the SPARQL operation has been cancelled, under the
  timeout problem type. It now says query cancelled: it ran past the 300 s time limit
  (raise it with --timeout, or ask a narrower question) — the same way every other failure
  is told what happened and what would change it. The type is unchanged.
• deploy/holos.env sets HOLOS_TIMEOUT=300. The server's own default is still no
  limit; the deployment patterns are for a server other people reach, where five minutes is
  the difference between a query that is wrong and a process that is over its ceiling for
  everyone until it finishes.

The geospatial review: axis order, UTM, coordinates off the planet, and a string for a system

Prompted by a query that put Amsterdam in the Indian Ocean. It built its points as
<…/EPSG/0/4326> POINT(4.9003 52.3791) — longitude first — and EPSG:4326 puts latitude
first, so the store read latitude 4.9, longitude 52.4, and buffered a spot off the Somali
coast. That reading is GeoSPARQL's: a literal uses the axis order its reference system
defines, and Apache Jena reads the same literal the same way. It is also the trap crs.rs
has warned about since the systems were added, and the review found the store right about
it and wrong, or short, about four things around it. Every number below was checked against
PROJ 9.

• A coordinate off the planet was read as a place. <…/4326> POINT(120 4.9) — a
  latitude of 120° — was buffered, measured and drawn at latitude 120. A geographic
  coordinate past ±90° of latitude or ±180° of longitude is now refused, in every function
  and in the spatial index: unbound, not somewhere. The commonest way to write one is to
  put the axes the wrong way round, and refusing it is the one case of that mistake a store
  can catch. The other case — both numbers plausible either way, which is the Amsterdam
  query — it cannot, and nothing can; the literal has to be written in the system's order,
  or in CRS84 with longitude first, or built with spatialF:transformSRS from a system
  whose order is not in doubt.
• UTM. spatialF:transformSRS(?p, "…/EPSG/0/25832") came back unbound: ETRS89 / UTM
  zone 32N, the Dutch and German working system, was not one the store could reach. Every
  UTM zone is now: 326zz and 327zz on WGS 84, north and south, and 258zz on ETRS89,
  which is taken as WGS 84 — the two have drifted under a metre apart since 1989, which is
  how PROJ treats them by default. The projection is the Transverse Mercator the National
  Grid already had, generalised over its ellipsoid and origin; the Grid's own tests still
  hold it to the Ordnance Survey worked example. Eight points across five zones agree with
  PROJ to a millimetre within 3° of a zone's meridian, 2 mm at 4°, 5 mm at 6°.
• A system named by a plain string was refused. The same query wrote the target as
  "http://…/27700", which in SPARQL is an xsd:string, and the function took only an IRI
  or an xsd:anyURI. Jena takes the string; so does this now.
• The National Grid, checked again. Amsterdam in EPSG:27700 agrees with PROJ's
  Ordnance-Survey Helmert to 2 mm, 7° from the Grid's meridian. PROJ's own default answer
  for a point outside Britain is 150 m from that — it falls back to a datum-free
  "ballpark" operation there — which is worth knowing before comparing.

What was already right and is now tested as such: geof:buffer in degrees and metres,
geof:envelope, geof:distance in metres (Amsterdam Central to Rotterdam Port, 58,337 m),
geof:sfWithin, a bare literal read as CRS84, and geof:getSRID reporting the declared
system.

---

Python bindings for this release are on PyPI: pip install holosdb==0.13.0

Each archive holds holos (the CLI) and holos-server, built with RocksDB.
Verify with sha256sum -c SHA256SUMS.

HOLOS 0.12.0

0.12.0 — 2026-09-20

A failed query explains itself to a program, not only to a person

A query or update that fails is now answered with an
RFC 9457 problem document —
application/problem+json with type, title, status and detail, and for a syntax
error the line and column — under the status the SPARQL Protocol requires. The Protocol
fixes the status and says the body should explain; the RFC is the HTTP-wide standard for
the explanation's shape, and it is what a script can match on where before it could only
display a line of text. The type is a URI under https://holos.dev/problems/ whose last
segment is the kind: syntax, bad-request, unknown-function, service, rdf-parse,
policy, read-only, refused, timeout, memory-ceiling, evaluation, internal.
OPERATIONS.md says what each means and what to do. A client whose
Accept names text/plain and nothing else still gets the line of text.

Two things were wrong before and are fixed by the same change. A failure that arrived while
the answer was being written — a timeout, or the memory ceiling reached mid-scan — escaped
the handler and reached the client as an empty 500 with the explanation in the server log
only; it is a problem document now. And a query refused from its estimate was classified as
a bad request, which it is not — it is well-formed and the deployment declined it — so it
has its own kind and the 500 the Protocol assigns.

Checked over a real socket: a syntax error at 1:26 answered with its type, line and column;
an unknown function named in detail; the plain-text form on request; a protocol mistake
and the read-only refusal as problems of their own kinds.

What the specification keeps silent stays silent: an expression that fails inside a query is
a value, not a failure, and a BIND that could not answer leaves its variable unbound with
no explanation. That would need a HOLOS extension, and is not claimed.

The map's tiles were refused: OpenStreetMap wants a referrer

0.11.0 sent no referrer from the console at all, and OpenStreetMap's tile servers answered
403 — their tile policy requires a
Referer, and a page that withholds one is treated as hiding. Tile requests now carry the
page's origin and nothing more — scheme, host and port, never a path or a query — set on
the tile layer alone, so the page's own policy stays no-referrer for the CDN and for any
link a user follows out of a result. The origin says which server asked, which the request's
address said already. Measured with the served console script in a headless browser: twelve
tiles requested, twelve loaded, every one carrying referrerpolicy="strict-origin".
--ui-tiles none still sends neither tiles nor referrer.

The console object is now window.holosConsole, so a script or a test can reach it.

Jena's spatialF: transforms, so its queries run unchanged

Apache Jena fills GeoSPARQL's missing transform with three functions in
<http://jena.apache.org/function/spatial#>, and they are now names over the code
holos:transform already had: spatialF:transformSRS(geom, srs) is holos:transform
argument for argument and answers identically for every system and for the refusal;
spatialF:transformDatatype(geom, datatype) rewrites a literal between WKT and GeoJSON
while keeping its reference system — GeoJSON only for CRS84, since RFC 7946 fixed it there,
and GML refused since this engine does not write it; spatialF:transform(geom, datatype, srs) does both, in Jena's argument order. Checked from SPARQL against the GeoSPARQL example:
a CRS84 point to Web Mercator, to GeoJSON, and to EPSG:4326 with its axes swapped.

The reach is what crs.rs has — CRS84, EPSG:4326, 27700 and 3857 — where Jena resolves any
EPSG code through GeoTools; a code this engine cannot transform comes back unbound, not
relabelled. Jena's other spatial functions are not claimed.

---

Python bindings for this release are on PyPI: pip install holosdb==0.12.0

Each archive holds holos (the CLI) and holos-server, built with RocksDB.
Verify with sha256sum -c SHA256SUMS.

HOLOS 0.11.0

0.11.0 — 2026-09-20

The console: MatGUI, and a policy that keeps it at home

The console is now MatGUI 6.1.0, the maintained MIT
fork of YASGUI, in place of the Zazuko fork. The swap was made on a measurement, not a
feature list: both consoles were driven headless against this server's own answers, under
the security policy below. MatGUI rendered the same tables, drew the ten WKT geometries of
the GeoSPARQL example on its map, and drew a CONSTRUCT as a node-edge graph; the Zazuko
fork has no graph view and the map it had was one written for this project, which read CRS84
only. The gains: a Graph tab for CONSTRUCT and DESCRIBE; a Geo tab (MIT) that
reads WKT, GeoJSON, GML and GeoHash, gets EPSG:4326's axis order right, reprojects the SRIDs
it knows, clusters, exports, and turns a drawn rectangle into a geof:sfWithin filter; a
table that scrolls rather than pages; a dark theme; a CodeMirror 6 editor. The costs, stated:
a 3.4 MB bundle where the old one was 1.0 MB, and a major version twelve days old. What did
not change: neither fork parses an RDF 1.2 triple term in a result — a SELECT binding
of "type": "triple" and a CONSTRUCT of <<( … )>> both land on the error tab, in both,
measured. The endpoints answer them correctly; the console's parsers are behind the syntax.

The graph and table plugins are Apache-2.0. They are loaded by the browser from the CDN and
copied nowhere, which is the footing the Apache-2.0 crates already stand on;
THIRD-PARTY.md lists them.

The console can no longer send anything anywhere but this server, and that is the
browser's guarantee rather than the bundle's:

• A Content-Security-Policy names this server, the script CDN and one tile host, and
  nothing else. connect-src 'self': the endpoint box will not query another endpoint,
  and no plugin can reach out — the geo plugin's lookup of unknown SRIDs at epsg.io is
  refused and the geometry skipped.
• Every CDN file is pinned by exact version and SHA-384 integrity hash; a CDN serving
  different bytes breaks the console visibly rather than running unreviewed code.
• No referrer leaves the page, and the page carries no inline script: its
  configuration and stylesheet are served from this origin at /ui/console.js and
  /ui/console.css.
• --ui-tiles (HOLOS_UI_TILES) names the basemap's tile template, or none. Tiles are
  the one disclosure a policy cannot close — which ones a map fetches says where its user is
  looking — and none draws geometries over a blank background. --no-ui remains the
  airtight option.

The served page was loaded in a headless browser under the header policy: every plugin
rendered, no refusal, no integrity failure. DESIGN.md §10's console section now also
lists the views the console owes the thesis — named graphs with their Graph Store verbs,
"your view" (who the server took you for and what that opens), holons with their versions,
tick timelines and per-statement provenance, and a boundary as a tree — each resting on
/query and /graph so that it shows a principal exactly what the scan lets them see.

---

Python bindings for this release are on PyPI: pip install holosdb==0.11.0

Each archive holds holos (the CLI) and holos-server, built with RocksDB.
Verify with sha256sum -c SHA256SUMS.

HOLOS 0.10.0

0.10.0 — 2026-09-20

The release that made a bulk load 4× faster: the 653.8-million-triple file that took
3 h 43 m at 0.9.1 takes 55 minutes, producing a store with the same counts and the
same answers. Seven changes, each measured on that file, in the order they were found —
because each one's measurement is what found the next. This release also carries 0.9.1,
below, which was cut on the 19th and never tagged: statistics kept with the store, and a
withdrawn claim.

	time	quads/s	peak memory
0.9.1	13,379 s	48,869	3,413 MiB
+ the seen filter and a hot set	5,241 s	124,752	3,915 MiB
+ parsing on its own thread	4,479 s	145,990	4,012 MiB
+ hot set by spread, not hits	4,417 s	148,011	—
+ a whole bloom filter on str2id	3,870 s	168,939	4,059 MiB
+ index runs sorted and written on a thread	3,597 s	181,793	4,208 MiB
+ dictionary windows written on a thread	3,360 s	194,574	3,855 MiB
+ the final merge three orders at a time, in bounded files	3,277 s	199,545	3,434 MiB

A bulk load stops asking the disk whether a term is new

A 3-million-triple profile of the same data predicted 206,859 quads/s; the load ran at
48,869. Every mid-load flush clears the term cache so memory stays flat, and after the first
flush the cache no longer knows what it has forgotten — so every new term, all 199
million, was looked up in str2id on disk before allocation, to prove it was new. Counted
on a 3-million load forced through sixty flushes: 1,268,458 reads that found nothing against
100,287 that found something, at 3.9 µs each where an empty store took 0.4. They were the
whole of the flush penalty, and the misses were 92% of it.

seen is a bloom filter in memory over every term the load has interned — 256 MiB, seven
hashes, 10.8 bits per key on this store. A term it has never seen was never interned by this
load; when the load began on an empty dictionary, that means it does not exist, and it is
allocated without a read. A false positive costs one read that was going to happen anyway; a
false negative cannot happen. A load into a populated store does not use it — a term
interned before the load is not in the filter — and a test loads a store in two halves to
prove the second half makes zero skips. The filter's own tests include one that failed on
the first version and was right to: a request for ten bits per key was rounded down to a
power of two and got five, and the false-positive rate came out at 11.8%, which is what
five bits gives. It is sized exactly now.

The hot set went in at the same time: cache entries hit often enough survive a flush. On
that run it saved 12% of the hits; the filter saved all of the misses. 653.8 million quads:
5,241 s against 13,379, a store that answers identically, and misses at 0.09% of terms.

Parsing runs on its own thread

With the misses gone the load sat at 99% of one core with seven idle, alternating between
parsing a quad and interning it. Parsing is a quarter of the work by loadprofile and shares
nothing with the store, so it runs ahead on its own thread over a bounded channel — four
batches of 8,192 quads — and the calling thread only interns. Ids are still issued in file
order by one thread; nothing about the dictionary changes, and the backend-parity test holds
through it. A scoped thread, so a byte slice or a borrowed file still works as input; the
reader gains a Send bound, which every existing caller already met.

Tests cover a document larger than the whole queue arriving complete and in order, a parse
error partway through surfacing as an error rather than a short load, and an empty
document. 4,479 s, the process at 140% CPU — the parser takes 40% of a second core and the
loading thread is the bottleneck.

The hot set keeps entries by spread, not by hit count

A subject's dozen quads are adjacent in the file, so it is hit a dozen times in a row and
never again; a postal code shared by a thousand subjects is hit a handful of times across
the whole window and again in every window after. Ranked by hits, 450,000 subjects a window
beat the vocabulary and filled the cap with terms that would never recur. Ranked by the
distance between first and last use in the window, they lose. Same hits at small scale,
7× fewer entries retained for them, and at full scale the cap is finally binding on
terms that deserve it.

A whole bloom filter on str2id, and how the fifth measurement found it

With the misses gone and parsing off the critical path, the load's rate still fell from
209k quads/s at ten minutes to 122k at sixty. Store::bulk_resolves — new counters the
CLI now prints — said why: 43.7 million reads that found a term, 1,230 s at 28 µs each,
28% of the load. Those are the mid-frequency vocabulary, used a few times across the file
in different windows, which no within-window signal can catch and which at 44 million
distinct terms no hot set can hold.

So each read had to get cheaper. Three things were tried, each measured on the same file:

• A partitioned bloom filter on str2id, the fourth time a filter on this family had been
  tried: 1,300 s at 29.6 µs. Nothing.
• Opening the sort runs and the source with FILE_FLAG_SEQUENTIAL_SCAN, on the theory
  that forty gigabytes of runs were evicting the dictionary from the page cache: slower, 27
  and 34 µs at the flushes where they had been 21 and 25, and the process at 139% CPU
  throughout — not waiting on anything.
• A block cache with index and filter partitions in it: 9.0 → 15.9 µs and still climbing,
  as data blocks evicted the partitions.

What separated them was a histogram rather than a mean: the per-flush line now buckets each
read at 10, 50 and 200 µs. The shape was not a fast majority with a slow tail, which a disk
would give; it was the whole distribution walking right as the dictionary grew — 66% of reads
under 10 µs at the first flush, 9% by the eighth — with RocksDB's own log showing the family
at two or three L0 files throughout. What grows with a dictionary at constant L0 is its
number of levels, and a get without a filter reads a data block at each to learn the key is
not there. A partitioned filter reads a filter partition instead, which on a cache too small
to hold it is the same syscall — which is why four measurements of one saw nothing.

A whole filter is held by the table reader outside any cache. A get checks each level in
memory and reads one data block: 9.5 → 15.3 µs and flat from the fourth flush, then
15.4 µs across all thirty-three at full scale. Whole is affordable here where the index
families needed partitioning in 0.9.0: this family's files are sixty megabytes a flush and
its filter over the entire dictionary is about 250 MB, bounded by the dictionary rather than
by the load. 3,870 s; dictionary reads 677 s against the 650 the histogram predicted.

What remains of the read cost is one block read per hit, which is the floor on this platform
without holding data blocks in memory. Below that means batching hits into a MultiGet, or a
dictionary that lives elsewhere than RocksDB. The loading thread is at 100% of a core with
six idle, and the parse thread at 40% of another; the next factor is more threads.

Two more threads: the index runs, and the dictionary windows

The loading thread was stopping to do two things that touch neither the dictionary nor the
id sequence. Every four million quads it sorted the buffer nine ways and wrote nine runs;
every 256 MB of dictionary rows it merged the window's runs and wrote a compressed, filtered
SST per family. Both now happen on their own threads while the loading thread goes on
interning. Ids are still issued in file order by one thread, and the backend-parity test
still holds.

The index runs go to a worker over a channel of capacity one: the loading thread hands
the buffer over and fills the next, and a loading thread that outruns the sorter blocks
rather than stacking buffers up. One buffer more in memory, at most. 3,870 → 3,597 s,
all of it off the streaming phase, 2,950 → 2,700 s, the process at 172% CPU where it had
been 140%. The first measurement of this
change came back at 3,878 s — no gain — and a minute-by-minute curve showed why: the run led
by 6–9% except during the twenty minutes when builds and small-scale benchmarks were
running on the same disk. Measured again with nothing else running.

The dictionary windows are the more delicate one, because of what a flush is for. A
term may only be forgotten once both of its rows are readable on disk, and a term looked up
while its rows are still being written must be found in the cache, or it is allocated a
second id. So the merge and the write go to a thread, and the ingest and the trim stay on
the loading thread and happen when the files are in — polled at one atomic load per write,
forced before the next flush. The first version trimmed everything at that point, including
the terms interned since the flush, whose rows were in the next window and nowhere on
disk; the repeated-flush test showed it as 32 disk misses where there had been 0, each one
a term about to be allocated twice. A term is told apart by its id now — ids are issued
densely per tag, so one at or past the counter as it stood at the flush is one the flush
did not see — which is exact rather than timing-dependent. 3,597 → 3,360 s, streaming
2,700 → 2,462 s. Memory during streaming is flat across all three runs, 3.4–3.5 GB
sampled: the window's rows are already on disk as a run when it is handed over, so the
thread holds readers, not rows, and the cache holds one window plus a few seconds of the
next. The peaks in the table are from the final merge, where committed memory cycles
between 2.9 and 4.2 GB each minute and a once-a-minute sampler catches a different point
of the cycle each run.

Dictionary reads cost 707 s and then 729 s against 677 — the ...

HOLOS 0.9.0

0.9.0 — 2026-09-11

The release that made a load unable to take the process down, after 0.8.0 did the same
for queries. Found by loading 653,839,702 triples and watching.

A bulk load's memory did not depend on the load. Then it did.

The streaming phase behaved exactly as designed: 2.4 GB peak across three and a half
hours, sawtoothing as the dictionary filled, ingested and cleared. Before 0.7.0 the term
cache alone would have wanted about 62 GB, so this was the fix confirmed at forty-seven times
the scale it was verified at synthetically.

Then the final merge reached 18 GB on a 32 GB machine — three times, once per triple
order. It survived. It should not have had to.

The memory was not HOLOS's. index_opts asked every index family for a bloom filter, and a
full filter is built in one piece: RocksDB's FullFilterBlockBuilder keeps a 64-bit hash
per key and turns the lot into bits at finish. A memtable flush does that for a few thousand
keys and nobody notices. A bulk load writes one file per order for the whole store, so the
vector is 8 bytes times every triple loaded — 5.2 GB at 653 million, and twice that at the
instant it doubles, because it holds the old buffer while filling the new one.

The fix is to cut the filter into partitions as the file is written, each one finished and
released, so what is held is one partition rather than one file. It needs the two-level index,
so both are set.

It applies to the families, not only to the bulk writer, and that was a second decision
taken after a second measurement. The first version partitioned the writer alone, on the
grounds that the read cost was unknown and the write side was on fire. Two things then turned
up. A whole filter is pinned by the table reader for as long as the file is open, which is
what makes the 653.8-million-triple store cost about 4 GB before it answers anything. And
a compact rewrites every ingested file with the family's options — so partitioning only
the writer meant routine maintenance handed back exactly what the load had been fixed to
avoid.

holos-bench's filterread measured the read side at 30 million triples, flipping that one
call and nothing else:

	whole	partitioned
peak resident	743 MiB	275 MiB
probe that misses	2 us	2 us
probe that hits	1602 us	1583 us
full scan	5.7 s	5.9 s
load	100.0 s	91.6 s

No read penalty and 63% less held. The miss probe is the one that had to stay flat — it is the
case a bloom filter exists for, so an unchanged 2 us says the filter is still skipping files
rather than having quietly stopped working.

Measured by loading the same synthetic triples at two sizes, chosen so that only the index
grows — 4096 subjects and 64 predicates crossed with as many objects as the count needs, which
holds the dictionary at a few thousand terms whatever the count:

triples	full filter	partitioned
25,000,000	723 MiB	271 MiB
100,000,000	2,558 MiB	395 MiB
growth for 4× the data	3.54× — linear	1.46×

The streaming phase took 55.2 s against 55.9 s and 220.1 s against 221.8 s across the two
builds — the bias check, since a change to the writer cannot affect the parse, and a
difference there would have meant the comparison was measuring something else. Extrapolating
the linear arm to 653.8 million predicts 16.3 GiB against the 18.0 GiB observed.

The merge got slightly faster too, by about a tenth at 100 million. Not building an 800 MB
vector turns out to be quicker than building one.

Why nothing caught it

crate::memory's ceiling reads a counting allocator, and a counting allocator counts Rust
allocations. This vector belongs to RocksDB's C++ one, so the ceiling could not see it, could
not have refused it, and would not have reported it. A limit that cannot observe the thing
it is limiting is not a limit — recorded here because the same blind spot covers every byte
RocksDB allocates, and the next one will not announce itself either.

A bulk load needs half again the source file in scratch

Not a change, a measurement, because nothing had written it down and the number is large. The
653.8 million triple load held 48.3 GB of sorted runs beside a 30.4 GB source, peaking
around 55 GB before the merge consumed them, to produce a 20.7 GB store.

Scratch goes in holos-ingest beside the database, deliberately, so the ingest can move files
rather than copy them across a filesystem boundary. It is cleaned up on success and on
failure. But it means a load wants room for the source, the scratch and the store at once, and
compact checks its headroom before starting while a load does not. Sizing guidance is in
OPERATIONS.md; the preflight is the next piece.

Verified end to end at 653.8 million triples

The whole path an operator takes, on this build: holos stats --store E:/store2 --data <30 GB Turtle> --bulk, then deploy/run.sh, then queries over HTTP.

	before	0.9.0
load, final-merge peak	18,007 MiB	3,413 MiB
server, memory with the store open	~4,000 MiB	199 MiB
server, peak during COUNT(*)	—	1,230 MiB
COUNT(*) over HTTP	653,839,702	653,839,702 in 3 m 56 s
load rate	50,210/s	48,869/s

Same count, same predicate histogram, same rate; the merge spike is 5.3× smaller and the
store costs twenty times less to have open, because the partitioned filters are no longer
pinned by the table readers.

The server names its store on its first line of output

That verification run began by reproducing a failure. A store loaded into one directory with
the CLI, deploy/run.sh started on its default HOLOS_STORE=./var/store, and every query
answered from an empty database with no error anywhere — the server had opened the empty
directory and created a fresh store in it. Geospatial functions with literal arguments kept
working because they never touch the store, which made it look like a partial failure rather
than the total one it was. Ten lines of startup output and none said where the data was.

Now the first line is store E:/store2 — 653839702 quads, or on the directory that was
being served, store ./var/store — empty. Nothing has been loaded here; if that is a surprise, check that this is the directory the load wrote to. One META_QUADS read, so it is free.
OPERATIONS.md's loading section now leads with the pitfall and the holos.env.local override
that run.sh sources last.

The spatial index's first build is one read, not 150 million

refresh_spatial runs unconditionally when the server starts, and its first build has to
look at every dictionary literal to ask whether it is a geometry — that is the invariant
the index rests on (everything below the watermark is indexed), and it cannot be skipped by
checking whether any GeoSPARQL predicate has triples, because a geometry is decided by
datatype and a wktLiteral interned under any predicate at all is one.

It was doing that with a point lookup per literal. On the 653.8-million-triple store, with
zero geometries in it, that was 3½ minutes before "listening" appeared, on every restart.

Storage::for_each_in_range walks from..to for a tag and hands each term to a callback.
The trait's default is the old loop, correct for any backend; RocksDB overrides it with one
bounded iterator over id2str, since the ids for a tag are dense and their big-endian bytes
are a contiguous key range. Terms minted in an open scope, which have no id2str row yet,
are walked afterwards from the scope, exactly as decode consults it first.

The override must say precisely what the loop says, and a test asks both — every
dictionary-backed tag, whole range and a window starting and ending mid-range, plus empty and
inverted ranges. An off-by-one on the upper bound fails it by name.

Startup on that store: 210 s → 46 s. The remaining 46 s is decoding 150 million literals
sequentially rather than fetching them; going lower means not decoding a literal whose stored
bytes already say its datatype is not a geometry, which is a codec-level peek and a separate
change.

Measured, not changed: what a large store is actually slow at

A sweep of representative shapes against the 653.8-million-triple store, timed net of a 2.7 s
store open. Nothing here is a fix; it is what the next piece of work should be aimed at.

shape	net	note
subject star, 11 rows	~0.1 s
object bound, selective	~0.5 s
two-hop join, LIMIT 10	~0.1 s
count over a bound predicate, 48.4M rows	~19.6 s	2.47M quads/s
DISTINCT over 3 distinct values	~38 s	scans all 48.4M to find them
range FILTER, 48.4M rows	123 s	slower than the plain scan above
ORDER BY + LIMIT 10, 48.4M rows	>15 min	8.4 GB buffered, then stopped by hand

The range pushdown fires and still loses. bounded scans = 3 — it is not failing to
engage. But a numeric span must also include the whole Tag::Literal range, because
xsd:decimal is not an inline type and a dictionary-backed literal could satisfy the
comparison. For a predicate whose objects are literals, that reads all of them. The same query
with a cut matching zero rows took 123 s: the cost does not depend on selectivity at all.
xsd:date is not inline either, so ranges over dates pay this too. Inline is Integer,
Float, DateTime, Small — and xsd:date being absent while xsd:dateTime is present is
the kind of gap that looks like a typo in a dataset rather than a performance cliff.

--reorder does not help, and on the CLI it cannot. The statistics build is a full scan —
about 300 s on this store — and the CLI redoes it on every invocation, so admission control
and the DISTINCT spill are both effectively unreachable from the command line. A server
builds them once at startup, which is where they work.
...

HOLOS 0.7.0

0.7.0 — 2026-09-09

The release that made a load's memory stop growing, and the first one you can download a
binary from.

A bulk load's memory no longer tracks the file

0.6.0 held every dictionary row until the end of a load and ingested them as one sorted
file. That made the term cache load-bearing: resolve reads the database, a buffered row is
not there yet, so the only thing stopping a term being interned twice was the in-memory
cache — which therefore had to hold every term the load had seen.

It grows at 222 bytes per distinct term, measured across a 6.7× range. A generated
person dataset carries 1,268,458 distinct terms per 3M triples, so the 32 GB file it came
from has around 281 million and the cache alone would want 62 GB. On a 32 GB machine
that load cannot finish — and it aborts the process rather than failing the request, exactly
as the query path did before 0.6.0 put a ceiling on it.

Flushing both dictionary families when the buffer reaches its budget makes the rows
readable, which makes the cache an optimisation again rather than a correctness requirement,
so it can be cleared. Memory then tracks the budget instead of the file:

distinct terms	0.6.0	0.7.0
900,000	714 MiB	500 MiB
6,000,000	1,796 MiB	392 MiB

The slope goes from +222 bytes per additional distinct term to −22. Six million distinct
terms now peak lower than nine hundred thousand.

The cost is several ingested files per load instead of one. At an artificially small 32 MiB
budget that is 52,419 quads/s against 38,844 — a consistent 26% — but at the 256 MiB default
the two rounds disagree about which build is faster, so it is inside the noise.
set_dict_spill_bytes is the knob.

Both families are flushed together and the cache cleared only afterwards, because a term is
safe to forget only once both of its rows are readable. The test for the failure this
could cause — a repeated term handed a second id after the cache is cleared, splitting one
node into two so a join that should match silently does not — asserts on dictionary_len,
so double-interning is visible rather than inferred.

Binaries, at last

The Releases page stopped at 0.1.1 while PyPI went to 0.6.0: every version since was tagged
and published as wheels, and none produced anything a person could download and run.
pip install holosdb serves Python callers and does nothing for someone who wants the
server or the CLI, for whom the alternative was a Rust toolchain and a RocksDB compile.

A tag now builds holos and holos-server for five targets — Linux x86-64 and aarch64,
macOS arm64 and x86-64, Windows x64 — checks that both binaries start before packaging
them, and publishes a release with a SHA256SUMS file. The release notes are the changelog
section for that version, so the two cannot drift.

It is a separate workflow from the wheels on purpose: that one is gated on a PyPI trusted
publisher, this one needs permission to write to the repository, and a single workflow
holding both is a wider blast radius than either job needs.

Repository

Consolidated to one branch. feature/initial_release held only four ceremonial merge
commits and nothing unique; release tags keep those commits reachable.

---

Python bindings for this release are on PyPI: pip install holosdb==0.7.0

Each archive holds holos (the CLI) and holos-server, built with RocksDB.
Verify with sha256sum -c SHA256SUMS.

holosdb 0.1.1

A metadata-only release. The compiled extension is byte-identical to 0.1.0 — if you are already on 0.1.0, upgrading gains you nothing functional.

pip install --upgrade holosdb

What changed

0.1.0 shipped its Documentation URL pointing at a main branch this repository has never had — it was feature/initial_release when that line was written, and is develop now. The link returned 404 from the PyPI project page, which is the first thing someone discovering the package clicks.

PyPI metadata is immutable once a version is uploaded, so a new version is the only way to correct it. The replacement uses HEAD, which GitHub resolves to whatever the default branch is, so it survives the next rename too.

The other four project links — Homepage, Repository, Issues, Changelog — were correct in 0.1.0 and are unchanged.

Verified before release

Rehearsed on TestPyPI first, then checked from PyPI proper:

• the corrected Documentation link resolves (200)
• pip install holosdb with no index flags gives 0.1.1
• has_rocksdb() is True; an INSERT DATA and SELECT round-trip through a persistent store
• 38/38 tests pass against the downloaded wheel
• all six attached files match their PyPI SHA-256 digests exactly

Wheels

abi3 for CPython 3.9+, RocksDB compiled in.

Platform	Tag
Linux x86_64	manylinux_2_28_x86_64
Linux aarch64	manylinux_2_28_aarch64
macOS Intel	macosx_10_13_x86_64
macOS Apple silicon	macosx_11_0_arm64
Windows x64	win_amd64

Published by PyPI trusted publishing (OIDC) — no API token exists in repository secrets.

Not in this release

Work landed on develop since 0.1.0 that does not affect the Python package, and so is not part of it:

• a map view for the SPARQL console, drawing geo:wktLiteral and geo:geoJSONLiteral results on Leaflet — that lives in holos-server, not the bindings
• CI hardening: per-job timeouts, Node 24 action versions, and a fix for the cargo metadata warning on containerised Linux builds

See the full changelog.

Licensed MIT OR Apache-2.0. Copyright Peter Winstanley.

holosdb 0.1.0

First published release of the Python bindings.

pip install holosdb

from holosdb import Store, Principal, Policy

store = Store("./var/store")
store.load("data.nt")

# ask a question *as somebody* — policy is applied at the index scan, so the
# answer is the one that principal is entitled to, for every query shape
for row in store.query("SELECT ?s WHERE { ?s ?p ?o } LIMIT 5",
                       principal=Principal("urn:user:alice", roles=["staff"])):
    print(row["s"])

What it is

An RDF 1.2 triplestore with SPARQL 1.2, SHACL, GeoSPARQL, and access policy enforced at the index scan rather than by query rewriting. The property that buys:

the answer to Q equals the answer Q would have over the sub-dataset the principal may see

That holds for every query shape without anyone enumerating them — a COUNT cannot leak the existence of hidden rows, and a FILTER NOT EXISTS cannot probe for them. Measured cost: 8 ns per quad.

Conformance

Suite	Correctness	Coverage
SPARQL 1.1	523 / 524 · 99.8%	84%
SPARQL 1.2	262 / 266 · 98.5%	99%
SPARQL 1.0	262 / 263 · 99.6%	93%
SPARQL Protocol	34 / 34	100%
Graph Store Protocol	13 / 13	100%
SHACL 1.2 Core	127 / 138	—

3,145 of 3,284 W3C tests pass, and all 139 failures are upstream — none is a HOLOS bug. Both numbers are reported because either alone misleads: correctness is how much of what runs passes, coverage is how much of the suite runs at all.

Both protocol suites are scripted HTTP conversations rather than queries, so they run by starting the real server on an ephemeral port and replaying each script against it.

Wheels

abi3 for CPython 3.9+, one per platform, with RocksDB compiled in — has_rocksdb() is True on every published wheel.

Platform	Tag
Linux x86_64	manylinux_2_28_x86_64
Linux aarch64	manylinux_2_28_aarch64
macOS Intel	macosx_10_13_x86_64
macOS Apple silicon	macosx_11_0_arm64
Windows x64	win_amd64

Published by PyPI trusted publishing (OIDC) — no API token exists in repository secrets. The attached files are byte-identical to those on PyPI; their SHA-256 digests match.

Also in this release

• SPARQL 1.1 Update at POST /update, all-or-nothing, with policy on the write path — a principal cannot delete what it cannot see, and SILENT never suppresses a policy refusal.
• Graph Store Protocol at /graph. PUT replaces, POST merges, DELETE removes the graph rather than emptying it, and multipart file uploads merge every part.
• 45 GeoSPARQL functions, including geof:buffer and geof:boundary.
• Federation via SERVICE, and CSV/dataframe → named graph through a TARQL-style mapping.
• BGP reordering from characteristic-set statistics: a badly-written five-pattern join at 7.5M quads went 4,560 ms → 321 ms, matching the well-written form.

Known limits, stated plainly

• No TLS in the server. Terminate at the front door; deploy/ has configs for Caddy and nginx.
• No online backup. RocksDB checkpoints are designed but not built, so a consistent snapshot needs the service stopped.
• Entailment is not implemented — that is 70 of the SPARQL 1.1 skips and needs the reasoner.
• A property path with a variable on the left of a zero-length operator is very slow. ?x ex:partOf* ?y unbound makes every term a candidate; bind the subject.

Full detail in DESIGN.md, deployment in OPERATIONS.md, packaging in PACKAGING.md.

Licensed MIT OR Apache-2.0. Copyright Peter Winstanley.