4.0.1

JVector Release Notes

Version: 4.0.0-rc.9 - 4.0.1

---

New Features

Runtime Configuration for Graph Index Builder (JMX by default, pluggable backend)

PR: #703

Description
Introduces GraphIndexBuilderConfig, a managed singleton that exposes GraphIndexBuilder
construction parameters as runtime-tunable attributes. Before this change, options such as
addHierarchy, refineFinalGraph, and parallelBuild could only be set at the construction
call site and required a code change or application restart to modify. With this change, all
parameters — graph topology flags, write path selection, build-time compression type, and PQ
compression tuning — can be inspected and updated live, without restarting the JVM.

Changes take effect the next time a GraphIndexBuilder is constructed; they do not affect
indexes that are already being built or have already been built.

Architecture

GraphIndexBuilderConfig implements GraphIndexBuilderSettings, a plain domain interface with
no dependency on JMX or any other transport, and registers itself with ManagementRegistry
(package io.github.jbellis.jvector.management). The registry hands the registration to
whichever ManagementBackend is active for the JVM. By default, that's JMX: the
io.github.jbellis.jvector.management.jmx.JmxManagementBackend wraps GraphIndexBuilderConfig
in a javax.management.StandardMBean and registers it with the platform MBeanServer under the
object name io.github.jbellis.jvector:type=GraphIndexBuilderConfig — behaviorally identical to
prior releases, so JConsole/jmxterm usage below is unchanged. All attributes are stored as
volatile fields so writes from a management-backend thread are immediately visible to
application threads without additional synchronization. Registration is best-effort throughout
this stack: a failure (for example, in a restricted JVM environment) logs a warning but does not
disrupt normal operation — the singleton continues to supply its default values.

The management backend is selectable per JVM via the jvector.management.backend system
property:

Value	Behavior
(unset) or jmx	JMX (default) — identical to the behavior described below.
none	No external exposure. GraphIndexBuilderConfig remains fully usable programmatically; nothing is registered with any transport. Use this where JMX is disallowed.
a fully-qualified class name	A custom io.github.jbellis.jvector.management.spi.ManagementBackend implementation, loaded reflectively (must have a public no-arg constructor).

See docs/admin-service-interfaces.md for the full design.

Managed Attributes

Graph topology

Attribute	Type	Default	Description
AddHierarchy	boolean	true	When true, builds HNSW-style hierarchy layers on top of the base Vamana graph. false produces a flat level-0 (plain Vamana) index, which uses less memory and may build faster on small datasets.
RefineFinalGraph	boolean	true	When true, runs a second diversity-refinement pass over each node's edges after the initial build. Improves recall at the cost of additional build time.

Write path

Attribute	Type	Default	Description
ParallelBuild	boolean	false	When true, serializes level-0 node records concurrently via OnDiskParallelGraphIndexWriter. Both writers produce an identical on-disk format; switching this flag does not require re-indexing existing data.

Build-time compression

Attribute	Type	Default	Description
BuildCompressionType	String	"NONE"	Compression used for scoring during graph construction. Valid values: "NONE" (full-precision), "PQ" (Product Quantization), "BQ" (Binary Quantization).

PQ build compression parameters — consulted only when BuildCompressionType is "PQ"

Attribute	Type	Default	Description
PqMFactor	int	8	Subspace divisor: the number of PQ subspaces m is computed as dimension / mFactor. Must be a positive integer that evenly divides the vector dimension.
PqK	int	256	Number of centroids per PQ subspace. Must be positive; conventionally a power of two (256 is the standard value).
PqCenterData	boolean	false	When true, globally centers the dataset before PQ cluster training. Recommended for Euclidean similarity; typically not needed for cosine or dot-product.
PqAnisotropicThreshold	float	-1.0	Anisotropic loss threshold for PQ encoding. -1.0 disables anisotropic weighting; positive values bias the quantizer toward directions that matter most for inner-product search.

How to Enable

GraphIndexBuilderConfig is initialized automatically on first access and registers its MBean
with the platform MBeanServer. No application code changes are required to activate JMX
management — connecting a JMX client to a running JVector process is sufficient.

Programmatic access — read or set values directly from application code:

import io.github.jbellis.jvector.management.GraphIndexBuilderConfig;

GraphIndexBuilderConfig config = GraphIndexBuilderConfig.getInstance();

// Read current values
boolean addHierarchy            = config.isAddHierarchy();
boolean refineFinal             = config.isRefineFinalGraph();
boolean parallelBuild           = config.isParallelBuild();
String  buildCompressionType    = config.getBuildCompressionType(); // "NONE", "PQ", or "BQ"
int     pqMFactor               = config.getPqMFactor();
int     pqK                     = config.getPqK();
boolean pqCenterData            = config.isPqCenterData();
float   pqAnisotropicThreshold  = config.getPqAnisotropicThreshold();

// Update at runtime (affects all subsequent GraphIndexBuilder constructions)
config.setAddHierarchy(false);
config.setParallelBuild(true);

// Switch to PQ build-time compression with custom parameters
config.setBuildCompressionType("PQ");
config.setPqMFactor(4);        // dimension / 4 subspaces
config.setPqK(256);
config.setPqCenterData(true);  // recommended for Euclidean similarity

Non-deprecated constructor — GraphIndexBuilder constructors that do not accept explicit
boolean flags read from GraphIndexBuilderConfig at construction time:

// Reads addHierarchy and refineFinalGraph from JMX config
var builder = new GraphIndexBuilder(scoreProvider, dimension, M, beamWidth,
                                    neighborOverflow, alpha);

Constructors that accept explicit flags continue to honor the caller-supplied values and do
not consult the singleton, enabling call-site overrides when needed.

Using JConsole

JConsole is the standard JMX browser included with every JDK installation.

1. Launch JConsole

   jconsole
   

   In the connection dialog, select the target JVM process by name or PID and click
   Connect. If connecting to a remote process, use
   <host>:<jmxPort> after enabling remote JMX on the target JVM:

   -Dcom.sun.management.jmxremote
   -Dcom.sun.management.jmxremote.port=9999
   -Dcom.sun.management.jmxremote.authenticate=false
   -Dcom.sun.management.jmxremote.ssl=false
   

2. Navigate to the MBean

   Select the MBeans tab. In the left-hand tree expand:

   io.github.jbellis.jvector
     └── GraphIndexBuilderConfig
           └── Attributes
   

3. Read an attribute

   Click on Attributes. The right-hand panel lists all attributes with their
   current values:

   AddHierarchy              true
   RefineFinalGraph          true
   ParallelBuild             false
   BuildCompressionType      NONE
   PqMFactor                 8
   PqK                       256
   PqCenterData              false
   PqAnisotropicThreshold    -1.0
   

4. Set an attribute

   Double-click the value cell next to the attribute you want to change, type the new
   value (true or false), and press Enter. The change takes effect immediately;
   the next GraphIndexBuilder constructed in that JVM will use the new value.

   Attribute changes are also logged at INFO level by JVector:

   INFO  GraphIndexBuilderConfig - JMX: addHierarchy changed true → false
   

Using jmxterm (command-line alternative)

# Connect to the target JVM by PID
java -jar jmxterm.jar
open <pid>

# Navigate to the MBean
bean io.github.jbellis.jvector:type=GraphIndexBuilderConfig

# Read all attributes
info -b

# Read a specific attribute
get AddHierarchy
get BuildCompressionType

# Set graph topology flags
set AddHierarchy false
set ParallelBuild true

# Switch to PQ build-time compression
set BuildCompressionType PQ
set PqMFactor 4
set PqK 256
set PqCenterData true
set PqAnisotropicThreshold -1.0

Notes

• The GraphIndexBuilderConfig class is annotated @Experimental. The attribute set and
  object name may change in a future release.
• JMX is the default management backend but not the only one: set
  -Djvector.management.backend=none to disable external exposure entirely (the config remains
  usable programmatically), or point it at a fully-qualified class name to supply a custom
  io.github.jbellis.jvector.management.spi.ManagementBackend. See
  docs/admin-service-interfaces.md.
• JMX attribute changes are global — they apply to all GraphIndexBuilder instances created
  after the change in the same JVM. Per-graph overrides are not supported in this release;
  callers that need per-graph control should pass values explicitly to the appropriate
  constructor.
• The parallelBuild attribute requires OnDiskParallelGraphIndexWriter to be on the
  classpath (it is part of jvector-base). Both writers produce an identical on-disk format;
  switching this flag does not require re-indexing existing data.
• The PQ parameters (PqMFactor, PqK, PqCenterData, PqAnisotropicThreshold) are only
  consulted when BuildCompressionType is "PQ". Setting them while BuildCompressionType
  is "NONE" or "BQ" has no effect on the current build but the values are retained and
  will apply if BuildCompressionType is later changed to "PQ".
• setBuildCompressionType accepts values case-insensitively ("none", "None", and "NONE"
  are all valid) and normalizes to the canonical name on storage. It validates immediately and
  throws IllegalArgumentException for unrecognised strings, so JMX clients receive an error
  at set time rather than silently at build time.

---

Enhancements

PR: #691

Description

Speeds up graph compaction (merging pre-built partition indexes into one) — 5.6–11.5× faster
cold, at roughly the same recall. Five changes:

1. Eliminate the cold disk scan — computeLayerInfoFromSources called getNodes(0), which
   seeks + reads every node record on disk just to count live nodes. Replaced with an in-memory
   liveNodes.cardinality() popcount (no I/O). The big one on large datasets.
2. Warm-start PQ retrain — retrain the codebook via Lloyd's refinement from the existing
   codebook instead of full k-means++ from scratch (far fewer passes, no recall loss).
3. Windowed streaming prefetch — sources are mmapped MADV_RANDOM, so bulk phases faulted one
   page at a time. Now each phase (retrain sampling, code pre-encode, L0 batches) streams the exact
   records it is about to read into the page cache first.
4. rerankK = searchTopK — the L0 cross-source search used a beam 2× the candidate budget; the
   extra candidates are pruned by diversity selection anyway. Halving it cuts ~19% off L0 for
   ~0.006 recall.
5. Post-compaction refinement optional (default off) — it costs ~20–25% of compaction time and
   contributes ~0 recall (a query-latency/navigability pass), so it is now opt-in.

The two structural fixes (1. scan elimination, 3. prefetch) remove the disk-bound overhead that
dominated cold compaction; 4./ 5. trim the remaining compute. Recall stays within ~0.006 of main.

main vs PR (perf-fix + prefetch): compaction time & recall

220g heap, -wi 1 -i 1 single fork ("cold" = warmup iteration),
drop_caches before each arm so both start disk-cold. Graph degree 32, beam width 100.
Recall is search@10 on the compacted graph.

• main = b86a94c8 (stock)
• PR = compaction-perf-fix @ 8324dd6d = disk-scan fix + warm-start retrain + streaming
  prefetch + rerankK + refinement-off

Dataset	Workload	main cold	PR cold	Speedup	main recall	PR recall
cohere-1M	2-TIERED	2m58s	26s	6.8×	0.6094	0.6116
cohere-1M	2-UNIFORM	2m58s	19s	9.4×	0.6202	0.6145
cohere-1M	4-UNIFORM	3m09s	27s	7.0×	0.6239	0.6230
cohere-10M	2-TIERED	27m09s	2m44s	9.9×	0.5649	0.5631
cohere-10M	2-UNIFORM	27m00s	2m46s	9.8×	0.5706	0.5648
cohere-10M	4-UNIFORM	29m24s	4m34s	6.4×	0.5689	0.5668
cap-1M	2-TIERED	3m26s	27s	7.6×	0.6646	0.6633
cap-1M	2-UNIFORM	3m15s	26s	7.5×	0.6671	0.6630
cap-1M	4-UNIFORM	3m54s	39s	6.0×	0.6706	0.6659
cap-6M	2-TIERED	17m15s	2m11s	7.9×	0.6330	0.6277
cap-6M	2-UNIFORM	16m56s	2m14s	7.6×	0.6360	0.6310
cap-6M	4-UNIFORM	18m36s	3m20s	5.6×	0.6427	0.6356
dpr-gemma-1m	2-TIERED	2m48s	19s	8.8×	0.7844	0.7781
dpr-gemma-1m	2-UNIFORM	2m44s	17s	9.6×	0.7757	0.7769
dpr-gemma-1m	4-UNIFORM	2m55s	24s	7.3×	0.7830	0.7846
dpr-gemma-10m	2-TIERED	25m41s	2m14s	11.5×	0.6895	0.6868
dpr-gemma-10m	2-UNIFORM	26m01s	2m20s	11.2×	0.6872	0.6852
dpr-gemma-10m	4-UNIFORM	26m57s	3m42s	7.3×	0.7052	0.6961

Takeaways

• 5.6–11.5× faster cold compaction than main across every dataset.

---

PR: #694

Description

Add FloatArray interface to allow easy float[] vector extraction from MemorySegmentVectorFloat and ArrayVectorFloat instances. It solves the problem that we have to:

• use explicit typecast to known types (float[], MemorySegment) to figure out what kind of the holder object VectorFloat has
• deal with reflection (MemorySegment available in JDK-22 but OpenSearch baseline is JDK-21) or per-element access semantics which is inefficient

The suggestion is to introduce new FloatArray interface to have a fast track float[] array conversion for VectorFloats that support it. The change is 100% non-breaking.

Closes #696

---

---

PR: #707

Description

This branch eliminates scattered version-comparison branching (if version >= X, if version < Y, version == Z) from the on-disk graph index read/write path and replaces it with a Strategy + Factory pattern: each
supported format version (2–6) gets its own GraphIndexFormat implementation, and GraphIndexFormatFactory centralizes version→format dispatch and detection.

What changed

New abstraction (graph/disk package):

• GraphIndexFormat — strategy interface covering header/footer read+write, feature ordering, and full sequential/random-access write orchestration for a single format version.
• AbstractGraphIndexFormat — shared default implementation; versions override only what's actually different for them.
• GraphIndexFormatV2..GraphIndexFormatV6 — one small class per version, built on a natural inheritance chain (V4 extends V3, V5 extends V4 + footer, V6 extends V5 + feature reordering + fused-PQ
  support).
• GraphIndexFormatFactory — forVersion(int), detectVersion(reader) (magic-number sniffing), getCurrentVersion().
• WriteContext — bundles the ~7 parameters that used to be threaded individually through every serializer method.

Migrated call sites:

• AbstractGraphIndexWriter (and its OnDiskSequentialGraphIndexWriter / RandomAccessOnDiskGraphIndexWriter subclasses) now delegate all format-specific work to the resolved GraphIndexFormat instead of
  branching on version inline. Feature-support validation (Builder.build(), fused-PQ gating) now asks the format (supportsFeature) instead of hardcoding version thresholds.
• Header / CommonHeader now delegate header size/read/write and common-header size/read/write to the per-version format, removing the version >= 3 / >= 4 / >= 6 chains that used to live in both classes.
• OnDiskGraphIndex load/footer dispatch goes through GraphIndexFormat.loadOnDiskIndex, removing the inline version >= 5 && useFooter branch; the two remaining fused-PQ-layout checks now ask
• CompactWriter (streaming N:1 compaction writer) validates once, at construction, that fused PQ is actually supported by the target format; the three per-call version == 6 checks in its hot paths were simplified
  to the already-existing fusedPQEnabled flag, since this writer only ever targets the current version. No added per-record overhead.

Fixed along the way:

• A regression where the writer/reader split silently dropped eager in-memory cache priming (getInMemoryLayers/getInMemoryFeatures) for non-footer format versions — caught by a ramBytesUsed() test failure,
  fixed by centralizing construction + priming in OnDiskGraphIndex.construct().
• allFeatures() / nonFusedFeatures() helpers now return explicit, frozen EnumSet.of(...) listings instead of deriving from EnumSet.allOf(...) / complementOf(...), so a future FeatureId addition can't
  silently change what an already-shipped version claims to support.

Testing

• New TestGraphIndexFormatFactory: version→format dispatch, unsupported-version rejection, per-version characteristics (supportsMultiLayer, usesFooter, getSupportedFeatures), and magic-number version
  detection.
• New TestOnDiskGraphIndex.testVersionRoundTrip: explicit write/read round trips for versions 3, 4, and 5 (single-layer and, for 4/5, multi-layer), in addition to the existing version-2 and current-version (6)
  coverage.
• Full jvector-tests suite passes (208 tests, 0 failures).

---

PR: #708

Description

This pull request adds comprehensive native C++ unit tests and benchmarks for the SIMD kernel dispatcher and vector similarity functions, and integrates them into the build and CI pipeline. The most important changes are:

Testing and Benchmarking Enhancements:

• Adds a new test_simd_kernels executable using Google Test, with tests for ISA detection and SIMD kernel correctness, and integrates it into the Meson build and GitHub Actions CI workflow.
• Adds a new bench_simd_kernels executable using Google Benchmark to micro-benchmark cosine_f32, dot_product_f32, and euclidean_f32 functions at realistic embedding dimensions.

Build System and Code Organization:

• Moves the core SIMD kernel and dispatcher source files into a src/ subdirectory and updates the Meson build rules and include paths accordingly. The native code layout has been reorganized in preparation for modularizing it into a standalone library, improving structure and maintainability.

CI Improvements:

• Updates the GitHub Actions workflow to install libgtest-dev, build, and run the new native C++ tests for all relevant ISA tiers, ensuring dispatcher correctness across hardware.

---

Skip jextract binding generation by default

PR: #716

Description

Modifies the process by which maven builds the native SIMD library so that bindings are not generated by default.
This change only impacts you if you build jvector from source.

Motivation

The existing build process re-generates the bindings on each build as long as jextract is in the path. This can cause issues since the jextract version on the system may not match the jextract version used to generate the checked-in bindings. This not only creates noise in the git working tree, but can also cause build errors since newer jextract releases may use language features from newer JDKs.

Usage notes

• jextract_vector_simd.sh is split into two scripts, build_native_lib.sh and jextract_generate_bindings.sh. Bindings are not generated by default during the build process.
• To force binding generation, set the maven property native.jextract.skip to false at runtime or in the pom. Alternatively, run jextract_generate_bindings.sh directly.
• If you have multiple jextract versions installed, or if jextract is not installed, you can set the environment variable JEXTRACT_BIN to the full path of the executable.

Example:

JEXTRACT_BIN="/opt/jextract-22/bin/jextract" mvn compile -pl jvector-native -Dnative.jextract.skip=false

---

Performance Improvements

Native SIMD Acceleration via Google Highway

PR: #668

Description

This PR introduces a comprehensive native SIMD backend to JVector, porting
all Panama Vector API kernels to native code backed by Google
Highway, a portable, header-only C++
library for SIMD operations. Previously, only two kernels in jvector-native
were implemented natively; all other SIMD operations relied on the Panama
Vector API as a fallback. Google Highway is currently enabled on Linux
x86-64 only; Windows and macOS are not yet supported. The native shared
library (libjvector.so) is built as a single
binary containing three ISA-specific variants: AVX-512 (Skylake), AVX2
(Haswell), and SSE4.2. At JVM startup, the most suitable ISA is automatically
selected via CPUID detection. SSE4.2 serves as the minimum supported baseline;
CPUs lacking this capability will trigger a SIGILL at runtime.

The ported kernels cover all hot paths:

• FP32 similarity — dot_product, cosine, euclidean
• Product Quantization (PQ) — assemble_and_sum, calculate_partial_sums,
  pq_decoded_cosine_similarity
• Non-uniform Vector Quantization (NVQ) — nvq_quantize_8bit, nvq_loss,
  nvq_square_l2_distance_8bit, and related helpers
• Element-wise arithmetic — add_in_place, sub_in_place, max,
  min_in_place, and others

A new calculatePartialSelfSum kernel was also added. On Linux x86-64, when
native vectorization is enabled, the Google Highway backend is used
exclusively for all similarity and quantization operations — the Panama
Vector API path is bypassed almost entirely. On other platforms (Windows,
macOS) or architectures (ARM, RISC-V, etc.), the Panama Vector API continues
to provide SIMD acceleration as before; native Highway support for these
platforms is planned for a future release if required.

Purpose / Impact

• Consistent, portable SIMD coverage across AVX-512, AVX2, and SSE4.2
  without per-target intrinsic duplication
• Automatic ISA selection at startup based on CPUID — no manual configuration
  needed
• Eliminates reliance on Panama Vector API for similarity kernels, removing
  JDK-version sensitivity for the fast path
• Runtime ISA can be capped for testing via the JVECTOR_MAX_ISA environment
  variable (avx3, avx2, or sse42)
• Minimum requirement: SSE4.2-capable x86-64 CPU; older CPUs will crash
  with SIGILL

How to Enable

Native vectorization is opt-in. Add the following JVM flags to enable it:

java --enable-native-access=ALL-UNNAMED \
     -Djvector.experimental.enable_native_vectorization=true \
     -jar your-app.jar

Once enabled, NativeVectorizationProvider loads libjvector.so and the
Highway-backed kernels are used automatically for all similarity and
quantization operations.

To additionally cap the ISA for debugging or benchmarking:

JVECTOR_MAX_ISA=avx2 java --enable-native-access=ALL-UNNAMED \
     -Djvector.experimental.enable_native_vectorization=true \
     -jar your-app.jar

Notes

• The build now uses Meson + Ninja to compile the
  three ISA variants; see
  jvector-native/src/main/native/jextract_vector_simd.sh for build and
  jextract binding-regeneration instructions.
• Google Highway is vendored as a Git submodule under
  jvector-native/src/main/native/third_party/highway at commit
  16f5f5c7
  (v1.4.0+15).
• CI workflows (unit tests and compaction) were updated to install Meson/Ninja
  and initialise the Highway submodule before building.

---

Bug Fixes and Issue Resolutions

PR: #730

Description

This PR is a bug fix for the issue reported by the Cassandra team (riptano/cndb#19259) that results in the first 128k vectors copied during compaction being invisible to GraphIndex.View, causing recall scores to plummet after compaction.

---

Testing Enhancements

PR: #692

Description

This PR is a fix for an issue in how file sizes were calculated by the JVector test harness, i.e. BenchYAML and AutoBenchYAML.

Problem 1: The index cache directory accumulates across runs

In Grid.runOneGraph (and runAllAndCollectResults), BenchmarkDiagnostics monitors two directories:
diagnostics.startMonitoring("testDirectory", workDirectory);
diagnostics.startMonitoring("indexCache", Paths.get(indexCacheDir)); // "index_cache/"
getTotalBytes() sums both. The index_cache/ directory is persistent — it is never wiped between runs. So the reported disk figure grows monotonically across the run, regardless of which configuration is under test. The delta
between a fused-PQ run and a non-fused-PQ run is invisible.

Problem 2: Even within a single run, the snapshot captures all feature sets, not just one

When multiple feature sets are built in the same runOneGraph call, the directory total covers all of them. You can't attribute disk usage to a specific configuration from that number.

There is even a TODO in the code acknowledging this at line 233:
// TODO this does not capture disk usage for cached indexes. Need to update

---

The fix

Rather than directory-level monitoring, measure the specific graph file for each feature set directly after it is written. In buildOnDisk, the path for each feature set's graph file is already computed:

if (handles.containsKey(features)) {
graphPath = handles.get(features).writePath();
} else {
graphPath = outputDir.resolve("graph" + n++);
}

After the build completes, Files.size(graphPath) gives the exact on-disk size for that specific configuration. This value should be returned from buildOnDisk (as part of a result map keyed by feature set) and then included
in the BenchResult params or metrics map alongside the other per-configuration numbers.

---