Reference Benchmark catalogue

Benchmark catalogue

Every benchmark class in falco-benchmarks, its kind, its parameters and what it answers. Plus the harness versions the suite is built on.

This page is a generation target, currently written by hand. Source of truth: the @Param annotations and class names under falco-benchmarks/src/jmh, and the pins in settings.gradle.kts.

Anvil

Benchmark	Kind	Parameters	What it answers
BitPackerBenchmark .pack .unpack .roundTrip	library	bitsPerEntry 4, 5, 8, 15	What the packing loop of one section costs. It runs once per section on every load and every save, so a full-height chunk pays it 24 times. The parameter is the only thing that changes the iteration count per long.
PaletteDataBenchmark .encode .unpack .roundTrip	library	distinctStates 1, 8, 64, 200	What collecting a palette costs as the section gets busier. 1 is a section of pure air or pure stone — the majority of every world — and is answered without packing anything. 200 already needs eight bits per entry.
ChunkCompressionBenchmark .compress .decompress	library	compression ZLIB, GZIP, NONE × distinctStates 8, 200	The most expensive single stage of a chunk transfer, and the one the loader deliberately performs outside every lock. The payload is real serialised chunk NBT built through the same codec the save path uses, not random bytes — random bytes do not compress and would make zlib look far worse than it is.
RegionFileBenchmark .writeRaw .readRaw .roundTrip	library	–	The byte transfer alone, on an already compressed payload. readRaw uses positional channel reads and takes no lock; writeRaw takes the region lock for the sector allocation and the header update. Not a statement about a disk: the page cache is warm throughout.
ChunkSaveStageBenchmark .snapshot .codec .codecWithoutCompression .transfer .full	library	distinctStates 8, 200	The interesting one. It splits a whole chunk save into its stages and records which of them holds a lock, which is what turns the loader's structural claim into a number. The split is set out directly under this table.

ChunkSaveStageBenchmark splits a whole chunk save into the three stages it consists of, because the central claim of the loader is structural and this is what turns it into a number:

Stage	Lock held
snapshot — copy the section arrays	the read lock of the chunk; a game thread waits here
codec — palettes, packing, NBT, compression	none
transfer — hand the finished bytes to the region file	the region lock, for the allocation and the header
full — all three, so the sum can be checked against the whole	–

codecWithoutCompression splits the middle stage again — but not into a palette half and a zlib half, as its javadoc says. It returns a CompoundBinaryTag and never serialises it, so the difference against codec covers NBT serialisation as well as compression. full is what the decomposition should be checked against, and it has never been published.

The chunk is a ChunkColumn of plain arrays, not a Minestom chunk. A real Section needs a started server, and the registry time would land inside the codec stage and hide the very thing the benchmark isolates.

Light

Benchmark	Kind	Parameters	What it answers
LightNibblesBenchmark .getUniform .getAllocated .setUniformUnchanged .setAllocating .ofArray	library	–	What the uniform shortcut is worth. A section whose blocks all carry the same level keeps no array and answers from a field; every other section shifts a nibble out of 2048 bytes. Most sections of a world are completely dark or completely sky-lit, so the shortcut is the common path, not the exception.
SectionOpacityBenchmark .of	library	distinctStates 1, 8, 64, 200 × resolveCost 0, 50	What "resolve each distinct state once" is worth. distinctStates sets how often the cache misses; resolveCost sets what a miss costs. At resolveCost = 0 you are measuring the hash map and the two array writes per block, so the table looks like pure overhead. At resolveCost = 50 you are measuring what it saves: seven resolutions per distinct state instead of seven per block. What 50 tokens corresponds to on a real registry is not established.
LightPropagatorBenchmark .propagate	library	lightSources 0, 1, 8, 64 × occlusionPercent 0, 25	How the single-section search scales with the amount of queued positions. lightSources = 0 is answered without a search at all, which is the case for the overwhelming majority of the sections of a world. occlusionPercent = 25 is there because solid blocks stop the search early — measuring only an open section reports the worst case and calls it normal.
ChunkLightPropagatorBenchmark .propagate .propagateSky	library	sectionCount 4, 16, 24 × lightSourcesPerSection 1, 8	The same search across section borders, over a flat map (4), a shallow world (16) and a full-height overworld (24). Both searches are measured because the engine runs both per chunk and they behave very differently: block light is bounded by the amount of emitting blocks, sky light seeds nearly every block of an open column.
AreaVsPerChunkBenchmark .area .perChunk	decision	chunkCount 1, 4, 9, 16	Whether area forming earns its complexity: an area of n chunks against n separate calculateWithNeighbours calls. Written as a decision rather than a report — had it not held, the simpler per-chunk design was the better one and area forming was to be dropped rather than tuned. The chunkCount = 1 row is the control: a lone chunk has no loaded neighbours, so neither side has a ring to read and the two must come out level.
IncrementalVsFullBenchmark .incremental .full	decision	sky false, true	Whether replaying a changed position earns the memory the kept light costs. Both sides toggle the same block, compute the same nine chunks plus the same ring and write into the same sections; only the origin of a chunk's light differs. The block is toggled rather than placed, so both directions of an incremental update are measured — adding brightness and taking it back — and the world alternates between two states instead of drifting. sky is a parameter because a tick pays for both kinds and they gain very differently.

Both propagators keep their working buffers between runs, so the benchmarks reuse one instance for the whole trial and warm the buffers in @Setup. A fresh instance per invocation would measure two array allocations instead of the search. The residual cost is that the measured path never pays for growing a buffer, which is realistic for a running server and not for the first chunk after startup.

The last two start a server, unlike everything else in this table, because an area and a scheduler pass are defined over real chunks of a real instance. They are also the only two classes in the harness that set no heap flags, so their numbers rest on a default, unrecorded heap configuration. ChunkLightService has no benchmark of its own; what it costs is covered by those two and by the comparison benchmarks below.

Against the implementations Minestom ships with

These are the ones that answer "is this actually better", and they are the reason the loader and light engine documents can state factors instead of intentions. Each measures the original, not a reimplementation of it — which is why three of them live in Minestom packages, where the measured types are package-private.

Benchmark	Parameters	What it answers
RegionFileComparisonBenchmark .falcoRead .minestomRead .falcoWrite .minestomWrite	distinctStates 8, 200, and the JMH thread count	The central claim of the loader: what the lock granularity is worth. Run it with -t 1 and no difference is resolvable; the difference appears only under contention, which is why the thread count is the parameter that matters here. There is no @Threads annotation anywhere in the harness, so a plain ./gradlew jmh measures this at one thread and never exercises the claim at all.
ChunkSaveComparisonBenchmark .falcoSave .minestomSave .compressFalcoLevel .compressMinestomLevel	distinctStates 1, 16, 64, 256, 1024	Whether the palette handling shows up in a whole chunk save. Both sides run the identical Adventure writer over byte-identical payloads. The compression level is deliberately not equalised, because the levels differ in what the two loaders actually ship — Minestom 6 against ChunkCompression.DEFAULT_LEVEL = 2 — and the two compress* methods exist so that difference can be subtracted out. Ten of its twenty configurations are published — four methods over five distinctStates levels make twenty, and the ten belonging to .falcoSave and .minestomSave appear in How the Anvil loader is built under Measured: saving a chunk, as five table rows carrying both loaders side by side. Exactly one of those rows resolves a difference, and that row is confounded by the compression level. What is unpublished is the compress*Level pair that would remove the confound, without which the palette claim cannot be separated from the zlib level.
LightEngineComparisonBenchmark .falco .minestom	lightSources 1, 8, 64 × occlusionPercent 0, 30 × emissionMix UNIFORM, MIXED	By how much each light engine wins, and on what shape of section. All three parameters are needed: the margin moves with each of them.
LightEngineStageBenchmark .falcoReadStates .falcoOpacity .falcoPropagate .falcoCollect .falcoFull .minestomQueue .minestomFull	lightSources 1, 8, 64 × occlusionPercent 0, 30	Why one of them wins, which the comparison never says. It splits the Falco path into reading the palette, building the opacity table, searching and packing, and the built-in path into building the seed queue and the rest. This is what identified the allocation the opacity table used to make, and it is the source of the stage table in Comparing the light engine with Minestom's.

emissionMix decides whether every source of the section emits level 15 (UNIFORM, glowstone throughout) or whether the sources differ (MIXED: glowstone 15, lantern 15, torch 14, redstone torch 7, magma block 3, at the same positions drawn from the same seed). The Falco search assumes its queued positions are ordered by level, which only holds while every source starts at the same one, so this parameter is what would show whether a bucket queue is worth adding. One cell of the cross product measures nothing: with a single source MIXED places glowstone as well and is a duplicate of UNIFORM. The recommended run therefore leaves it out — the commands are in Reproducing a published table.

The comparison verifies the two engines agree before it measures them. Its @Setup runs both paths over the section it just built and aborts the trial when the 2048 bytes differ, so a faster number cannot come from computing something else. LightEngineEquivalenceTest pins the same property down in the normal test run, over 54 scenarios. Both are recent: the byte identity was stated in How the light engine works long before anything in the build checked it.

Because three of them start a server, their absolute numbers include registry time and are not comparable with the library benchmarks above. Compare them only against their own counterpart.

Scaling beyond the sizes anyone uses

ScalingBenchmark measures this library against itself rather than against Minestom, along two axes that the other benchmarks sample too coarsely to expose a bend in:

Method	Parameter	What it answers
blockLightBySectionCount skyLightBySectionCount	sectionCount 1 … 256, fifteen steps	Whether cost per section stays flat as the world grows taller. Block light does across the whole range. Sky light does not: a least-squares fit over the vanilla range (≤ 24 sections) understates the measured cost at 256 sections by about 20 %, while the same method lands within 2 % for block light.
paletteByDistinctStates packingByDistinctStates	distinctStates 1 … 1024	The same question for the codec as a section fills up.

Fifteen steps rather than three, because the point of this benchmark is to find where a curve stops being straight — and that is exactly what a coarse parameter set hides.

Two things about this class a reader should weigh before quoting it. It is the only class in the harness that measures three iterations rather than five, which makes its intervals the widest in the suite for a given dispersion — and no table from it is published, so those intervals cannot be inspected. And the extrapolation claim is a fit with unpublished residuals extended ten times beyond its support: "about 20 %" is as precise as it can honestly be stated. The durable part of the finding is the mechanism rather than the number — sky light seeds a queue at every open cell, so its work grows with the volume it can see, and that is why the curve bends where block light's does not.

The class also contains an unclaimed repeat. sectionCount and distinctStates share one state class, but blockLightBySectionCount and skyLightBySectionCount never read distinctStates, and the two codec methods never read sectionCount. Every light measurement in the class is therefore already performed five times over, and every codec measurement fifteen times, each as an independent JMH trial with its own warmup. The spread across those repeats is a direct empirical estimate of exactly the run-to-run variability the single-fork objection is about, and it costs one run and no code change to recover.

Notes on the setup

Piece	Version	Why
me.champeau.jmh	0.7.3	Latest release. Gives the benchmarks their own source set so JMH never lands on the main or test classpath of a published library.
org.openjdk.jmh:jmh-core	1.37	Latest release. Not managed by mycelium-bom — the BOM covers adventure, minestom, cyano, junit and mockito only — so both versions are pinned explicitly in settings.gradle.kts.
net.kyori:adventure-bom	5.1.1	The main source set gets adventure through compileOnly(minestom), which never reaches a runtime classpath. The benchmarks run their code for real and need adventure at runtime, so they import the platform directly. Keep this in sync with the version Minestom resolves to — check with ./gradlew dependencyInsight --configuration compileClasspath --dependency adventure-nbt.
net.minestom:minestom	not pinned here	Declared withoutVersion() and resolved through mycelium-bom 1.7.2. The published comparisons were measured against 2026.06.20-26.1.2; a republication of the BOM moves the Minestom side without any commit in this repository.

The plugin generates the harness classes with its bytecode generator. No jmhAnnotationProcessor is declared on purpose: with both present, the two generators emit the same classes and the jar ends up with two copies of every benchmark.

On Java 25, JMH 1.37 prints a warning about sun.misc.Unsafe::objectFieldOffset being terminally deprecated. It is harmless and comes from JMH itself, not from this repository.

Related: How-to Run the JMH benchmark suite · Explanation What the benchmarks establish · Reference Measured results