Skip to content

Benchmarks and Demo

TheMeinerLP edited this page Aug 1, 2026 · 3 revisions

falco-benchmarks and falco-demo are the two modules that are never published (see Publishing for how the other four modules are). Both exist purely as tools for people working on or evaluating Falco itself.

falco-benchmarks

This module exists because ScalingBenchmark measures the Anvil loader and the light engine in one run, and because BenchmarkConstants and SectionStates are used from both sides. With a jmh source set inside each library module, those would either have to be duplicated or the two library modules would have to depend on each other just for benchmarking. Keeping the benchmarks in their own module also keeps the jmh Gradle plugin, and everything it pulls in, out of the published artefacts entirely.

dependencies {
    jmhImplementation(platform(libs.mycelium.bom))
    jmhImplementation(platform(libs.adventure.bom))
    jmhImplementation(project(":falco-anvil"))
    jmhImplementation(project(":falco-light"))
    ...
    jmhImplementation(libs.minestom)
    jmhImplementation(libs.fastutil)
}

Minestom is needed here by the comparison benchmarks, which measure Falco's code against the engine Minestom ships with. The other benchmarks avoid depending on Minestom on purpose, so that what they measure is this library's own code and not, say, a registry lookup somewhere in Minestom's plugin machinery.

No JMH annotation processor is declared, deliberately. The JMH Gradle plugin already generates the benchmark harness classes with its own bytecode generator; declaring the annotation processor as well would make both mechanisms emit the same classes, leaving the jar with two copies of every benchmark.

jmh {
    jmhVersion.set(libs.versions.jmh)
    includeTests.set(false)
    ...
    val include = providers.gradleProperty("jmh.include").orNull
    if (include != null) {
        includes.set(listOf(include))
    }
}

includeTests is off because the tests of this project need a Minestom server, which a benchmark jar cannot start. A full benchmark run takes the better part of an hour, so a single benchmark needs to be reachable without editing the build file — -Pjmh.include='BitPackerBenchmark.pack' does that.

Finally, the benchmark sources are still compiled by a normal build, just never executed by one:

tasks {
    named("check") {
        dependsOn(named("compileJmhJava"))
    }
}

A benchmark that stopped compiling after a refactoring elsewhere in the codebase should fail the build like any other source set would.

falco-demo

falco-demo is a runnable comparison of the Falco and the Minestom chunk loader on the user's own world. It is deliberately absent from the root build's publish list, for the same reason as falco-benchmarks: it is a tool for someone who wants to check the loader's performance claims on their own hardware, not something a consumer of the library should ever resolve as a dependency.

dependencies {
    implementation(platform(libs.mycelium.bom))
    implementation(project(":falco-anvil"))
    implementation(project(":falco-light"))
    implementation(project(":falco-instance"))
    implementation(libs.minestom)
    ...
}

Unlike the library modules, where Minestom is compileOnly because the consumer picks the version (see Dependency Management), here it is the opposite: this module starts a real server and calls the loader Minestom ships with, so it needs Minestom on the runtime classpath, which implementation provides.

falco-light is included because the Falco server hands its instance a FalcoLightingChunk through ChunkLightScheduler — the half of the stack a pure loading benchmark cannot show, and whether the light of a streamed-in chunk looks right is usually the first thing anybody notices while flying around. falco-instance is included even though nothing in the demo runs it: FalcoInstance is named in the log output and in the Javadoc of ServerStack as the component this demo deliberately leaves out, and a reader following that explanation should land on the real type rather than a string that was accurate only when it was written.

The measurement tasks

tasks.register<JavaExec>("runFalcoLoader") { configureDemo("falco") }
tasks.register<JavaExec>("runMinestomLoader") { configureDemo("minestom") }

Both run tasks execute the identical class with the identical measurement; the loader is the only thing that differs between them. Anything else would make the comparison meaningless, so the shared configuration lives in one place (configureDemo) and the two task registrations only pass a different --loader.

A few details inside configureDemo are worth calling out:

  • The JVM used for the run is pinned explicitly through a JavaToolchainService launcher rather than whatever JVM happens to run the Gradle daemon, so the measurement doesn't carry an unreported variable.
  • The world directory is passed as an explicit system property (falco.demo.world) rather than relied upon via the working directory, so the demo doesn't silently depend on how it was launched (a run started from an IDE would otherwise look in a different place).
  • Options such as threads, chunks, warmup, rounds, and dimension are read as Gradle properties, so a run reads ./gradlew :falco-demo:runFalcoLoader -Pthreads=8.
  • isIgnoreExitValue is deliberately left at its default (false) for the measurement tasks: the demo exits zero for everything it handles itself, including a missing world, so a non-zero exit status really is a defect and should fail the build rather than scroll past unnoticed.

The server tasks

tasks.register<JavaExec>("runFalcoServer") { configureServer("falco") }
tasks.register<JavaExec>("runMinestomServer") { configureServer("minestom") }

This is the second half of the module: a joinable server somebody can judge by eye, sharing the world, toolchain, and option style with the measurement tasks above — for the same reason those two share theirs. The comparison is only meaningful if the two servers differ in the stack (--stack=falco vs. --stack=minestom) and in nothing else, so everything except --stack is configured once in configureServer.

The view distance is set as a system property, not a command-line option read by the demo's own argument parser:

val viewDistance = providers.gradleProperty("viewDistance").orElse("10")
...
systemProperty("minestom.chunk-view-distance", viewDistance.get())

This is forced by Minestom itself: ServerFlag.CHUNK_VIEW_DISTANCE is a static final field read from the minestom.chunk-view-distance system property when the class is initialised — which happens before main() gets a chance to apply anything. A -PviewDistance value that only reached the demo's own option parser would be logged correctly while the server itself silently used a different, default value.

Unlike the measurement tasks, isIgnoreExitValue is set to true for the server tasks. The normal way to stop a long-running server is a signal (Ctrl-C), and a signal leaves the JVM exit code at 130 or 143 regardless of how cleanly the process shut down. Without ignoring that exit value, every session stopped with Ctrl-C would end in a red "BUILD FAILED" printed underneath the shutdown lines that actually say the loader closed properly. A genuine crash still prints its stack trace to the same console, which is the output anyone watching a running server would be reading anyway, so nothing is lost by ignoring the exit code here.

Finally, falco-demo/build.gradle.kts accesses sourceSets and javaToolchains through extensions.getByType<...>() rather than through Kotlin's generated type-safe accessors. The java-library plugin is applied to this module from the root build's subprojects block (see Build Setup) rather than from a plugins { } block in this file, and relying on accessor generation across that boundary is exactly the kind of thing that silently breaks on a Gradle upgrade.

Falco


Start here

The measured record

Why it is built this way

Working on the build

Clone this wiki locally