Skip to content

Testing Toolkit

Jump to bottom
Pétrus Pradella edited this page Oct 23, 2026 · 38 revisions

Testing Toolkit

What this page covers: evernifecore-common-tests - the test apparatus EverNifeCore uses on itself, published so your plugin can use the same one. Platform doubles, the PlayerData/storage lifecycle, the command harness, the senders, the fake plugin, and the conformance suites.

The point of publishing it is that a plugin testing against EverNifeCore otherwise has to invent a fake platform, a fake plugin handle and a fake storage layout - and every invented one drifts from what the core actually does.

Depending on it

repositories {
    maven { url = 'https://maven.petrus.dev/public' }
}

dependencies {
    testImplementation 'br.com.finalcraft:evernifecore-common-tests:3.0.1'

    // The platform classes common declares but does not carry (McFCScheduler and the
    // platoverride family) come from the platform jar itself, on the test classpath.
    testImplementation 'br.com.finalcraft:evernifecore-minecraft:3.0.1'
}

It brings JUnit 5, evernifecore-common and commons-lang3 along as api dependencies - every one of those shows up in its public signatures.

The one thing it cannot bring is the api-contracts stubs: those are compile-time-only placeholders that are never published and never packaged, because each platform ships the real class under the same fully-qualified name - see Platform Abstraction. Putting evernifecore-minecraft (or evernifecore-hytale plus evernifecore-common) on the test classpath is what supplies them.

It is compiled to Java 8 bytecode, like common, so a plugin whose tests run on an older toolchain can still use it.

Bukkit-side tests: evernifecore-minecraft-tests

The deployable evernifecore-minecraft jar relocates its Jackson stack and expects the server to download the rest at boot, so a plugin's Bukkit-side test cannot pair it with evernifecore-common-tests cleanly. For those tests depend on the Bukkit test kit instead:

dependencies {
    testImplementation 'br.com.finalcraft:evernifecore-minecraft-tests:3.0.1'
    testImplementation 'org.spigotmc:spigot-api:<the API your plugin compiles against>'
}

It carries:

  • the Jackson + snakeyaml stack the core downloads at boot, relocated the same way as in the deployable jar;
  • the whole evernifecore-common-tests engine (Platforms, Plugins, FinalCmdTestHarness, ...), rewritten onto the relocated names;
  • br.com.finalcraft.evernifecore.testing.minecraft.ItemWorld - a described server with an item factory, named worlds and the Bukkit config codecs:
try (ItemWorld ignored = ItemWorld.withMetadata(MCDetailedVersion.v1_21_R1)) {
    ItemStack sword = FCItemFactory.from(Material.DIAMOND_SWORD).displayName("&bDoom").build();
    // ...
}

ItemWorld.registerConfigTypes();          // the Bukkit config codecs, once per JVM, after a platform is installed
World lobby = ItemWorld.world("lobby");   // a named world, e.g. for a Location in a config

Its POM brings evernifecore-minecraft and junit-jupiter at compile scope, and commons-lang3, jackson-annotations and log4j-api at runtime (what a server would otherwise supply). spigot-api is provided, so it never reaches you - declare the server API your plugin compiles against yourself.

⚠️ Never put evernifecore-common-tests or evernifecore-common on the same test classpath. Their classes are already inside the kit, rewritten onto the relocated names; a second, canonical copy of the same class names means whichever loads first wins.

Parsers for platform types (Player, World) and the relocated storage drivers (H2/Hikari) are not in the kit. There is no Hytale counterpart and none is needed: evernifecore-hytale is published thin, with canonical names and a POM that brings evernifecore-common, so evernifecore-common-tests runs against it directly.

The doubles are strict by default

ECoreTestWorld world = Platforms.strict().install();

An unconfigured method on a strict double throws, and names the builder call that fixes it. That is the default on purpose: a double that quietly answers "nothing" is a double that agrees with your code instead of describing a server.

Platforms.lenient() is the old no-op behaviour, opt-in.

ECoreTestWorld world = Platforms.strict()
        .platformProviderId("test")
        .onlinePlayers(Arrays.asList(steve, alex))
        .pluginsLoaded("Vault", "PlaceholderAPI")      // installed and running
        .pluginsDisabled("WorldGuard")                  // installed, not running
        .papiPresent(true)
        .actionBarSupported(true)
        .mainThreadInline()          // a "hop to the main thread" runs right here
        .capturingCommands()         // record what the code asks the server to run
        .recordingShutdowns()
        .install();

pluginsLoaded(...) answers true to both isPluginLoaded and isPluginEnabled; pluginsDisabled(...) answers only isPluginLoaded - a plugin that failed to enable or was disabled. It needs pluginsLoaded(...) too, even an empty one. A strict double asked either question without them throws.

install() registers the double as the live platform and hands back an ECoreTestWorld that must be close()d; build() returns the TestPlatform without installing it. close() is strict about the global event bus: a listener watch or native audience that was not there at install() and still is fails the close with an AssertionError naming each one, so a test that leaks bus state is caught by name instead of changing some later class's gate counts. It stops the region singletons (RegionRuntime.shutdown()) first - the engine started them, so they are not the test's leak.

What a TestPlatform records

Accessor What it holds
getCaptured(label) / registrationOrder() / getUnregisteredLabels() command registration and unregistration
getConsoleCommands() every command handed to makeConsoleExecuteCommand, in order
getSenderCommands() every command handed to makePlayerExecuteCommand, as DispatchedCommand(sender, command)
getActionBars() every action bar handed to the platform, as ActionBarSend(player, text)
getInfoMessages() / getLoggedMessages() what was logged through an adapter this platform created
getShutdownReasons() what asked the server to stop
reset() clears all of the above

The three capture lists are the answer to a specific problem: forwarding a command or writing an action bar is the whole observable effect of some code. An alias, a locale sweep or a priority queue has nothing else to assert on, so a double that swallowed the call left the entire feature untested.

Both makeConsoleExecuteCommand and makePlayerExecuteCommand answer true under capturingCommands(): a double that kept the command did take it.

actionBarSupported(...) says whether this server can show an action bar at all - a 1.7.10 server without NecroTempus cannot, and the code that sends one asks before building anything. It does not decide what is recorded: an action bar that does reach the platform is kept either way.

The senders

TestCommandSender console = Senders.console();
TestFPlayerSender steve   = Senders.player("Steve");
TestFPlayerSender alex    = Senders.player("Alex", someUuid);

steve.grant("myplugin.use");
steve.online(false);                    // ... and anything still ticking for them reads this
steve.playedBefore(true);               // hasPlayedBefore() - false until set
steve.hide(alex);                       // steve.canSee(alex) is false from now on, as a vanish

TestFPlayerSender.online(boolean) exists so a test can cover what happens after a player leaves - an auto-terminating queue, a scheduled message with nobody left to send it to. Before it, isOnline() was a hard-coded true and those branches were unreachable.

Both senders keep every message they were sent, so assertions read off getMessages() rather than a captured stream. TestFPlayerSender goes further and reads the rich text back: hoverTextOfMessageContaining(snippet), clickValueOfMessageContaining(snippet), anyMessageContains(snippet), plus assertAnyMessageContains / assertNoMessageSent.

Senders at a point: at(world, x, y, z)

A TestFPlayerSender has no location until a test places it: getLocation() returns null. The command paths never read it; region code does - RegionLookup.resolve(flag, player), at(player), the here verb - so a region test stands the player somewhere first:

TestFPlayerSender steve = Senders.player("Steve").at("world", 5, 64, 5);   // getLocation() answers this from now on
steve.teleportTo(otherLocation);                                         // moves it too, and records it in getTeleports()

Regions: RegionWorld

RegionWorld is one plugin that owns regions: its ECPluginData, an in-memory ECStorage that counts calls (getBackend().callsTo("save")), its RegionManager already loaded, and - shared by every RegionWorld open at once - a region clock that moves only when told and a region scheduler that runs only when told. It needs a platform with mainThreadInline() and worldEvents().

try (RegionWorld prot = RegionWorld.open("FakeProt", tempDir)) {
    Region arena = prot.cuboid("arena", "world", 0, 0, 0, 10, 10, 10);   // an AssertionError names a refusal
    prot.advance(Duration.ofMinutes(6));                                // the clock moves, nothing runs yet
    prot.runDue();                                                      // the due tasks run once (sweeper, writes)
    List<RegionRemovedEvent> removed = prot.recorded(RegionRemovedEvent.class);
}

openUnloaded(...) gives a manager that never loaded - what a plugin sees while its regions are still loading. flush() writes the owner's pending changes now; getLookup() is the one global lookup every manager publishes to, so two RegionWorlds open in one test are two owners over the same lookup.

The Bukkit event pipeline: BukkitEventWorld

Inside this repository (not published), minecraft/src/test/.../testkit/BukkitEventWorld runs Bukkit's real SimplePluginManager over a Server double that answers the main thread and a scheduler that queues, with EverNifeCore enabled on top and its two native audiences on the global bus. install(tempDir) in a try-with-resources (it puts Bukkit.server, the shared handler lists and the global bus back on close()), then listen(type, handler) as a plugin would, subscribe(...) on the bus, registerECListener(listener) the way ECListener.register does, and getPlatform() for what was logged. The on-demand producers, the ProtectionAll bridge and the Bukkit side of the event bus are tested on it.

The fake plugin

IECPluginExtractor extractor = Plugins.fake("MyPlugin", dataFolder);

...answers for a single fake plugin, whatever object it is handed. That is convenient and it is also a lie: a real extractor answers plugin instanceof JavaPlugin, and refuses everything else.

IECPluginExtractor extractor = Plugins.fakeRecognisingOnly("MyPlugin", dataFolder, thePlugin);

...recognises only thePlugin. Anything else is refused by IECPluginExtractor.validateJavaPlugin, which is what registering a stray object looks like on a server - and the branch that says so is otherwise untestable.

Plugins.setLanguage(plugin, "pt_br") sets the language a plugin handle answers with.

The PlayerData / storage lifecycle

PlayerDataWorld world = PlayerDataWorld.with(Storages.h2("test"))
        .sections(CoinsSection.class, StatsSection.class)
        .accountSections(NetworkStatsSection.class)
        .boot(tempDir);

// ... exercise the code ...

world.reboot();     // flush, close, and load again from the same files
world.close();

sections / accountSections derive the section id from the class simple name; section(id, cls) and accountSection(id, cls) state it, for a test that asserts on the storage identity.

Storages writes the storage.yml a real server would have, with the same keys and the same validation:

Factory Backend
Storages.memory() in-memory
Storages.localFile() / Storages.groupedFile() file backends
Storages.h2(dbName) H2

...plus backendId, dataPath, extraBackend(...), extraBackendDisabled(...), networkBackendId, networkLines, withoutNetworkBlock, loadModeAll / loadModeRecent(days), playerdataLines and rawLines for the cases that need a hand-written block. toYaml(baseDir) renders it and writeTo(baseDir) puts it on disk.

No Docker is needed for any of this - the unit-test backends are in-memory, H2 and files.

The command harness

FinalCmdTestHarness harness = Commands.harness("mytest", dataFolder);
Call What it gives you
Commands.harness(prefix, dataFolder) registers commands, dispatches lines, and asserts on the resulting tree
Commands.referenceTree() a four-level reference tree to assert traversal against
Commands.shapeErrors() the whole catalogue of illegal command declarations, as cases
CommandShapeErrors.check(harness) runs that catalogue against a harness

So a plugin that builds command trees tests them without copying a fixture. See Command Framework.

Conformance suites

List<String> problems = PlatformConformance.check(myPlatform);
PlatformConformance.checkArgParsers(plugin, expectedContextualParsers);
PlatformConformance.summarize(myPlatform);

EconomyConformance.check(provider);                          // read-only
EconomyConformance.checkMutating(provider, scratchAccount);  // moves money on an account you name

ECEventConformance.checkAll(MyEvent.class, MyOtherEvent.class);

PlatformConformance is what a new IPlatform implementation is held to. EconomyConformance is the same idea for an IEconomyProvider - see Economy. ECEventConformance holds an ECEvent subtype to the names its platform base reserves; it checks the classes you name, so a new event only becomes guarded once you add it to the call - see Events.

ECEventConformance checks three things: the reserved names (isAsynchronous, getEventName, getHandlers, callEvent), isCancelled/setCancelled on an event that does not implement ECCancellable, and - for the one Bukkit member an event may declare - the shape of getHandlerList. Bukkit finds that method by name up the hierarchy, invokes it static and casts what comes back, so the guard accepts only a public, static, no-parameter, non-void one and names the ECEvent.getHandlerListOf(...) form in the failure line.

Every check/checkAll here reports rather than throws: it returns one line per violation and an empty list when the subject conforms, so the caller decides whether that is an assertion failure or a log line.

These run headless against whatever you hand them. In-Server Conformance runs the same three against the platform a real server registered, inside the server.

Other helpers

Helper What it does
Logs.capture(Runnable) the log lines a piece of code produced, as a list
Logs.captureThrowing(Runnable) the same, letting the exception through
Locales.perPlayerLocale(baseDir) a world where each reader has their own language; reader(name, lang) builds one
Locales.message(plugin, key, langAndText...) a LocaleMessage built inline
Economies.inMemory().balance(uuid, amount).build() a working economy provider; Economies.absent() is none at all
EventBuses.mirroring() / mirroring(handler) an event bus that mirrors into its audiences without being the global one; the second form gives it an ECEventExceptionHandler of its own
EventBuses.installExceptionHandler(bus, handler) swaps the handler of any bus - the global one included - and hands back the previous one to restore
new FailingExceptionHandler() the handler that fails the test: a broken subscriber or watch callback becomes an AssertionError at the post, with the real failure as its cause
new RecordingExceptionHandler() the handler that writes failures down and throws nothing: getFailures() (subscription or watch, event, throwable), getFailureCount(), reset()
new RecordingAudience() a native audience with no native bus behind it: getDispatched(), getDispatchedOf(Class), setHasListeners(boolean), getGateChecks(), getGateCheckLog(), reset()
Plugins.fakePluginData(name, dir) / Plugins.forget(name) a ready ECPluginData for a plugin that exists only for the test - to own subscriptions and watches - and its removal

RecordingAudience implements the audience gate as hasListeners(Class<? extends IECEvent> eventType)

  • the bus asks per class, before any event exists - and records every ask in getGateCheckLog() (event type, thread, the codebase frames that led to it; getGateChecks() is its size), so zero is the proof the bus never even considered the audience, and a non-zero count names who asked. The bus asks on its own too - every listener-watch refresh (addNativeAudience, subscribe) and every post from any thread - so a test measuring one action calls reset() right before it. setHasListeners(...) flips the gate but does not tell the bus by itself: a test driving the listener watches calls bus.refreshListenerWatches() afterwards, exactly as a real native audience does when its own listeners appear or vanish.
ECEventBus bus = EventBuses.mirroring();
RecordingAudience audience = new RecordingAudience();
audience.setHasListeners(false);
bus.addNativeAudience(audience);

List<String> log = new ArrayList<>();
bus.watchListeners(() -> log.add("first"), () -> log.add("last"), SampleEvent.class);
assertEquals(Collections.emptyList(), log);

audience.setHasListeners(true);
bus.refreshListenerWatches();
assertEquals(Collections.singletonList("first"), log);

A broken subscriber fails the test. @ECoreTest installs a FailingExceptionHandler on the global bus for the length of the class and puts the previous handler back afterwards, so a subscriber that throws on the global bus fails the test at the post that drove it - instead of a SEVERE nobody reads. A test whose point is the breakage uses a scoped bus with a RecordingExceptionHandler and asserts on what the handler was handed:

RecordingExceptionHandler recording = new RecordingExceptionHandler();
ECEventBus bus = ECEventBus.create(recording);
bus.subscribe(SampleEvent.class, event -> { throw new IllegalStateException("broken on purpose"); });
bus.subscribe(SampleEvent.class, received::add);

bus.post(new SampleEvent());

assertEquals(1, received.size());                       // the queue behind the failure still ran
assertEquals(1, recording.getFailureCount());
assertTrue(recording.getFailures().get(0).getThrowable() instanceof IllegalStateException);

The rule this toolkit exists to enforce

A double that agrees with the code instead of imitating the server makes the guarded branch unreachable, and the suite stays green while the guarantee rots.

That is not a slogan. Two serious production defects were found in this repository within a day of each other, both invisible to a suite of over 1400 passing tests, and both for the same reason:

  • No GUI screen opened on the 1.7.10 floor. Production needed the Inventory carried by the open event to be the same object that was passed to openInventory. Real CraftBukkit builds its own container and hands out a different wrapper, so the check never matched. The test double built its view over the very instance it was given, synchronously - it could not fail that check even in principle.
  • A stored inventory came back empty from every backend. Production refuses an item write off the main thread. Storage answers on a worker thread, so the decode threw, the value was quietly dropped and the next save wrote the emptiness over the real data. The test double answered isPrimaryThread() by comparing against the test's own thread, so no test could ever be off it.

In both cases the test existed and passed. What was missing was not coverage - it was a double refusing to satisfy the condition by construction.

Two practical corollaries:

  1. A lenient double is the dangerous direction. One that is stricter than the server produces a false red, which somebody investigates. One that is more agreeable than the server produces a false green, which nobody does. This is why Platforms.strict() is the default.
  2. A knob is not enough - the double has to record. A flag that makes a double accept something it then silently discards closes nothing: the branch is reachable and still unobservable. Every capability added to this engine answers a question, it does not merely permit one.

Working in this repository

  • common/src/test and minecraft/src/test are JUnit 5 on the Java 25 test toolchain (main code stays capped at 8).
  • Reach for the engine before writing a @BeforeAll. A test that needs setup the engine does not have is a reason to enrich the engine, then use it.
  • Change the platform, update the engine in the same commit. Compatibility with an older engine is explicitly not a goal - the two move together.
  • Re-running a test task needs --rerun-tasks; otherwise Gradle reports UP-TO-DATE and reuses the previous result, including a SKIPPED from a run when a backend was down.
  • docker-compose.yml starts MariaDB/PostgreSQL/MongoDB/Valkey/Redis for manual testing of the network backends. The unit suite never needs it.

See also

EverNifeCore · Home · made by Petrus Pradella

EverNifeCore

🏠 Home

Getting Started

Commands & Text

Player Data & Storage

Config & Minecraft Systems

Architecture & Reference

Clone this wiki locally