In Server Conformance

In-Server Conformance

What this page covers: ECoreConformance - the plugin that runs the core's conformance suites inside a real server, and the driver that runs it across the whole server matrix. It is this repository's own check-up: it is never shipped with the core and no published artifact carries it.

The unit suite proves the core against doubles. Some questions only a live server answers: does a tag written on an item survive on 1.7.10, does a GUI window open with the size the client expects, does a placeholder resolve through a real PlaceholderAPI, which platform did the core detect. ECoreConformance asks those questions where they happen.

---

The module

conformance/
  common/      :conformance:common     the contract: suites, cases, verdicts, the runner, the
                                       ENCTEST marker writer and the NDJSON report - no Bukkit
  minecraft/   :conformance:minecraft  the Bukkit plugin ECoreConformance (/ecoreconformance, /ecc)
  hytale/      :conformance:hytale     the Hytale plugin EverNife:ECoreConformance (/ecc), Java 25
  scripts/     conformance-matrix.py   the matrix driver (stdlib Python) and its offline test

./gradlew :conformance:minecraft:shadowJar   # conformance/minecraft/build/libs/ECoreConformance-3.0.1.jar
./gradlew :conformance:hytale:shadowJar      # conformance/hytale/build/libs/ECoreConformance-3.0.1-Hytale.jar
./gradlew :conformance:common:test :conformance:minecraft:test :conformance:hytale:test

The plugin is an ECBukkitPlugin with depend: [EverNifeCore]; its commands go through the core's own @FinalCMD framework. Its jar carries the same relocations as the deployable core (one applyCoreRelocations in the root build file) and the three *Conformance classes of evernifecore-common-tests (without JUnit). checkLinkage scans the module like the rest of :minecraft, so a suite that calls an API a supported server lacks fails the build unless the call is guarded by version and allowlisted.

The Hytale plugin is a JavaPlugin whose manifest.json depends on EverNife:EverNifeCore; it reuses :conformance:common (suites, runner, markers, NDJSON) and goes through the same @FinalCMD framework.

The default build never needs the test framework that drives the matrix: the modules depend only on the core.

---

Running it by hand

Drop EverNifeCore-3.0.1.jar and ECoreConformance-3.0.1.jar into plugins/, start the server, then:

command	what it does
/ecc list	the suites, in the order run all takes them
/ecc run <suite|all> [--player <p>] [--cid <id>]	runs the suites' main cases
/ecc write <suite|all> ... / /ecc read <suite|all> ...	the halves that must survive a restart (nbt, playerdata): write, restart, read
/ecc screen <screen> <player>	opens one sample screen and leaves it open, for a client to look at
/ecc click <player>	hands the player the selection wand and arms the recorder for one block click
/ecc actions <player>	subscribes to the command and break posts and records that player's, for the actions suite
/ecc text <player>	sends the text-metrics lines, for a client to read back what it drew

Cases that need a player are SKIPPED without --player, and a skipped case never fails a suite. --cid tags every marker of the run (one is minted when omitted).

suite	what it checks
items	the selection wand, an item tag line and setUnbreakable survive on the real server
nbt	FCNBTUtil end to end - text, items, entities, players, tile entities, block and chunk data, files; write/read across a restart
platform	PlatformConformance; world-thread-next-tick (from the main thread, runOnWorldThreadNextTick runs its task on the main thread a tick later, counted by a timer of the case's own - 1.7.10 has no tick number); ECEventConformance (the region events) and EconomyConformance against Currencies.money() (SKIPPED, "no economy", on a server without one)
version	detected version, server software, the NMS revision with the two strings it was resolved from, world height, action bar, PlaceholderAPI detection
commands	registration, dispatch and tab-complete on the real command map, registering twice, /list owned by the core
screen	19 screens: open, window type and size, title, close seen once as PLAYER_CLOSED; plus what each exists to show - a pushed and popped screen, a chat prompt that times out, an anvil prompt the server closes (prompt-anvil: kind=ANVIL, CANCELLED, the screen back without an onClose, level and inventory unchanged; CHAT/TIMEOUT where NMSUtils.isSupported() is false), an editable area, a page turned, the seeded layout, ConfirmScreen, a seeded delivery claimed from DeliveriesScreen, a three-entry RankingScreen, an InputField asked in chat, a ProgressBar drawn at 50%, a Choice gated by a permission the player lacks drawn disabled, and a dependsOn button that holds item-pickup needed only while open
scheduler	main-thread hand-off from async, next tick and sync runs never inline
playerdata	a PDSection written before a restart reads back after it (write/read only)
events	an ECEvent reaches the bus and the Bukkit @EventHandler once; Forge detection matches the server
selection-click	a real block click of the wand reaches the selection listener
particles	the outline renderer draws every style
text-metrics	glyph widths, chat-line width and centering; /ecc text lets a client read the lines back
hooks	PlaceholderAPI present/absent, and without Vault Currencies.money() is unavailable and a take answers UNAVAILABLE without throwing
regions	the core's flags registered, a plugin's own event source followed by need both ways, a denied break, a bot found inside a region that starts denying entry (expelFoundInside) sent to its fixed point, flag settings across a restart (write/read), and for each of the sources fire-spread, respawn, item-drop, item-pickup and vehicle its listener off while no region needs it and on the server once one does (LegacyPickup on 1.7.10)
nms	the embedded ECNmsUtils bridge: item registry names both ways, localized name, handle, tool and armor slots, entity and world round trips, fake player, the item dictionary in the revision's own names (logWood up to 1.12.2, minecraft:logs after)
logic	the stock variables registered and exp-level/stat supported; a gate of permission + compare exp-level >= 0 passes for the bot; exp-level and exp-points read back what was set (5, 37) on every era; a command effect as: console runs (say); gamemode with scope: held switches the bot and gives its mode back on release; daily at 04:00 in the server zone started within the last day
actions	no core handler of PlayerCommandPreprocessEvent before the first CommandActionEvent.Post subscriber, one once it subscribed, and the bot's /ecc list heard with ecc among the names; the bot's block break heard as BreakActionEvent.Post (the driver runs /ecc actions, then the bot acts, before run all)
deliveries	a give-item of a stack of diamonds lands in the bot's inventory as Bukkit reads it and leaves nothing pending; with the 36 main slots full the same effect leaves one delivery from the case's caller, which the case purges - the inventory is given back as it was
hud	getHud().supports(c) for each of the 8 capabilities against the version table (NecroTempus for action bar and title on 1.7.10); a title and a boss bar shown and hidden on the bot where supported; the sidebar set and cleared; a toast drawn twice with the same text loads one advancement and the repeat reuses it (the server never unloads an advancement)
quests	QuestsSuite, over the plugin's in-memory engine: three BlockBreakEvents of the bot complete ecc-mine (count 3) with BREAK listened only while it is active; a stack of 5 dropped at the bot's feet counts 5 when picked up; `/ecc quest start
npcs	no NPC plugin is installed: the providers citizens, fancynpcs, znpcsplus are registered, each available exactly when its plugin is enabled (none here), provider:id round-trips lowercased, lookups of an unknown NPC answer empty without throwing, and interact-npc is served with its Pre; the boot leaves no linkage error (the reflection names the plugins only as strings)
groups	a party of the conformance plugin with the bot: glow-members TRUE makes the bot glow (Player.isGlowing(), read by reflection) from 1.9 on, and below it /ecore group reports glow-members: unsupported on this platform (the case fails if the version and the provider disagree); the group's deny-friendly-fire resolves TRUE at the bot through the server's lookup, the bot shares the group with a second (offline) member and the DAMAGE source is served - the hit between two real players stays in the unit suite, the matrix runs one bot per server; max-members refuses a third member and the bot's join of a full party says is full; /ecore group <bot> (CoreGroupCommands.show) lists the party by its id

The sample screens by hand

/ecc screen <screen> <player> builds the screen for a person at the keyboard: it answers where the check only measures, and the check itself still reads the values it always read. What each one shows:

command	what a person sees
/ecc screen basic <player>	a diamond with a two-line lore, red wool coloured by data, paper; nothing can be taken
/ecc screen hopper <player>	a hopper window with an emerald in slot 2
/ecc screen dispenser <player>	a dispenser window with an emerald in slot 4
/ecc screen workbench <player>	a crafting-table window with an emerald in slot 5
/ecc screen nav <player>	the chest opens a second screen; closing that one brings the first back
/ecc screen storage <player>	a row of stones ("border - not takeable") over two editable rows kept per player: what is left there is back on the next open, until a restart
/ecc screen prompt <player>	the paper asks in chat (5 s); the screen comes back after the answer or the timeout
/ecc screen prompt-anvil <player>	the paper asks "how many?" on an anvil (chat without the NMS bridge); a number is accepted and said back in chat, anything else is refused with the way out
/ecc screen pager <player>	30 numbered papers over two pages, turned by slots 18 and 26
/ecc screen title-long <player>	a 40-character title, cut to 32 up to 1.8
/ecc screen layout <player>	the layout seeded to guis/SeededLayout.yml: an emerald in the middle, panes in the corners
/ecc screen confirm <player>	ConfirmScreen: CONFIRM and DENY each close the window, and nothing can be taken after the click
/ecc screen deliveries <player>	DeliveriesScreen with a diamond delivery queued just before it opens; a click claims it into the inventory
/ecc screen shop <player>	the ecoreconformance:shop menu: left buys a diamond for 10, right sells one back for 4 - in money when an economy (Vault) answers, otherwise in tokens, 100 of them given on every open
/ecc screen ranking <player>	RankingScreen of alpha, beta, gamma, each entry with its score in the lore
/ecc screen input <player>	the name tag asks in chat for 30 s and says back what was typed, or that the time ran out
/ecc screen progress <player>	a bar that starts at 50% and fills a tenth every second, starting over after 100%
/ecc screen choice <player>	the open choice says "choice picked" in chat; the gated one is drawn disabled, its reason naming the missing node
/ecc screen reactive <player>	a hopper named "Re-checked N x": drop an item and pick it up, and each pickup redraws it one higher

On Hytale

Drop EverNifeCore-3.0.1-Hytale.jar and ECoreConformance-3.0.1-Hytale.jar into mods/, start the server, then ecc list and ecc run <suite|all> [--player <p>] [--cid <id>] from the console. There is no write/read half, no GUI and no client: the cases fire the game's own ECS events themselves (store.invoke(ref, event)), with the player found by name.

suite	what it checks
platform	PlatformConformance against HyPlatform (the same check the Bukkit side runs: classpath probe, the IHudAccess refusals, the forbidden nulls, the thread hops); world-thread-next-tick (from the default world's thread, runOnWorldThreadNextTick runs its task on that thread in a later World.getTick()); the core detected HYTALE; the region SPIs (wand, outline) are the real ones; getHud() answers title, toast, tracker and sound (hud-capabilities); the region Systems take a second register as a no-op; a player's /ecc whoami through sudo arrives as that player and as a CommandActionEvent.Post (raised by FinalCMD: the server has no command event); FISH is not served and requireServed names Hytale
regions	a source followed by need (no region: the System decides nothing; with one: inside cancelled, outside not); a break by an NPC denied inside; an explosion keeping the inside of deny-block-break; deny-entry reverted with its deny-message; a player found inside a region that starts denying entry (expelFoundInside) sent to its fixed point; the wand closing a shape; a denied spawn that never ticks; one deny-message for a run of hits; the outline sent; a real death (DeathComponent, then the respawn the client's button asks) landing on the region's spawn-point; a hoe's Secondary chain started with its block in the chain data, failed at its start inside deny-interact and run outside
cost	median server tick with the move System needed, and the client-prediction measurement (native rule volumes against the core's own path, 100 regions)
logic	the stock variables registered, with exp-level, exp-points, food and stat unsupported on Hytale; a gate on compare health > 0 passes for the player; a command effect as: console runs; gamemode with scope: held switches between adventure and creative and gives the mode back; daily at 04:00 in the server zone - each player case on the player's world thread
deliveries	the same pair as on Bukkit with hytale:Rock_Stone: it reaches the hotbar and storage, and with every slot full it leaves one delivery; inventory-readers: five stone in the bot's active hotbar slot are what getHeld reads, getContents lists them, and roomFor has room that is gone once every slot is full - on the player's world thread
groups	deny-friendly-fire and max-members as on Bukkit, on the player's world thread; no glow provider on Hytale, so no glow case
quests	HyQuestsSuite: three real BreakBlockEvents complete ecc-mine; native-marker-accepts-runtime-asset loads an objective asset made at runtime (off the world thread) and starts it for the player - what hytale-native: {markers: true} rests on; a refusal is skipped with the server's reason
npcs	the hytale provider registered and always available, provider:id round-trips, an unknown uuid is found nowhere on the world thread, and interact-npc is served with its Pre
screen	the page renderer: the page reaches the player's PageManager; a click and an escape played through PageManager.handleLegacyCustomPageEvent reach the button's handler and close as PLAYER_CLOSED; a state change goes out as a page update; view.close() and a second screen close as REQUESTED; a screen with an editable area is refused naming STORAGE_SLOTS; a question is asked on a page and answered through its field. The synthetic player swallows the pages, so none of this proves that a client DRAWS them: that is checked with a real client and /ecc screen sample <player>

---

What a run reports

Every case comes out three ways.

A human line in the server log:

[ECoreConformance] nbt/item-pdc REFUSED expected=true since=1.14 ...
[ECoreConformance] nbt summary 12/12 (0 skipped) on 1.7.10 ...

An ENCTEST marker - one flat JSON object per line, the protocol a test harness reads off the log:

ENCTEST {"v":1,"e":"suite_start","t":<ms>,"cid":"<cid>","runId":"<r>","suite":"ecore-items","total":3}
ENCTEST {"v":1,"e":"test_result","t":<ms>,"cid":"<cid>","runId":"<r>","suite":"ecore-items","test":"wand","ok":true,"durationMs":1,"error":null}
ENCTEST {"v":1,"e":"suite_end","t":<ms>,"cid":"<cid>","runId":"<r>","suite":"ecore-items","passed":3,"failed":0,"skipped":0,"durationMs":49,"report":"reports/<r>.ndjson"}

And the same events in plugins/ECoreConformance/reports/<runId>.ndjson, one per line, suite_end last.

The verdicts: PASS, FAIL, REFUSED and SKIPPED. A REFUSED is a call the server turned down; it is ok only when the refusal was expected. Version-dependent cases go through Verdict.since(...), which writes expected=<is a refusal expected here> since=<version> into the detail and turns a call that quietly succeeds below its version into a FAIL - the version table is wrong then. A case that throws is a FAIL naming the exception. A SKIPPED case emits no test_result; it is counted in suite_end.skipped.

---

A new suite

public final class MySuite implements ConformanceSuite {

    @Override
    public String getId() {
        return "my";            // bare id; markers say "ecore-my"
    }

    @Override
    public List<ConformanceCase> getCases(ConformanceContext ctx) {
        return Arrays.asList(
                ConformanceCase.of("answers-now", () -> Verdict.of(somethingHolds(), "detail")),
                ConformanceCase.later("answers-next-tick", () -> futureVerdictWithItsOwnTimeout()));
    }
}

Then one more entry in Suites.all() - the registry is a list, never a classpath scan. Rules that keep the suite honest:

• getId() and getCases(...) must not touch Bukkit: the registry is built in the headless command tests. Only the case bodies run against the server.
• A case that answers on a later tick uses ConformanceCase.later and completes its own future, timeout included; the runner waits as long as it takes.
• A version-dependent case uses Verdict.since(...) with a *_SINCE constant at the top of the suite.
• A case that needs a player goes through PlayerCases, which skips it when there is none.
• Write/read pairs override getWriteCases/getReadCases.
• A failing case is a defect to explain - in the core, or in the suite's expectation, corrected with the value read on the server. Never loosen a case until it passes.

---

The matrix driver

conformance/scripts/conformance-matrix.py runs the plugin across the server matrix through the test framework's command-line client, one call per step. For each Minecraft flavor (crucible-1.7.10, mohist-1.12.2, mohist-1.16.5, arclight-1.21.1, paper-1.21.1):

1. boots the server with the core jar from this tree, ECoreConformance and the harness (plus PlaceholderAPI on paper-1.21.1); a routed plugin that does not enable fails the flavor;
2. joins a headless player of the matching client version - its client boots beside the server's boot (mc_start with wait=false, then mc_hmc_start on the still-starting server, which connects once the server is healthy) - then reads the boot's plugins and linkage_errors from mc_status, ops the player, arms /ecc click and has it click a block;
3. ecc run all, then ecc write all;
4. opens each of the 18 screens of the driver's SCREENS (the suite's 19 minus shop) and records both sides: what the server says is open (screen/<screen>.server.json) and what the client drew (screen/<screen>.client.json);
5. restarts the server, reconnects the player, waits until the server sees it online, ecc read all;
6. collects the markers through the framework's event reader into events.ndjson, copies the plugin's NDJSON reports, and stops everything.

On crucible-1.7.10 the driver also drops the WorldEdit-GTNH mod into mods/ (the only WorldEdit there that keeps tile entities), so the boot warning for a 1.7.10 server with the WorldEdit plugin and no mod never shows in the matrix: that warning is pinned by the unit test WorldEditForgeTest (WorldEditForge.bootWarning), not by a server.

The Hytale flavor hytale-0.7.1 is run only when named (--flavors hytale-0.7.1) and goes another way:

1. boots the server with the core, ECoreConformance and the framework's Hytale harness routed into mods/; the flavor fails unless EverNife:ECoreConformance enabled;
2. joins the framework's synthetic player (player_join) - there is no headless Hytale client;
3. ecc run all --player <bot> and waits for the eight suite_end markers;
4. reads what only the player's side shows - the deny-entry message, the count of break denials, the outline's particle packets, the screen's CustomPage packets (an open and at least one update) - off the framework's view of the player;
5. copies the [ECoreConformance] <suite>/<case> lines of the server log into the flavor's summary.md and the NDJSON reports, and stops the server. There is no restart.

The harness answers each teleport for the synthetic player, as a client would; the suite only reads whether one is still pending (HyWorld.isTeleportPending) - see Gotchas and Pitfalls.

Flavors in parallel

Every flavor runs in its own worker, all of them at once by default; --parallel N caps how many run together (--parallel 1 is one after the other). The results, the summary.md and the exit code come out in the order the flavors were asked, and a flavor whose worker throws is a failed flavor naming the exception - the others carry on. The driver keeps the framework's daemon from being stampeded:

• boots (mc_start, mc_hmc_start) reach the daemon at least 12 s apart;
• a boot refused for the daemon's budget (START_LIMIT_REACHED, CLIENT_LIMIT_REACHED) is a wait, not a failure: it is asked again every 10 s for up to 30 min;
• the end of a half waits on mc_await for the next suite_end of its cid, not on a poll;
• a GUI screen is read as soon as the server has the container open and the client drew it (or said why it cannot), polling every 0.5 s for up to 5 s;
• a headless client that dies in its own join (seen under load: a NeoForge network-filter race, the 1.7.10 GuiConnecting NPE) is replaced once, and so is one that does not come back after the restart;
• a server whose boot fails ends its flavor with the boot's own failure (mc_status last_failure), never with a second client;
• on Hytale the bot is the first and only synthetic join: the harness's harness_ready already waited for the default world's spawn chunk.

All six flavors together take five servers, five clients and the Hytale server at once, so the daemon needs room for them: MCTF_MCP_MAX_RESERVED_GB = 42, MCTF_MCP_MAX_RUNNING = 6 and MCTF_MCP_MAX_HMC_RUNNING = 6 in ~/.mctf/config.json, read only when the daemon starts (relaunch it with no server up after changing them). With that, the six flavors run in about 5-7 minutes instead of the ~21 of one after another. If the machine cannot hold six, --parallel 3 halves the load.

Each flavor writes <out>/<flavor>/driver.log: what its worker did and how long each stage took (start+join, run, write, gui+text, restart, read, collect, stop; Hytale: start, join, run, collect, stop), the same lines it echoes to stdout prefixed with the flavor.

The driver reads two environment variables and refuses to start (exit code 2, before any call, --dry-run included) when either is missing:

Variable	Value
MCTF_HOME	the test framework's root; its final-artifacts/ carries the harness and the companion mods
MCTF_CLI	the framework's mctf.py client

The matrix is a maintainer tool that needs the framework's daemon running; the Gradle build and the unit tests do not need it. The PlaceholderAPI routed into paper-1.21.1 (2.11.6 - 2.11.2 dies in its static init on 1.21.1) comes from a public repository: :conformance:minecraft:shadowJar copies it to conformance/minecraft/build/matrix/PlaceholderAPI.jar, the driver's default --papi-jar.

./gradlew :minecraft:shadowJar :conformance:minecraft:shadowJar
python -I conformance/scripts/conformance-matrix.py                       # all five flavors
python -I conformance/scripts/conformance-matrix.py --flavors paper-1.21.1 --keep

./gradlew :hytale:shadowJar :conformance:hytale:shadowJar
python -I conformance/scripts/conformance-matrix.py --flavors hytale-0.7.1

# everything at once (the default parallelism is the number of flavors asked)
python -I conformance/scripts/conformance-matrix.py \
    --flavors crucible-1.7.10,mohist-1.12.2,mohist-1.16.5,arclight-1.21.1,paper-1.21.1,hytale-0.7.1

Output in build/conformance/: one folder per flavor (events.ndjson, reports/, screen/, summary.md, driver.log) and a summary.md with the suite x flavor matrix of passed/failed/skipped, ending with the matrix wall time the driver also prints last. The exit code is 0 only when every flavor booted and every suite ended with failed=0. --dry-run prints the planned calls and touches nothing; --keep leaves the servers and clients up for inspection.

---

See also

• Testing Toolkit - the headless conformance suites the platform suite reuses
• Version Compatibility - the version branches several suites pin
• Building from Source - checkLinkage and the shadow jars