Add addon version, feature adoption and command failure metrics#3038
Merged
Conversation
Six new bStats charts, plus a cleanup of three that were sending data into the void. Addon and game mode versions (drilldown pies) and addon API versions (advanced pie): we knew *that* a server ran Level, but not *which* Level, so there was no way to see how long the tail is after a release or judge when an API break is safe. Non-default settings (bar chart): only deviations from default are reported, so every bar is a signal. A default-off setting appears under its own name when enabled, a default-on setting as no-<name> when disabled. Comparing a bar to the server count gives the adoption or rejection rate of a feature. Command failures and top failing commands (#3022): CompositeCommand.call is the single funnel where every failure mode is distinguishable, so it now records the kind of failure and the command it happened on. Only the failure kind and a stable command key are sent - never what the player typed - because bStats data is public. Labels cannot be used as that key: the top level label comes from the game mode config and getLabel() is overwritten with whatever alias was typed. The permission is hard coded in setup() and carries the addon prefix, so bskyblock.island.team.invite means the same thing on every server. Commands with no permission fall back to their class name. Distinct keys are capped at 200 between submissions so an addon with generated commands cannot grow the map without limit. Removed the addons, gameModeAddons and hooks advanced pies. Those chart ids do not exist on bstats.org, so the data was discarded on arrival. The addonsBar, gameModeAddonsBar and hooksBar charts carry identical data and do have charts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ATRqygFLZ5CA3zrWmutx7H
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Adds six bStats charts and removes three that were sending data nowhere. All six chart IDs already exist on bstats.org/plugin/bukkit/BentoBox/3555, so data starts landing ~30 minutes after servers update.
Closes #3022.
New charts
addonVersionsgameModeVersionsaddonApiVersionsfeaturescommandFailurescommandFailurePathsAddon versions. We knew that a server ran Level, not which Level. That was the biggest blind spot: no way to see how long the tail is after a release, or to judge when a breaking API change is safe.
Non-default settings. Reports only deviations from default, so every bar is a signal rather than a restatement of the defaults. A default-off setting appears under its own name when a server enables it (
chunk-pregen); a default-on setting appears asno-<name>when disabled (no-brigadier-commands). Comparing a bar to the total server count gives that feature's adoption or rejection rate. 24 settings covered, including all four dialog toggles, both did-you-mean toggles and Brigadier — so we can finally see whether servers are switching off recently shipped work.Command friction (#3022).
CompositeCommand.call()is the single funnel where every failure mode is already distinguishable, so it now records which kind occurred and on which command.UNKNOWN_SUBCOMMANDis recorded only when did-you-mean actually produced a suggestion — the bare "leftover args and the command has subcommands" condition is too loose to be trusted, since plenty of commands legitimately have both. Everything else falls through and is classified asCANNOT_EXECUTEorBAD_ARGSnaturally. That slice is therefore a lower bound, and it is gated ondid-you-mean-subcommandsbeing enabled.The command key
Labels cannot be used to key this data. The top-level label comes from the game mode config, and
getLabel()is overwritten bygetCommandFromArgs()with whatever alias the player typed — so the same command would key differently across servers, and differently on the same server depending on typing.The permission is hard-coded in
setup()and carries the addon's prefix, sobskyblock.island.team.invitemeans the same thing everywhere.getStatsKey()uses it, prependingbentobox.for core commands (which have an empty prefix) and falling back to<addon>.<ClassName>for commands with no permission.Privacy
Only the failure kind and that stable key are submitted — never arguments, player names or anything typed. bStats data is public, so this is the line to hold. Distinct keys are capped at 200 between submissions and only the busiest 20 are sent, so an addon with generated command names cannot grow the map without bound; failure counts by kind are never dropped.
Removals
The
addons,gameModeAddonsandhooksadvanced pies are gone. Those chart IDs do not exist on bstats.org — verified against/api/v1/plugins/3555/charts— so every submission was discarded on arrival.addonsBar,gameModeAddonsBarandhooksBarcarry identical data and do have charts. This is why the Hooks pie never showed anything.Testing
./gradlew buildgreen: 3366 tests, 0 failures (+11 new).BStatsTest— accumulation by kind and by command, the 200-key cap, and that capped commands keep counting while new ones are dropped.CommandFailureStatsTest— each failure path records the right kind and key, successful commands record nothing, andgetStatsKey()for both the permission and class-name cases.Also diffed every registration in
BStats.javaagainst the live chart config: 26 IDs, each either type-matched or a bStats built-in, zero mismatches in either direction.🤖 Generated with Claude Code
https://claude.ai/code/session_01ATRqygFLZ5CA3zrWmutx7H