Skip to content

feat(hyperion): a boss bar is an entity with an audience, and the host has two - #1026

Merged
andrewgazelka merged 1 commit into
mainfrom
feat/boss-bar-api
Jul 28, 2026
Merged

feat(hyperion): a boss bar is an entity with an audience, and the host has two#1026
andrewgazelka merged 1 commit into
mainfrom
feat/boss-bar-api

Conversation

@andrewgazelka

@andrewgazelka andrewgazelka commented Jul 28, 2026

Copy link
Copy Markdown
Member

BossEvent reached the wire for the first time in #1018, as one bar, hand
rolled in a game event's adapter: one UUID per player, Add for every change,
and a diff in the game half to stop it resending. That was correct and it was
measured, and it did not survive the next ask, which was two more bars carrying
something that is not a game concept at all.

A bar is an entity and its audience is a set of edges

let bar = world
    .entity()
    .add(BossBar::id())
    .set(Title(Text::text("CPU 240% of 1600%").color(NamedColor::Aqua)))
    .set(Progress(0.15))
    .add((ShownTo, operator));

and later bar.set(Progress(0.31)), which sends one UpdateProgress to each
viewer and nothing else. smash's Server::set_boss_bar is now four sets on
one of these, and the packet it used to build is gone.

The load-bearing choice is not the components; it is where the per-viewer sent
state lives. Sent is the data on the (Sent, viewer) pair, holding exactly
what that client was last told, and it is what makes the two cases that are
usually bugs stop being code:

  • Joining. A player who was not there has no (Sent, bar) pair, so the
    next tick sends them Add. There is no join handler in this change.
  • Leaving. A bar is destroyed, a player disconnects, or an audience
    shrinks. All three end with the pair going away -- flecs does the first two
    itself under (OnDeleteTarget, Remove) -- and one OnRemove observer on
    (Sent, Wildcard) sends the Remove. Teardown cannot leak, because the fact
    that says "this client has this bar" and the thing that gets cleaned up are
    the same fact.

There is no explicit join or leave path anywhere in boss_bar.rs. That was the
test of whether the shape was right.

The change the wire gate found

A change of state usually moves several fields at once, each operation is its
own packet, and between the first and the last the client is holding a bar that
is half one state and half the next. There is no ordering that fixes it: put
the title last and a countdown ticking over draws the next second's fill under
this second's number; put it first and a lobby bar becoming a countdown is a
countdown in the lobby's blue.

smash-hud-e2e caught the second of those as a real failure -- one of #1018's
own checks, and it is yellow, not the lobby's blue, against a bar that was
briefly exactly that. So one field moving is that field's narrow operation, and
two or more is a single fresh Add. Add is also the smaller packet: every
operation repeats the sixteen byte id, so two updates already cost more than
the one that carries all four fields, and BossHealthOverlay.add is a put
into a map keyed on the id, so it replaces the bar in place.

Packet count is unchanged from #1018's 1,135 -- both send one packet per change
-- and each packet is now at most the size of the Add it replaced.

The two bars that carry the host

/serverload, Admin, opt in. Fifteen people in a lobby staring at server
telemetry is a strange default and three stacked bars is most of the strip.

CPU is getrusage(RUSAGE_SELF), not ps. ru_utime + ru_stime is a
cumulative counter, so a rate over a window is the only reading it can give and
the first sample reports that it has nothing yet. ps answers the lifetime
average instead, which is a different question wearing the same units: a server
busy for a minute an hour ago reads busy under ps forever.

Memory is RSS, which is what the kernel counts when it decides who to kill.
/proc/self/statm field one on Linux, proc_pidinfo/PROC_PIDTASKINFO on
darwin. The Linux path cannot run on the box the gates run on, so the parser is
a pure function over a string with a fixture whose seven fields all hold
different numbers -- an off-by-one cannot pass it.

The bar that has no natural full

Neither quantity is a fraction. A process can use more than one core's worth of
CPU, and it can allocate until the machine refuses. So both are drawn against
the point where the machine stops being able to give more -- every core it has,
every byte of physical memory -- and both print that ceiling beside the
reading:

CPU 240% of 1600%
MEM 3.20 GiB of 128.0 GiB

which turns the choice into something checkable rather than a matter of taste:

the fill is the quotient of the two numbers in the bar's own label.

hud-check.py reads both numbers off the wire and divides. A ceiling nobody
printed cannot satisfy that, and neither can one invented to make the bar look
busy. It is also what keeps the label unnormalised: the honest figure is the
numerator, and the fill is that same fact taken against the denominator
standing next to it. 200% is two cores, which is what top says and what the
question meant.

Cores come from available_parallelism, not the raw processor count, because
in a container the honest ceiling is what the container is allowed.

The gate

nix run .#smash-hud-e2e, 48 checks, up from 25.

decode_boss_event read only Add and returned None for everything else,
because Add was all the server sent. Left alone against this server it would
not have failed -- it would have reported that no boss bar ever arrived, which
is a silent pass. It now decodes all six operations into a model of the
client's own BossHealthOverlay, so #1018's twenty-five checks ask the same
questions of the same states, and the operations are kept separately as the
only place the diffing is visible at all.

One check that was already there was also red before this branch touched
anything. #1019 moved LobbyConfig::min_players from four to two, and the hub
bar's 1/4 was written into hud-check.py as a literal, so the gate has been
failing on main against a server saying 1/2. #1019 rewrote six
tests/lobby.rs tests for exactly this reason and the one in tools/ was
missed, because nothing in CI runs the wire gates. It now reads the denominator
off the bar and additionally checks that the fill under the label is
1/needed, which is a stronger claim than the literal it replaces and cannot
go stale again. Filed as ENG-10878.

The new checks, beyond the two bars' own numbers:

  • An update sends only the update, over every boss bar packet of the whole
    run rather than at one moment. Three rules -- no packet says anything the
    client already knew, a whole bar is only resent where two or more fields
    moved, nothing arrives for a bar the client was never given -- checked
    against 849 packets in the run below.
  • A mid-match joiner gets the bar for the match it walked into, as an Add
    carrying the whole thing.
  • A second viewer joining a running bar's audience gets it under the same id
    the first one holds. The load bars are the only ones in the game with an
    audience of more than one person, so they are the only place this is
    observable.
  • Leaving that audience sends a Remove to that viewer and nothing to anybody
    else.
  • A viewer disconnecting with a bar on their screen leaves the server drawing
    everybody else's.

Broken, and watched to fail

Every new guard was inverted and run. Selected output.

The parser reads /proc/self/statm's first field instead of its second

statm_reads_the_resident_field_and_not_a_neighbour
  left: Some(287072256)  right: Some(27807744)
a_statm_line_that_is_not_one_reads_as_nothing
  left: Some(287072256)  right: None

The fixture's seven fields all hold different numbers, so the off-by-one is a
wrong answer rather than a coincidence, and the second test caught it too.

The CPU label is normalised, which is the whole point of the ask

the_cpu_label_is_unnormalised_and_the_fill_is_not
  left: "CPU 15% of 100%"  right: "CPU 240% of 1600%"

and on the wire, with the same edit:

FAIL the CPU label is unnormalised, so its ceiling is every core and not one: CPU 0% of 100%

The first CPU sample answers with the lifetime average, the way ps does

the_first_sample_reports_nothing_rather_than_an_average
  left: Some(90.0)  right: None

Sent is cleaned up with Delete rather than Remove

a_viewer_leaving_removes_only_their_own_record
  one viewer leaving destroyed the bar

Which is the whole teardown design failing in the direction that would be
hardest to notice in play: one person quits and the bar goes off everybody's
screen.

Every change resends the whole bar

FAIL and a whole bar was only ever resent where two or more fields moved at once:
  827 of 872 [('H1', '6879705f', ['title']), ('H1', '6879705f', ['title']), ...]
FAIL a new CPU reading that moves only the label costs one packet carrying only
  the label: ['add', 'add', 'add', 'add', 'add']

A client is recorded as having a bar it was never sent

FAIL and nothing arrived for a bar the client had never been given:
  39 of 981 [('H1', '6879705f', 'update_name'), ...]
FAIL the hub puts a boss bar up saying what it is waiting for: []

The second line is what this failure looks like from the outside, and it is why
the first one exists: a bar that never arrives is indistinguishable from a
server that never drew one, until something is watching for updates to a bar
nobody has.

The id is per viewer rather than per bar, and the teardown observer is gone

FAIL a viewer joining a running bar's audience is sent it under the same id the
  others hold: ['...0584', '...5816a', '...5816d', '...58784']
FAIL with a Remove, while the viewer who is still watching keeps theirs:
  [] / ['...057e', '...5787e', '...57e6a', '...57e6d']

Four ids where there should be two, because every viewer minted its own, and no
Remove at all where a viewer had just left the audience.

Verified

Local only, per ENG-10817.

  • nix run .#test -- 704 passed, 1 skipped
  • nix run .#smash-hud-e2e -- 48 checks, 0 failed
  • nix run .#fmt -- clean
  • nix run .#lint -- one failure, and it is not this branch's:
error: unused import: `flecs_ecs::prelude`
   --> events/smash/tests/abilities.rs:595:9

Reproduced against a detached worktree at origin/main with its own
CARGO_TARGET_DIR, running the same command flake.nix:182 runs, with no part
of this branch present:

$ CARGO_TARGET_DIR=/tmp/mainlint-target cargo clippy --all-targets --all-features -- -D warnings
error: unused import: `flecs_ecs::prelude`
   --> events/smash/tests/abilities.rs:595:9
rc=101

Filed as ENG-10890 rather than fixed here, because it belongs to #1023 and not
to this change. Worth knowing while chasing it: plain cargo clippy -p smash --tests over the same tree exits 0, because unused_imports is only a failure
under the -- -D warnings the flake adds -- which is how it got past #1023's
own check, and how it briefly looked like mine.

Deliberately not built

  • No synthetic load past 100% on the wire. The gate proves the fill stays a
    fraction, that the ceiling is every core, and that the fill is the quotient
    of the label's own two numbers; it does not observe a reading above one
    core's worth, because forcing the server process over a core from a test
    client would mean a burn command in production code. The unnormalised case is
    a unit test with the numbers injected: 2.4 cores of 16 reads CPU 240% of 1600%, and 2.4 cores of 1 reads CPU 240% of 100% with a full bar.
  • The /proc/self/statm fixture is written to proc(5), not captured off a
    running host.
    The gates run on darwin. What the fixture does prove is the
    field index, which is the failure that would ship, because every field holds
    a different number.
  • The sidebar is still hand rolled in smash's adapter. It has the same
    argument as the boss bar and it should follow, but porting it is a second
    change and this one already moves a gate.
  • ShownToEveryone has no caller. It is in the API because "everyone" is a
    rule rather than a list -- a player who connects a minute from now is in it --
    and it is covered by the drive system's own path, but nothing in the tree
    shows a bar to everybody yet.

Also filed

ENG-10878: smash-hud-e2e was red on main before this branch, as above.
Fixed here. The follow-up worth having is a job that runs the eight gates in
e2eOffsets, because a change that breaks only one of them is invisible today
until somebody runs it by hand.

ENG-10871: /perms set <player> Admin is gated at Normal, so any player
who can connect can make themselves Admin. The derive gates a whole clap enum
at once, so Get and Set cannot differ. This gate depends on the hole -- it
promotes its own clients, because there is no other way for a test client to
reach an Admin command -- so closing it needs a way to seed a group at startup
landed alongside. hud-check.py names the ticket at the line that depends on
it.


Written by Claude (Opus 5) with human direction.

…t has two

#1018 put one boss bar on the wire, hand rolled in a game event's adapter.
This is the API it should have been, and the two bars the operator asked for
on top of it.

A bar is an entity, its audience is `(ShownTo, player)` edges, and what each
viewer was last told is the data on `(Sent, viewer)`. That last choice is
what removes the two cases that are usually bugs: a mid-match joiner has no
pair, so the next tick sends them `Add`, and every way a bar stops being
shown ends with that pair going away, which one `OnRemove` observer turns
into a `Remove`. There is no join or leave handler in this change.

One field moves and it is that field's narrow operation; two or more and it
is a single fresh `Add`, which is both atomic and smaller than the updates it
replaces. The wire gate found that one: a lobby bar becoming a countdown was
briefly a countdown in the lobby's blue.

`/serverload`, Admin, opt in, puts the host's own CPU and RSS on two bars.
CPU is `getrusage`, not `ps`, so the first sample reports nothing rather than
a lifetime average. Both are drawn against the point the machine stops being
able to give more and print that ceiling beside the reading, so the fill is
the quotient of the two numbers in the bar's own label and `hud-check.py`
divides them to check it.

`smash-hud-e2e` goes from 25 checks to 48, including one that reads every
boss bar packet of the whole run and asserts none of them said anything the
client already knew. It also fixes a check #1019 left red on main.
@github-actions

Copy link
Copy Markdown

Benchmark Results for general

ray_intersection/aabb_size_0.1                     [  18.7 ns ...  18.7 ns ]      -0.04%
ray_intersection/aabb_size_1                       [  18.6 ns ...  18.6 ns ]      -0.19%
ray_intersection/aabb_size_10                      [  18.8 ns ...  18.8 ns ]      +0.03%
ray_intersection/ray_distance_1                    [   0.7 ns ...   0.7 ns ]      +0.22%
ray_intersection/ray_distance_5                    [   0.7 ns ...   0.7 ns ]      -0.22%
ray_intersection/ray_distance_20                   [   0.7 ns ...   0.7 ns ]      +0.12%
overlap/no_overlap                                 [  12.2 ns ...  12.2 ns ]      +0.21%
overlap/partial_overlap                            [  12.1 ns ...  12.1 ns ]      +0.01%
overlap/full_containment                           [  11.9 ns ...  11.9 ns ]      +0.15%
point_containment/inside                           [   5.0 ns ...   5.0 ns ]      +0.42%
point_containment/outside                          [   5.0 ns ...   5.0 ns ]      -0.03%
point_containment/boundary                         [   5.0 ns ...   5.0 ns ]      +0.03%

Comparing to a25c3a9

@github-actions

Copy link
Copy Markdown

Benchmark Results for general

ray_intersection/aabb_size_0.1                     [  18.7 ns ...  18.7 ns ]      -0.12%
ray_intersection/aabb_size_1                       [  18.8 ns ...  18.7 ns ]      -0.40%
ray_intersection/aabb_size_10                      [  18.8 ns ...  18.9 ns ]      +0.28%
ray_intersection/ray_distance_1                    [   1.3 ns ...   1.3 ns ]      +0.06%
ray_intersection/ray_distance_5                    [   1.3 ns ...   1.3 ns ]      +0.02%
ray_intersection/ray_distance_20                   [   1.3 ns ...   1.3 ns ]      +0.03%
overlap/no_overlap                                 [  15.6 ns ...  15.6 ns ]      +0.02%
overlap/partial_overlap                            [  15.5 ns ...  15.5 ns ]      +0.13%
overlap/full_containment                           [  14.5 ns ...  14.5 ns ]      -0.01%
point_containment/inside                           [   5.4 ns ...   5.4 ns ]      +0.17%
point_containment/outside                          [   5.7 ns ...   5.7 ns ]      -0.03%
point_containment/boundary                         [   5.4 ns ...   5.4 ns ]      +0.05%

Comparing to a25c3a9

@andrewgazelka
andrewgazelka merged commit 8318617 into main Jul 28, 2026
9 of 11 checks passed
@andrewgazelka
andrewgazelka deleted the feat/boss-bar-api branch July 28, 2026 11:04
@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 68.12977% with 167 lines in your changes missing coverage. Please review.
✅ Project coverage is 51.46%. Comparing base (a25c3a9) to head (3b49ac7).
⚠️ Report is 8 commits behind head on main.

Files with missing lines Patch % Lines
crates/hyperion/src/egress/boss_bar.rs 59.81% 87 Missing and 1 partial ⚠️
events/smash/src/adapter.rs 0.00% 39 Missing ⚠️
crates/hyperion/src/egress/server_load.rs 90.32% 19 Missing and 5 partials ⚠️
crates/hyperion-clap/src/lib.rs 0.00% 16 Missing ⚠️
@@            Coverage Diff             @@
##             main    #1026      +/-   ##
==========================================
+ Coverage   51.14%   51.46%   +0.32%     
==========================================
  Files         335      337       +2     
  Lines       30052    30554     +502     
  Branches     1137     1167      +30     
==========================================
+ Hits        15369    15725     +356     
- Misses      14438    14577     +139     
- Partials      245      252       +7     
Files with missing lines Coverage Δ
crates/hyperion/src/egress/mod.rs 69.09% <100.00%> (+1.16%) ⬆️
events/smash/src/module/hud.rs 99.61% <ø> (ø)
crates/hyperion-clap/src/lib.rs 0.00% <0.00%> (ø)
crates/hyperion/src/egress/server_load.rs 90.32% <90.32%> (ø)
events/smash/src/adapter.rs 0.00% <0.00%> (ø)
crates/hyperion/src/egress/boss_bar.rs 59.81% <59.81%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant