screen reports what is open but not what is in it:
$ gamebridge --devbridge 8604 screen
screen: net.minecraft.client.gui.screens.DeathScreen
title: You Died!
width: 427
height: 240
That is enough to know a click is possible and not enough to know where.
Both clicks I needed today were arrived at by arithmetic outside the game:
- DeathScreen's Respawn button - derived from vanilla's own layout, buttons
at height / 4 + 72, 200 wide and centred, giving (213, 132). Correct, but it
depends on knowing Minecraft's source rather than asking the game.
- The FTB Quests chapter tab - measured off a 1280x720 screenshot at roughly
(105, 116) and divided by the 3x GUI scale to get (35, 39). Correct, and pure
guesswork.
Neither is knowable from what screen returns, and neither survives a GUI scale
change, a resize, or a mod moving a widget. The coordinate space is also not
stated anywhere: I inferred GUI-scaled rather than window pixels from the
reported 427x240, and only confirmed it by trying.
What would make clicking scriptable is screen listing the widgets it can see -
something like a class or type, the text if there is any, and x/y/width/height in
the same space click takes. Then a caller finds "Respawn" or "Salvage" by name
and clicks its centre, instead of reproducing layout maths that belongs to
whoever drew the screen.
Vanilla screens expose their children, so most of this is reachable; modded
screens that draw without widgets would come back empty, which is honest and
still better than silence.
Pairs with #27: without bounds you guess the coordinate, and without a
trustworthy return you cannot tell whether the guess was right.
screenreports what is open but not what is in it:That is enough to know a click is possible and not enough to know where.
Both clicks I needed today were arrived at by arithmetic outside the game:
at
height / 4 + 72, 200 wide and centred, giving (213, 132). Correct, but itdepends on knowing Minecraft's source rather than asking the game.
(105, 116) and divided by the 3x GUI scale to get (35, 39). Correct, and pure
guesswork.
Neither is knowable from what
screenreturns, and neither survives a GUI scalechange, a resize, or a mod moving a widget. The coordinate space is also not
stated anywhere: I inferred GUI-scaled rather than window pixels from the
reported 427x240, and only confirmed it by trying.
What would make clicking scriptable is
screenlisting the widgets it can see -something like a class or type, the text if there is any, and x/y/width/height in
the same space
clicktakes. Then a caller finds "Respawn" or "Salvage" by nameand clicks its centre, instead of reproducing layout maths that belongs to
whoever drew the screen.
Vanilla screens expose their children, so most of this is reachable; modded
screens that draw without widgets would come back empty, which is honest and
still better than silence.
Pairs with #27: without bounds you guess the coordinate, and without a
trustworthy return you cannot tell whether the guess was right.