Skip to content

bench: hold judgement feature tags to a checkable floor - #19

Merged
macabeus merged 2 commits into
mainfrom
bench/judgement-tags
Aug 4, 2026
Merged

bench: hold judgement feature tags to a checkable floor#19
macabeus merged 2 commits into
mainfrom
bench/judgement-tags

Conversation

@macabeus

@macabeus macabeus commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Stacked on #18. #16 fixed the machine-checked half of the feature vocabulary and left the judgement half alone — so the audits' findings there survived. This closes them, and adds a floor so they cannot come back.

The idea

A judgement tag cannot be decided mechanically: is this bulk memory movement? is this arithmetic the point of the function? is a human call. But nearly every fabricated tag the audits found sat below a floor that is decidable — table on a function with no array, arithmetic on a body with no operator, branch on two straight-line assignments.

So JUDGEMENT_FLOOR checks the floor and features.test.ts asserts it. The sufficient condition stays with the reviewer; only the indefensible is rejected.

Removed

tag rows evidence
table af:atans_table reads no array — it calls U_GetAtanTable(). The tag came from the name.
pokeemerald:DoForcedMovement no [ in the body at all; it dispatches through a function-pointer parameter
kleod:ConfigureEntityBehavior indexes only with literals (gEntityInfo[0x23])
marioparty3:DecodeData its table is the jump table, which is now a derived codegen tag
arithmetic 11 marioparty3 functions no + - * / % in the body. GWMgNoSet's entire compiled form is lui/jr/sb/nop
af:lbRk_IsLeapMonth one equality test
pokeemerald:FeebasRandom the arithmetic is inside ISO_RANDOMIZE2 — retagged macro
branch pokeemerald:StopScript two assignments, no conditional
sa3:sub_806132C zero cmp across 177 lines of assembly
cast kleod:ReadUnalignedU32, snowboardkids2:loadAssetGroupResources no cast operator
array pokeemerald:AcroBikeHandleInputTurning no [
struct snowboardkids2:func_8005DF10_5EB10 no . or ->
global 3 snowboardkids2 rows, SetMauvilleOldManLanguage disproved by the relocation/pool evidence

Added

Each checked against the project's own headers:

  • union + bitfield on sa3:OamMalloc and sa3:ProcessOamBuffersOamData is a typedef union of bitfield structs (gba/types.h:125-155), and both write .split.fractional. The dataset's two most obviously-union rows had neither tag.
  • union on three more rows; bitfield on pokeemerald:TrySetCantSelectMoveBattleScript (gProtectStructs' u32 palaceUnableToUseMove:1, written five times).
  • fnptr on marioparty3:HuMemAllocTag — the dataset's only jalr.

nested is retired: it meant nested loops in sa3 and nested ifs in af, and collided with the machine-checked nested-loop.

What deliberately gets no floor

global, memory, pointer and the type-ish tags. kleod spells several globals as address macros (#define gStreamPtr (*(u8**)0x03004D84)), which emit a raw .word 0x3004d84 rather than a symbol; af:gfxopen is return gfxHead + 1; — pointer arithmetic with no *, [ or -> in the body. A floor that flagged those would be worse than none.

Three of my own floors were wrong

Caught by running them over the corpus before applying anything, not by inspection:

  • cast missed a lowercase project typedef — (s_xyz*) in af:ValueSet__s_xyz.
  • struct demanded member access, so sizeof(struct MemBlock) in pokeemerald:PutFirstMemBlockHeader failed it.
  • pointer missed -> entirely, flagging every struct-pointer function in snowboardkids2.

All three would have removed correct tags. That is the third time in this stack a detector of mine was wrong in the same direction, which is the argument for the corpus check being part of the workflow rather than an afterthought.

Verification

743 rows, dirty: false, stale-check green, board unchanged (asmlift 354 / m2c 342) — tags only. 157 benchmark tests (2 new) and 34 web tests pass; typecheck, lint, format:check clean.

🤖 Generated with Claude Code

@macabeus
macabeus force-pushed the bench/judgement-tags branch from 68ff8a7 to 97ff9d0 Compare August 4, 2026 13:37
@macabeus
macabeus force-pushed the bench/judgement-tags branch from 97ff9d0 to 93bc899 Compare August 4, 2026 13:46
@macabeus
macabeus force-pushed the bench/judgement-tags branch from 93bc899 to 379ef94 Compare August 4, 2026 13:56
Base automatically changed from bench/synthetic-unions to main August 4, 2026 15:35
macabeus and others added 2 commits August 4, 2026 16:35
#16 fixed the machine-checked half of the vocabulary and left the judgement
half alone, so the audits' findings there survived: `table` on functions with
no array, `arithmetic` on bodies with no operator, `branch` on straight-line
code.

A judgement tag cannot be decided mechanically — "is this BULK memory
movement?" is a human call — but most of them have a FLOOR that can be, and
the fabrications all sat below it. `JUDGEMENT_FLOOR` checks that floor and
`features.test.ts` asserts it; the sufficient condition stays with the
reviewer.

Removed (each verified against the body or the compiled reference):

  table        af:atans_table reads no array — it calls U_GetAtanTable, and
               the tag came from the NAME; pokeemerald:DoForcedMovement has no
               `[` at all; kleod:ConfigureEntityBehavior indexes only with
               literals; marioparty3:DecodeData's table is the jump table,
               which is now a derived tag
  arithmetic   11 marioparty3 functions with no `+ - * / %` in the body —
               GWMgNoSet's whole compiled form is `lui/jr/sb/nop`;
               af:lbRk_IsLeapMonth is one equality test;
               pokeemerald:FeebasRandom's arithmetic is inside ISO_RANDOMIZE2,
               so it is tagged `macro` instead
  branch       pokeemerald:StopScript is two assignments; sa3:sub_806132C has
               zero `cmp` across 177 lines
  cast         kleod:ReadUnalignedU32 and snowboardkids2:loadAssetGroupResources
               contain no cast operator
  array        pokeemerald:AcroBikeHandleInputTurning has no `[`
  struct       snowboardkids2:func_8005DF10_5EB10 has no `.` or `->`
  global       three snowboardkids2 rows and SetMauvilleOldManLanguage, each
               disproved by the relocation/pool evidence

Added, each checked against the project's headers: `union` + `bitfield` on
sa3:OamMalloc and ProcessOamBuffers (OamData is a typedef union of bitfield
structs, gba/types.h:125-155), `union` on three more rows, `bitfield` on
pokeemerald:TrySetCantSelectMoveBattleScript (gProtectStructs' `u32 x:1`),
`fnptr` on marioparty3:HuMemAllocTag — the dataset's only `jalr`.

`nested` is retired: it meant nested LOOPS in sa3 and nested IFS in af, and
collided with the machine-checked `nested-loop`.

`global`, `memory`, `pointer` and the type-ish tags get NO floor on purpose.
kleod spells several globals as address macros, which emit a raw `.word
0x3004d84` rather than a symbol; af:gfxopen is `return gfxHead + 1;`, pointer
arithmetic with no `*`, `[` or `->` anywhere. A floor that flags those would
be worse than none.

Three of my own floors were too strict and were caught by the corpus before
they removed a correct tag: `cast` missed a lowercase project typedef
(`(s_xyz*)`), `struct` demanded member access where `sizeof(struct MemBlock)`
should qualify, and `pointer` missed `->` entirely.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tags only — every outcome, score and gap size is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@macabeus
macabeus force-pushed the bench/judgement-tags branch from 379ef94 to 2b9461d Compare August 4, 2026 15:35
@macabeus
macabeus merged commit 0da44b6 into main Aug 4, 2026
2 checks passed
@macabeus
macabeus deleted the bench/judgement-tags branch August 4, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant