Skip to content

perf: midi-file Writer uses preallocated Uint8Array - #80

Closed
elicwhite wants to merge 1 commit into
perf/midi-writebytes-inplacefrom
perf/midi-uint8-writer
Closed

perf: midi-file Writer uses preallocated Uint8Array#80
elicwhite wants to merge 1 commit into
perf/midi-writebytes-inplacefrom
perf/midi-uint8-writer

Conversation

@elicwhite

Copy link
Copy Markdown
Owner

Replace Writer's backing Array with a growable Uint8Array +
manual cursor. Benefits:

  • Raw bytes instead of boxed JS numbers — ~8× less memory per byte
    and much less GC pressure.
  • writeBytes on a Uint8Array source is a single .set() call instead
    of a push loop.
  • writeVarInt no longer allocates a temp Array to hold the bytes
    then .reverse()s it — the 1-5 byte tail is written directly into
    the buffer.

API shape preserved for callers:

  • Inner sub-writers expose .used() which returns a subarray view.
  • Top-level writeMidi returns the used subarray; scan-chart's
    new Uint8Array(writeMidi(midiData)) still works (copies the
    subarray into a dedicated Uint8Array), and the few other direct
    callers in spotify-clonehero-next that wrap the result in
    Uint8Array also still work.

Measured on writer autoresearch bench (2000 charts, 8 workers):
previous (post-writeBytes fix): 6.721 ms mean, 25.263 ms p99, 77 ms max, 5.496s wall
this patch: 5.710 ms mean, 22.576 ms p99, 42 ms max, 5.132s wall
delta: -15.0% mean, -10.6% p99, -45% max, -6.6% wall

0 hash mismatches, 442/442 tests still green.

Replace Writer's backing Array<number> with a growable Uint8Array +
manual cursor. Benefits:
  - Raw bytes instead of boxed JS numbers — ~8× less memory per byte
    and much less GC pressure.
  - writeBytes on a Uint8Array source is a single .set() call instead
    of a push loop.
  - writeVarInt no longer allocates a temp Array to hold the bytes
    then .reverse()s it — the 1-5 byte tail is written directly into
    the buffer.

API shape preserved for callers:
  - Inner sub-writers expose .used() which returns a subarray view.
  - Top-level writeMidi returns the used subarray; scan-chart's
    `new Uint8Array(writeMidi(midiData))` still works (copies the
    subarray into a dedicated Uint8Array), and the few other direct
    callers in spotify-clonehero-next that wrap the result in
    Uint8Array also still work.

Measured on writer autoresearch bench (2000 charts, 8 workers):
  previous (post-writeBytes fix):  6.721 ms mean, 25.263 ms p99,  77 ms max, 5.496s wall
  this patch:                      5.710 ms mean, 22.576 ms p99,  42 ms max, 5.132s wall
  delta:                           -15.0% mean, -10.6% p99, -45% max, -6.6% wall

0 hash mismatches, 442/442 tests still green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant