Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Buffer with Uint8Array #452

Draft
wants to merge 22 commits into
base: master
Choose a base branch
from

Conversation

valadaptive
Copy link
Contributor

I still need to do some cleanups and replace uses of Buffer.compare, but I'm putting this PR here so you can benchmark it. Before I can complete this, #428 (minus the version bump) should be merged in, as the services code uses Buffer everywhere and expects fixed/bytes types to decode to Buffers.

I've made some tweaks to the benchmarking setup, so comparing this directly to the master branch won't work:

  • I've added an ArrayFloat benchmark to go along with ArrayDouble.
  • I use the --expose-gc flag when benchmarking in order to manually trigger garbage collection between benches, hopefully making results more consistent.
  • I've changed the length distribution of strings to be exponentially weighted, so shorter strings are still likely but longer strings will now be occasionally generated. The previous code was only benchmarking the manual path of Tap#writeString, since it only generated strings up to a length of 32.

I've cherry-picked those benchmarking changes into the bench-tweaks branch, which you can use to compare benchmarks.

@mtth
Copy link
Owner

mtth commented Mar 4, 2024

Thanks @valadaptive! I'll try to find time to merge #428.

@mtth
Copy link
Owner

mtth commented Mar 30, 2024

FYI @valadaptive - #428 is in.

@valadaptive valadaptive force-pushed the debufferify branch 2 times, most recently from 3e4bce6 to 6870ee6 Compare March 30, 2024 21:10
@valadaptive
Copy link
Contributor Author

Working on removing Buffer usage from types.js now. I noticed the isJsonBuffer function, which seems to check if a given object is the JSON representation of a Buffer. Under what circumstances are Avro types directly serialized to JSON and/or parsed back directly? I can't easily polyfill Uint8Array to stringify to a regular array, so I'll probably need to insert a fixup step when stringifying/parsing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants