DataView usage overhead

[sergeyzenchenko]

hi @gfx have you considered using direct buffer instead of DataView? DataView performance is still not optimal in many cases. I know it will require additional work on number decoding but it can give performance boost.

I hope I am not too annoying with performance stuff :) We have project with huge msgpack payloads (up to 10-20 MB) that we need to parse in browsers.

[sergeyzenchenko]

btw, in browsers msgpack-javascript now beats notepack.io :)
But in nodejs env notepack.io is still faster due to nodejs buffer usage.

[sergeyzenchenko]

Actually we have 10-20 MB of compressed payloads, real size few times bigger 😄

[gfx]

Does DataView have such overhead? I have read https://v8.dev/blog/dataview and in fact, DataView seems fast enough for NodeJS v12.

BTW My use cases are browsers, too. So I am focusing on browsers for now. Any optimizations are really welcome, though.

[sergeyzenchenko]

Take a look at this test https://jsperf.com/dataview-vs-typed-array-views
It's pretty old, but it shows not great number of DateView especially in GooglChrome.

I've tried to remove DateView partially from this project and use typed array approach for numbers decoding and it gave me a little bit more performance, but it's not ready for full testing yet. Float/Int64/Uint64 is not done yet.

[gfx]

I think t https://jsperf.com/dataview-vs-typed-array-views does not fare because of peekN().

https://jsperf.com/dataview-vs-typed-array-views/28 seems more accurate and it suggests DataView is the fastest. What do you think about it?

[sergeyzenchenko]

yeah, these tests are not perfect. I need we need more investigation in this area.
I will try to partially finish my version and show it to you.
I still can see performance improvements, but it may be an issue with my code or tests.

[sergeyzenchenko]

I've tried https://github.com/finscn/fast-dataview and their perf tests gives me next in nodejs env:

==== performance ( 26KB * 1000) ====
FastDataView wirte: 60.103ms
FastDataView read: 129.790ms
DataView wirte: 84.796ms
DataView read: 174.511ms
==== verify with DataView ====

[gfx]

Sounds an interesting idea. If the backend of DataView becomes faster, even if and only if on NodeJS, it is a kind of free lunch!

BTW I have released v1.2.2: v1.2.1...v1.2.2
and I'll become off-line because it is now 0:11 a.m. good night 😴

[sergeyzenchenko]

Thanks, Good night :)

[reZach]

Thanks for making this a better library @gfx !