Add dynamically extensible underlying buffer support #62

Open
RReverser opened this Issue Feb 12, 2014 · 5 comments

3 participants

@RReverser
jDataView member

Allow new jDataView() with no arguments for creating jDataView on dynamically extensible empty buffer.

@peterseverin

Is there a workaround for this issue? I'm trying to implement a binary serialization where the final buffer size is not known in advance. I would expect to have the possibility to grow the buffer as needed.

@RReverser
jDataView member

@peterseverin For now, in own projects I had to use fixed-size buffer with calculated size big enough to hold my data. The problem is that all the existing implementations (except old generic Array) don't support dynamic extension, so this needs to be solved on jDataView size with either "ropes" mechanism (where one instance can continue other) or copy-on-extend.

Since it's needed not just for me, I'll try to get back to it soon, but if you could implement it and send PR, I'd appreciate that.

@peterseverin

@RReverser I think copy on extend would be an acceptable solution. This is a common solution in Java when using arrays. I'll try to add something simple in _checkBounds method but a good solution would probably need a more systematic approach, probably in _setBytes method.

@yuraj11

Bump, Are you still working on this? I have found something helpful - https://github.com/KimLS/dynamic-buffer-js It would be nice feature If you could generate binary file dynamically (when you don't know final arraybuffer size).

@RReverser
jDataView member

@yuraj11 Looks like it just performs copy-on-extend that we already discussed, and in very inefficient manner (copying byte-by-byte): https://github.com/KimLS/dynamic-buffer-js/blob/master/src/dynamic_buffer.js#L134-L137

At this point I believe we can track the status of ES proposal ArrayBuffer.transfer that is essentially efficient realloc: https://github.com/tc39/ecma262 ( https://gist.github.com/lukewagner/2735af7eea411e18cf20 )

When it's accepted, we can just use it together with copy-on-extend purely as polyfill for old browsers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment