decodeArrayStream vs decodeStream

[grantila]

The documentation of decodeArrayStream and decodeStream is a bit vague, and the signatures and examples are identical.

Do we need both? And what's the exact difference? One accepts "an array of items" and the other "independent MessagePack items". Huh?

My exact question is this; Do any of these support byte streams (of arbitrary size) of data consisting of MsgPack items, or do the inputs have to be "complete"?

What I'm explicitly after is support for this:

Consider MsgPack-encoded objects:
+---------------------+ +----------------+ +----------------+
|    MsgPack enc 1    | | MsgPack enc 2  | | MsgPack enc 3  |
+---------------------+ +----------------+ +----------------+
Although split into Uint8Array chunks:
+-------+ +-------+ +-------+ +-------+ +--------+ +--------+ 
|   1   | |   2   | |   3   | |   4   | |   5    | |   6    |
+-------+ +-------+ +-------+ +-------+ +--------+ +--------+ 
When these chunks are provided to decode*Stream(), I get a stream of:
                       +---------------------+
                       |    MsgPack dec 1    |
                       +---------------------+
                                          +----------------+
                                          | MsgPack dec 2  |
                                          +----------------+
                                                             +----------------+
                                                             | MsgPack dec 3  |
                                                             +----------------+
               ------------ Time ----------->

Is this possible?

[gfx]

Those examples look not so good and should be improved.

In fact, both can handle chunk of encoded MessagePack binaries, but they handle different formats.

decodeStream handles {"name": "foo"}{"name":"bar"} (shown as JSON for readability), whereas decodeArrayStream handles [{"name": "foo"},{"name":"bar"}].

Does it make sense for you?

[grantila]

Kind of, clarifying that would surely be useful.

My question is rather, can I send a stream of individually incomplete chunks? Uint8Array's that are, say, at most n bytes large. This would be an input stream as your example of (n=6):

         +--------+
 Chunk 1 | {"name |
         +--------+
         +--------+
 Chunk 2 | ": "fo |
         +--------+
         +--------+
 Chunk 3 | o"}{"n |
         +--------+
         +--------+
 Chunk 4 | ame":" |
         +--------+
         +-------+
 Chunk 5 | bar"} |
         +-------+

Can I get this output stream of decoded:

           +---------------+
 Message 1 | {name: 'foo'} |
           +---------------+
           +---------------+
 Message 2 | {name: 'bar'} |
           +---------------+

Is this possible?

[gfx]

@grantila

Yes, it is possible and the default in all the async or stream decoders. For example, see tests for decodeAsync https://github.com/msgpack/msgpack-javascript/blob/master/test/decodeAsync.test.ts#L28-L37