max length / max size of message field between parent/child

[ORESoftware]

I am on Node.js version 9.
I am wondering, what is the max size of the message field 'm'?

//child
process.on('message', function (m, socket) {

});

//parent
child.send(m, socket);

how big can m be in size? I am thinking of putting some JSON data in there.

[advanceddeveloper]

Not sure if it is mentioned in the documentation. Probably it depends on the operating system. On linux it seems that the max length is 65536, according to this test. On windows the max value seems to be 2147483647 (tested locally on my PC).

[gireeshpunathil]

@advanceddeveloper - what happens if we give more than that? Does it truncate or send in iterations? I guess the later.

@ORESoftware - I guess you mean the size that can be sent in one chunk right? As these are normal sockets that are piped, I don't think there is any Node.js imposed limit, instead chunking occurs, at the OS level.

[advanceddeveloper]

@gireeshpunathil

what happens if we give more than that? Does it truncate or send in iterations?

Of course it splits the buffer and send in iterations, that is also the purpose of the if(m) return in the code I posted (log the length of the first iteration and dismiss others). Also, another test confirms it.

chunking occurs, at the OS level.

I am pretty sure @ORESoftware is asking for the size of one chunk (not the whole buffer from the other process), so it seems that my assumption "probably it depends on the operating system." is right...

[advanceddeveloper]

@ORESoftware

I am thinking of putting some JSON data in there.

I forgot to mention that even if you know the limit of the chunk size, it is not guaranteed that OS will not split it up in even smaller chunks. If you want to send several stringified JSONs from one process to another, you should somehow store previous chunks and concatenate results, and also detect boundaries of adjacent json strings. Here is a very simple example how you can make it work no matter how small chunk size is:

main-process.js

'use strict';

var cp = require('child_process');

var proc = cp.spawn('node', ['./child-process.js']);
proc.stdout.pipe(process.stdout);

proc.stdin.write(JSON.stringify([1, 2, 3, 4, 5]));
proc.stdin.write(JSON.stringify({key1: 'value1', key2: ['value2']}));
proc.stdin.write(JSON.stringify({numeric: 123.45}));

proc.stdin.write('{"a":');
proc.stdin.write('"b"}[1]');

proc.stdin.end();

child-process.js

'use strict';

var data = Buffer.alloc(0);

process.stdin.on('data', d => {
  var index = data.length + 1;
  data = Buffer.concat([data, d]);

  while(index !== data.length + 1){
    var str = data.slice(0, index).toString('utf8');
    var valid = false;
    var json;

    try{
      json = JSON.parse(str);
      valid = true;
    }catch{}

    if(!valid){
      index++;
      continue;
    }

    console.log('RECEIVED JSON:');
    console.log(JSON.stringify(json));
    console.log('');

    data = data.slice(index);
    index = 0;
  }
});

And here is the output:

Output

RECEIVED JSON:
[1,2,3,4,5]

RECEIVED JSON:
{"key1":"value1","key2":["value2"]}

RECEIVED JSON:
{"numeric":123.45}

RECEIVED JSON:
{"a":"b"}

RECEIVED JSON:
[1]

[ORESoftware]

Yeah in the original question, I am not talking about the socket, I am talking about 'm', just in case that wasn't clear.

[advanceddeveloper]

@ORESoftware

I am talking about 'm', just in case that wasn't clear.
What is the max size of the message field 'm'?
How big can 'm' be in size?

Which m? You have two ms in your post, The first one is argument of send method and the second one is argument of data listener callback. These are different ms.

The first one can be large up to require('buffer').constants.MAX_LENGTH, while the second one depends on the operating system. When you send some buffer, operating system may split it in smaller chunks and send one at a time to your child process. It is not guaranteed that you will receive the exact same buffer you sent. It will probably be splitted into smaller buffers, because it depends on how operating system itself handles it. Please see my comment above and tell me if it answers your question or not.

[ORESoftware]

Well yeah m would hopefully be the same data lol, but one is sending and one is receiving it

[gireeshpunathil]

not sure if there is anything outstanding here - @ORESoftware ?

[gireeshpunathil]

closing as answered, please let me know if anything is outstanding here.