Stream data splitted in different ways. How to configure?

[jamsouf]

I have 2 servers, on both is Ubuntu 22.04 installed.

On both servers I'm executing the same script, simplified version is:

connection.exec('apt remove curl', { pty: {cols: 1000} }, (err, stream) => {
   stream.on('data', (output) => {
      console.log('[OUTPUT] ' + output);
   });
});

On the first server I get this output:

...
[OUTPUT] The following packages will be REMOVED:
[OUTPUT] curl
[OUTPUT] 0 upgraded, 0 newly installed, 1 to remove and 1 not upgraded.
After this operation, 454 kB disk space will be freed.
Do you want to continue? [Y/n]

On the second server I get this output:

...
[OUTPUT] The following packages will be REMOVED:
[OUTPUT] curl
[OUTPUT] 0 upgraded, 0 newly installed, 1 to remove and 1 not upgraded.
[OUTPUT] After this operation, 454 kB disk space will be freed.
[OUTPUT] Do you want to continue?
[OUTPUT] [Y/n]

On the first, the text "Do you want to continue? [Y/n]" is still together in one data event.
But on the second, the text "Do you want to continue?" is one data event, and the text "[Y/n]" is a separate data event.

Why the difference?

In my use case I check if the returned output contains some patterns, like "Do you want to continue? [Y/n]" or others.
If so, I write the appropriate response back to the stream.
But for this to work I've to rely on the output not splitted in the middle.

Can this behavior be configured? How to solve?

[mscdex]

I don't know why offhand they would differ, you could try also setting the other possible pty parameters to see if that makes any difference. Otherwise in general when you're working with (byte) streams in node you should never make assumptions about chunks you receive.

For your specific situation though, you should look into buffering a line at a time and then doing your checking and/or look into using different apt CLI options to make things easier (e.g. no prompting, disabling pty requirement, etc.).

[jamsouf]

I tried the other pty parameters, but no change.
The apt command was really just an example, it could be any command prompting for input.

Yeah I also thought about buffering.

Thinking about the following logic:

1. Buffer the received data.
2. When the current buffer ends with a newline, treat the current buffer as one chunk of data and process it + clear the buffer
3. When the buffer doesn't end with a newline and there is no more data coming (I receive "Do you want to continue?" without a newline, then I receive "[Y/n]" also without a newline, then it's waiting for input), wait e.g. for 1 sec. and then treat the current buffer as one chunk of data and process it + clear the buffer

Can this work reliably to recognize line by line and if it waits for input without a newline, then still treat the data received so far as one chunk to process it (e.g. checking for patterns etc.)?

[mscdex]

That's totally up to you. I can't make guarantees for arbitrary software.

[jamsouf]

Thanks, keep up the great work 👍