Streaming subprocess output over websocket

[itmecho]

Hi!

I'm trying to implement a websocket server which executes shell commands and streams the output over a WebSocket. I've got it working using node:child_process but I can't seem to find a way to send lines back over the web socket using Bun.spawn.

With node:child_process, I can do

const p = spawn("some-command", ["arg1", "arg2"]);
p.on("close", () => {
  ws.close(10003, "program exited");
});
p.stderr.on("data", (data) => {
  ws.send(
    JSON.stringify({
      type: "output",
      source: "stderr",
      data: new TextDecoder().decode(data),
    } as OutputMessage),
  );
});

but I can't figure out an equivalent for Bun.spawn. I've tried looping over the piped stderr but it just send it through as one big string instead of line by line as they come in.

const cmd = Bun.spawn(["some-command", "arg1", "arg2"], {
  onExit: () => {
    ws.close(10003, 'program exited');
  },
  stderr: "pipe",
});
for await (const line of cmd.stderr) {
  // "line" here isn't actually a line, it's the entire stderr.
  ws.send(
    JSON.stringify({
      type: "output",
      source: "stdout",
      data: new TextDecoder().decode(line),
    } as OutputMessage),
  );
}

In the Bun case, cmd.stdout.on doesn't exist and I can't find any way to replicate the node behaviour.

Is this just something that is expected and I should be using the node library or should I be able to achieve the same behaviour via Bun?

[iitzkube]

@itmecho I just came across your post! In case you're still looking for an answer two weeks later, here it is :)

When you iterate over cmd.stderr, you get back a buffer chunk, which may be a line, multiple lines, half a line, you don't know.... Therefore, you need to implement a basic line reader like this:

const cmd = Bun.spawn(["some-command", "arg1", "arg2"], {
  stderr: "pipe",
  onExit: () => {
    ws.close(10003, 'program exited');
  },
});

const reader = cmd.stderr.pipeThrough(new TextDecoderStream()).getReader();

let remainder = '';
while (true) {
  const { done, value } = await reader.read();
  if (done) {
    break;
  }
  const lines = (remainder + value).split('\n');
  remainder = lines.pop()!;
  for (const line of lines) {
    ws.send(
      JSON.stringify({
        type: 'output',
        source: 'stdout',
        data: line,
      } as OutputMessage),
    );
  }
}

[itmecho]

Ah that's brilliant! Thanks =] Tested it and it works great