How to send server side events (on Mongodb watch() for example)

[russelgal]

No description provided.

[guilhermeoc97]

Set the Content-Type and Cache-Control headers to text/event-stream and no-cache respectively and send your event data with response.end.

Also, this isn't really an issue so I guess it can be closed.

[antoine-coulon]

Hello, just coming back to that issue.

Sending the event data with response.end seems to be contrary to the principle of SSE which keeps alive the connection open to receive events. Using response.end closes the stream forcing EventSource on the browser to keep re-opening connections. So I would say that this is still an issue as response.write with text/event-stream should write to the response without requiring response.end as it is done when using Node.js http server.

I'm not expecting any fix, just trying to clarify the issue for the ones who'll try to use polka with SSE.

[lukeed]

The last message is correct and is not Polka specific. The header is necessary for SSE in general and response.write is a native Nodejs API

[antoine-coulon]

@lukeed it is indeed not polka specific but given it uses Node.js http server, shouldn't it be working with these headers and writing to the response with response.write? I'm actually able to make it work with Node.js http server exact same API but when using polka it seems that no write is done in the response until calling response.end. I might be confused for some reason and it might be part of polka's default behavior.

For example:

import { createServer } from "node:http";
const srv = createServer((req, res) => {
  res.writeHead(200, {
    "Content-Type": "text/event-stream",
    "Cache-Control": "no-cache",
    "access-control-allow-origin": "*",
    Connection: "keep-alive",
  });

  setInterval(() => {
    res.write(`data: ${Date.now()}\n\n`);
  }, 1000);

  req.on("close", () => {
    res.end();
  });
});

srv.listen(4000, () => {});

This works, but if I now switch createServer for polka, this does not seem to write to the event stream anymore. Again I'm not asking for any fix (even though I could send a PR if this is an issue), just trying to clarify the situation for people (including me) using polka and trying to setup SSE.

[guilhermeoc97]

You're absolutely right @antoine-coulon, I probably confused write with end back then, my bad.

It's been almost 5 years since I last used Polka (or any Node server framework/lib), but I remember having to call ob_flush and flush on PHP whenever I wanted to send data; doesn't Node work the same way? I've found a discussion on SO and it appears that Node does have a buffer, but instead of flushing it you have to wait for it to flush itself... so you have to listen to the drain event and only then you can write more to the response stream.

Discussion (relevant comment): https://stackoverflow.com/questions/60372306/how-to-flush-data-to-client-just-after-res-write-is-written-before-res-end#comment111542785_60384570

[antoine-coulon]

@guilhermeoc97 actually the example I pasted just above works with Node.js http module, basically response.write does the trick you don't have to flush nor to end the response. I don't really know why polka does not behave the same I didn't go into the internals, but because it uses Node.js http server under the hood (it seems) I would have expected the same behavior allowing SSE to work very simply.

[lukeed]

Can you post the code using Polka?

[antoine-coulon]

Actually it works I don't know what dummy things I did to not make it work (I might have mixed up APIs while testing with fastify, polka and node:http) or maybe forgot the double escape \n\n allowing stream chunking, sorry for the inconvenience @lukeed.

So to close that discussion, I'm happily posting an example of SSE working very simply and as expected with our dear polka:

import polka from "polka";

const app = polka();

app.get("/subscribe", (request, response) => {
  response.writeHead(200, {
    "Content-Type": "text/event-stream",
    "Cache-Control": "no-cache",
    Connection: "keep-alive",
    // Might not be needed if running the front and backend one the same domain
    "access-control-allow-origin": "*",
  });

  const intervalId = setInterval(() => {
    response.write(`data: ${new Date().toLocaleString()}\n\n`);
  }, 1000);

  request.on("close", () => {
    clearInterval(intervalId);
  });
});

app.listen(4000, () => {
  console.log("Server is listening on port 4000");
});

By the way @lukeed do you think it could be useful to add that in the polka/examples section? I could provide this example with a very simple frontend (index.html) that shows how to fully make it work. Let me know, I can open a PR.

[lukeed]

Great! And sure, go for it!

[antoine-coulon]

Just went for it, opened there #204

[antoine-coulon]

I think I found what led me thinking that polka was not working with SSE in my previous setup, this was due to compression middleware I'm currently using in skott.

According compression's documentation:

Because of the nature of compression this module does not work out of the box with server-sent events. To compress content, a window of the output needs to be buffered up in order to get good compression. Typically when using server-sent events, there are certain block of data that need to reach the client.

You can achieve this by calling res.flush() when you need the data written to actually make it to the client.

Again this is not related to polka but to the middleware I was using, so be careful if using it, you'll need to manually call response.flush() otherwise no data will be sent to the client.

[lukeed]

That makes sense!