Do services break HTTP streaming (ie. server-sent events)?

[activeshadow]

I have an HTTP server that publishes server-sent events over a connection using the following response headers:

"Content-Type": "text/event-stream"
"Cache-Control": "no-cache"
"Connection": "keep-alive"

When talking directly to the server over a local connection (such as loopback or on my local network), my client continually gets published data. However, when I run the server in a pod and talk to it via a service IP address (more specifically, from a client completely outside the K8S environment via a service that has a NodePort defined), my client seems to stop getting published data after a while, even though the server continues to publish via the SSE connection.

Is it possible that passing the stream connection through service IPs is causing the connection to be broken after a while? I thought the EventSource object in my client would reconnect automatically, but as it turns out, the object never detects a loss of connection (ie. the onerror callback never gets executed)... perhaps due to how connections are proxied in K8S services?

I'm also curious to know if WebSockets would have the same fate? This is something that can be easily tested, but I haven't yet done so. My other option is to fall back to good old fashioned periodic request/reply via a timer in my client, but I've spent a lot of time implementing the SSE server to abandon it too quickly.

[thockin]

It's certainly possible - how long is "a while" Is this "a while with no
traffic" or is it actually getting traffic periodically?

On Fri, Dec 18, 2015 at 7:13 PM, Active Shadow LLC <notifications@github.com

wrote:

I have an HTTP server that publishes server-sent events over a connection
using the following response headers:

"Content-Type": "text/event-stream"
"Cache-Control": "no-cache"
"Connection": "keep-alive"

When talking directly to the server over a local connection (such as
loopback or on my local network), my client continually gets published data
However, when I run the server in a pod and talk to it via a service IP
address (more specifically, from a client completely outside the K8S
environment via a service that has a NodePort defined), my client seems to
stop getting published data after a while, even though the server continues
to publish via the SSE connection

Is it possible that passing the stream connection through service IPs is
causing the connection to be broken after a while? I thought the
EventSource object in my client would reconnect automatically, but as it
turns out, the object never detects a loss of connection (ie the onerror
callback never gets executed) perhaps due to how connections are proxied in
K8S services?

I'm also curious to know if WebSockets would have the same fate? This is
something that can be easily tested, but I haven't yet done so My other
option is to fall back to good old fashioned periodic request/reply via a
timer in my client, but I've spent a lot of time implementing the SSE
server to abandon it too quickly

—
Reply to this email directly or view it on GitHub
#18923.

[activeshadow]

I can setup a test to get you an accurate amount of time if you need me to, but "a while" is roughly 2-3 minutes. Once whatever happens happens, I get absolutely no published traffic at all.

I know the server is still responsive, because I see log messages saying data's being published and GET/POST requests are responded to.

[thockin]

Nothing that I know of should be timing out on that small timescale. Not
to say there's not something I don't know about. I'm assuming TCP here -
UDP has some oddities.

Is this on the userpsace proxy or iptables?

On Sat, Dec 19, 2015 at 6:49 AM, Active Shadow LLC <notifications@github.com

wrote:

I can setup a test to get you an accurate amount of time if you need me
to, but "a while" is roughly 2-3 minutes. Once whatever happens happens, I
get absolutely no published traffic at all.

I know the server is still responsive, because I see log messages saying
data's being published and GET/POST requests are responded to.

—
Reply to this email directly or view it on GitHub
#18923 (comment)
.

[activeshadow]

I'm not sure how to answer your last question. However, after doing some additional testing with different clients (ie. curl), I'm not seeing the same issue. So, it must be something weird with the JavaScript EventSource. Sorry to have wasted your time.