Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Server Send Events connections not closing #369

Closed
bwegman opened this Issue May 6, 2012 · 7 comments

Comments

Projects
None yet
4 participants
Contributor

bwegman commented May 6, 2012

I found two issues with SSE:

  1. When using SSE to connect to SignalR from Google Chrome (tested with version 18) upon refreshing the page using SignalR the initial connection to the server remains active until a timeout occurs. This causes issues when using IIS on Win7 as this version is limited to only 10 client connections. Observed behaviour is unresponsiveness from the server until the signalr connections timeout.
  2. When using SSE (might also affect other protocols) when a connection is lingering in the situation described above and the application pool is recycled due to updating a binary or touching the web.config the connections remain indefinetely until the w3wp.exe process is exited. This will not happen until IIS decides to recycle the worker process. This seems to be an issue with connections not being force closed server-side upon application recycling. Observed behaviour is multiple connections with very high connection times (300+ seconds) in the Requests overview of IIS (IIS Manager -> Worker Processes -> Requests).

Reproduction scenario for issue 2:

  • Setup the QuickStart sample.
  • Setup a couple of SignalR connections to your application using the technique described under 1. Make sure SSE is used.
  • Either rebuild your webapp or touch the web.config file (trigger a reload of the app domain).
  • You might need to repeat the above steps a couple of times.
  • One or more connections keep lingering around in the Requests overview.

Both issues do not occur when using longPolling.

A related issue is that connections are not closed correctly upon application recycling when IIS is not immediately closing the app domain. The implementation is incorrectly calling UnregisterObject on IRegisteredObject when the parameter Immediate for the Stop method is False. It should only do this after finishing clean up. This cleanup however is only triggered when the parameter Immediate is True. For SignalR connections you probably want to close the connections asap as no new connections are accepted by IIS during app domain shutdown. See bwegman/SignalR@ccfcb4a for a possible fix for this.

Owner

davidfowl commented May 6, 2012

  1. Isn't a but it's by design. The connection dies after a while but it's not immediate. Use fiddler to see this. Use IIS Express on win7 for dev or Server OS, you won't have these restrictions. That can happen with longpolling as well. Just open 12 browser windows with persistent connections and watch.
  2. Seems like a good idea to always cleanup.

Thanks.

@ghost ghost assigned davidfowl May 7, 2012

@davidfowl davidfowl closed this in 9053412 May 7, 2012

Contributor

bwegman commented May 7, 2012

Though that commit does solve the problem with the app domain not always shutting down it does not seem to help with force-closing lingering connections.

I think (but I am not sure) that the server side code does not have a way to force close a connection. It does send a Disconnect message to the MessageBus but that does not result in a (force) close of the connection. Problem with this is that upon an app recycle those connections go into some sort of zombie state where:

  1. ASP.Net does not seem to have any control anymore of the appdomain (it has been 'shutdown').
  2. IIS does not close down the app domain until the lingering connections have closed. And somehow the default timeout (if there is any) does not apply to those connections. This seems closely related to using overlapped recycling.
  3. Not closing down the connections causes the app domain to not unload. In my case this causes some other side effects with connections to our back-end bus not closing down.

Thanks for incorporating the shutdown fix, hopefully the issue described in this comment can also be resolved. My suggestion would be to close the connections serverside upon disconnect.

Owner

davidfowl commented May 7, 2012

We always close dead connections (look at transport heartbeat) it's the
web server's job to tell us the connection is dead, if it doesn't then
we don't know.

Try this, open one browser instance and fiddler (name sure streaming
mode is on). Close the browser, and watch hoe long it takes that
connection to go away. Closing the browser doesn't kill the connection
immediately , it does go away eventually.

The only way to have a lingering connection is if it never really does
go away (and that shouldn't happen if the connection is really dead
unless IIS is lying).

Sent from my Windows Phone
From: Benjamin Wegman
Sent: 5/7/2012 4:05 AM
To: David Fowler
Subject: Re: [SignalR] Server Send Events connections not closing (#369)
Though that commit does solve the problem with the app domain not
always shutting down it does not seem to help with force-closing
lingering connections.

I think (but I am not sure) that the server side code does not have a
way to force close a connection. It does send a Disconnect message to
the MessageBus but that does not result in a (force) close of the
connection. Problem with this is that upon an app recycle those
connections go into some sort of zombie state where:

ASP.Net does not seem to have any control anymore of the appdomain

(it has been 'shutdown').

IIS does not close down the app domain until the lingering

connections have closed. And somehow the default timeout (if there is
any) does not apply to those connections. This seems closely related
to using overlapped recycling.

Not closing down the connections causes the app domain to not

unload. In my case this causes some other side effects with
connections to our back-end bus not closing down.

Thanks for incorporating the shutdown fix, hopefully the issue
described in this comment can also be resolved. My suggestion would be
to close the connections serverside upon disconnect.


Reply to this email directly or view it on GitHub:
#369 (comment)

Owner

davidfowl commented May 7, 2012

@bwegman You also, can't write to a connection that's gone. So the notion of closing the disconnected connection doesn't make much sense.

I think I am having a similar issue with SSE.

  1. When I debug on my local machine I have a connection with SSE and then I kill the page, from chrome (48.0.2564.116 m) task manager and then I close the page. The connection does not disconnect at all. I've waited at least 15-20 mins. I checked it in ITransportHeartbeat.GetConnections() and the IsAlive property remains true for the described time. I caught this on my live environment where I have 1000+ connections with SSE mostly on mobile devices. The users have multiple connections for the same userID (which should not happen and it doesn't happen for other transports). I guess that on mobile devices they close the tab from the task manager and the same thing happens as if you killed the process (haven't been able to reproduce it).
  2. There is another thing that our QA team simulated, not sure if is related. When a mobile device is connected with SSE, then the browser is minimized, then the device is locked (iPhone) or the internet is stopped (Android tablet) and after 30 seconds it is brought back, we end up with a new connection on SSE and the old one is still alive and keeps being alive after 10-15 mins. I haven't had the time to simulate it in an isolated project, but I decided to write it anyway, because it might help knowing about it regarding the first issue.

We are using signalR 2.2.0 server and js client.

Contributor

moozzyk commented Mar 17, 2016

I would suggest opening a new issue. This issue was closed long time ago and as such will not show up when looking at active issues. It may also not be the same issue that was closed.

Thank you for the tip.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment