Skip to content
This repository has been archived by the owner on Dec 18, 2018. It is now read-only.

Timed out poll connections kill the connection #4

Closed
moozzyk opened this issue Nov 3, 2016 · 9 comments
Closed

Timed out poll connections kill the connection #4

moozzyk opened this issue Nov 3, 2016 · 9 comments
Assignees

Comments

@moozzyk
Copy link
Contributor

moozzyk commented Nov 3, 2016

If a poll connection times out the server throws:

System.InvalidOperationException occurred
  HResult=-2146233079
  Message=Concurrent reads are not supported.
  Source=System.Private.CoreLib
  StackTrace:
       at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Channels.Channel.Channels.IReadableBufferAwaiter.GetResult()
       at Microsoft.AspNetCore.Sockets.LongPolling.<ProcessRequestAsync>d__3.MoveNext() in C:\source\NetworkingProtoype\src\Microsoft.AspNetCore.Sockets\LongPolling.cs:line 21
  InnerException: 

and kills the connection. The client restarts the poll only to get:

GET http://localhost:5000/hubs/poll?id=87085e15-6bb5-492e-850b-f7965c7e9bbc&87085e15-6bb5-492e-850b-f7965c7e9bbc 500 (Internal Server Error)

@davidfowl
Copy link
Member

davidfowl commented Nov 4, 2016

This means the previous poll is still going. We need to define the behavior when a request is made to the same "connection". Do we reject the request or replace the previous one.

Old SignalR replaced the previous one.

@moozzyk
Copy link
Contributor Author

moozzyk commented Nov 4, 2016

We need to replace the previous one because it is the default client behavior - cancel a request after 110 secs of inactivity and start a new poll.

@davidfowl
Copy link
Member

davidfowl commented Nov 4, 2016

We do have to replace.

We need to replace the previous one because it is the default client behavior

That's not right. The server should end the response, not the client.

@moozzyk
Copy link
Contributor Author

moozzyk commented Nov 4, 2016

The client has to end long running requests otherwise IIS will terminate after 120 secs. That's why we have the 110 sec timeout on the client - or I am missing something.

@moozzyk
Copy link
Contributor Author

moozzyk commented Nov 4, 2016

or we move the timeout logic to the server somehow and we write 0 bytes after the timeout and the client restarts the poll

@davidfowl
Copy link
Member

or we move the timeout logic to the server somehow and we write 0 bytes after the timeout and the client restarts the poll

This is what old SignalR does. Do you think we should do the replace behavior for more than just long polling? I'm thinking we return a 409 conflict for other requests... (sse and websockets)

@moozzyk
Copy link
Contributor Author

moozzyk commented Nov 6, 2016

The old SignalR timeouts polls on the client.
There is no question that for sse and websockets we should reject a connect request if there is already an opened stream/websocket for this connectio id. For longpolling we may try handling reconnects by require the server to stop the poll requests but I am afraid of cases where something (e.g. a proxy) on the way terminates the current poll request and the client will try to start a new poll which will be rejected even though in theory it could/should just work.

@davidfowl
Copy link
Member

There is no question that for sse and websockets we should reject a connect request if there is already an opened stream/websocket for this connectio id

I have a feeling that we'll end up supporting reconnect everywhere after doing this.

It's server side https://github.com/SignalR/SignalR/blob/1fba14fa3437e24c204dfaf8a18db3fce8acad3c/src/Microsoft.AspNet.SignalR.Core/Transports/TransportHeartBeat.cs#L213-L217. The client side has a timeout as well but during normal operation it's never triggered.

@davidfowl davidfowl self-assigned this Nov 15, 2016
davidfowl added a commit that referenced this issue Jan 25, 2017
- The connection state object is manipulated by multiple parties in a non thread safe way. This change introduces a semaphore that should be used by anyone updating or reading the connection state. 
- Handle cases where there's an active request for a connection id and another incoming request for the same connection id, sse and websockets 409 and long polling kicks out the previous connection (#27 and #4)
- Handle requests being processed for disposed connections. There was a race where the background thread could remove and clean up the connection while it was about to be processed.
- Synchronize between the background scanning thread and the request threads when updating the connection state.
- Added `DisposeAndRemoveAsync` to the connection manager that handles`DisposeAsync` throwing and properly removes connections from connection tracking.
- Added Start to ConnectionManager so that testing is easier (background timer doesn't kick in unless start is called).
- Added RequestId to connection state for easier debugging and correlation (can easily see which request is currently processing the logical connection).
- Added tests
@davidfowl
Copy link
Member

Fixed

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants