Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Remote host closed the connection" unhandled exception polluting Elmah logs #4410

Closed
TechSavvySam opened this issue Sep 26, 2019 · 4 comments

Comments

@TechSavvySam
Copy link

@TechSavvySam TechSavvySam commented Sep 26, 2019

Note that this issue was previously logged but was closed during the backlog bankruptcy.

#3296

Expected behavior

By default, SignalR should handle remote host unexpectedly closing connections since this is the reality of life in a network environment. Users turn off their systems or close browser windows w/o logging out of web sites.

Actual behavior

Two main exceptions go unhandled and pollute the Elmah logs to the point they are unusable. I would say on my production web site, at least 80% of the errors in Elmah are caused by SignalR.

The request URLs are:

/signalr/start
/signalr/poll/
/signalr/hubs

The exceptions are:

System.Web.HttpException

The client disconnected.

System.Web.HttpException (0x80070040): The client disconnected.
   at System.Web.Hosting.IIS7WorkerRequest.BeginRead(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback callback, Object state)
   at System.Web.HttpBufferlessInputStream.BeginRead(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback callback, Object state)
   at System.IO.Stream.<>c.<BeginEndReadAsync>b__43_0(Stream stream, ReadWriteParameters args, AsyncCallback callback, Object state)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncTrim[TInstance,TArgs](TInstance thisRef, TArgs args, Func`5 beginMethod, Func`3 endMethod)
   at System.IO.Stream.BeginEndReadAsync(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at Microsoft.Owin.Host.SystemWeb.CallStreams.DelegatingStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at System.IO.Stream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.StreamReader.<ReadBufferAsync>d__97.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.IO.StreamReader.<ReadToEndAsyncInternal>d__62.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.OwinRequest.<ReadFormAsync>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNet.SignalR.Owin.ServerRequest.<ReadForm>d__20.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Mapping.MapMiddleware.<Invoke>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContextStage.<RunApp>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Security.Infrastructure.AuthenticationMiddleware`1.<Invoke>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContextStage.<RunApp>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContext.<DoFinalWork>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.StageAsyncResult.End(IAsyncResult ar)
   at System.Web.HttpApplication.AsyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

System.Web.HttpException

The remote host closed the connection. The error code is 0x80070057.

System.Web.HttpException (0x80070057): The remote host closed the connection. The error code is 0x80070057.
   at System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result, Boolean throwOnDisconnect)
   at System.Web.Hosting.IIS7WorkerRequest.ExplicitFlush()
   at System.Web.HttpResponse.Flush(Boolean finalFlush, Boolean async)
   at System.Web.HttpWriter.WriteFromStream(Byte[] data, Int32 offset, Int32 size)
   at Microsoft.Owin.Host.SystemWeb.CallStreams.OutputStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at Microsoft.AspNet.SignalR.Owin.ServerResponse.Write(ArraySegment`1 data)
   at Microsoft.AspNet.SignalR.Hosting.ResponseExtensions.End(IResponse response, String data)
   at Microsoft.Owin.Mapping.MapMiddleware.<Invoke>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContextStage.<RunApp>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Security.Infrastructure.AuthenticationMiddleware`1.<Invoke>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContextStage.<RunApp>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContext.<DoFinalWork>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.StageAsyncResult.End(IAsyncResult ar)
   at System.Web.HttpApplication.AsyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Steps to reproduce

Easiest way to see the issue occurring is to run Elmah and SignalR on a system. That way you can see the errors.

I have yet to figure out a definitive method for reproducing this, but my users have an uncanny ability to make it happen many times per day.

If I figure out a way to consistently reproduce it, I'll update this ticket.

@anurse

This comment has been minimized.

Copy link
Contributor

@anurse anurse commented Oct 1, 2019

Is ELMAH's filtering support not sufficient for suppressing these messages?

In ASP.NET Core, we have indeed suppressed most of these messages because you are correct they aren't terribly useful. Our limitation here is breaking changes. Suppressing these messages would negatively affect users who are expecting them. We could make it a config flag that allows you to suppress the exceptions but a) that isn't cheap in terms of time and resources and b) we don't have plans to ship a new minor version (in which an API change could be made) to do so.

If the ELMAH filtering is able to reduce these logs sufficiently for you, I'd strongly encourage using that. If not, let us know and we can look at our options.

@TechSavvySam

This comment has been minimized.

Copy link
Author

@TechSavvySam TechSavvySam commented Oct 2, 2019

Thanks--with all my searching, I never used a phrase that pointed me to Elmah's filtering support. I just wrote some filter rules and will deploy them early next week. I'll report back here what I did to help the community and then very likely close the ticket.

@TechSavvySam

This comment has been minimized.

Copy link
Author

@TechSavvySam TechSavvySam commented Oct 14, 2019

Thanks. Was finally able to deploy code where it could be tested and, YES, this is a reasonable work-around.

For anyone else that stumbles upon this thread, here is what I added to the web.config to exclude my specific errors:

  <elmah>
    <!--
        Other settings omitted for this illustration
    -->
    <errorFilter>
      <test>
        <or>
          <and>
            <equal binding="HttpStatusCode" value="500" type="Int32" />
            <regex binding="Exception.Message" pattern="The remote host closed the connection\. The error code is 0x80070057\..*" />
            <regex binding="Context.Request.ServerVariables['URL']" pattern="/signalr/.*" />
          </and>
          <and>
            <equal binding="HttpStatusCode" value="500" type="Int32" />
            <regex binding="Exception.Message" pattern="The client disconnected\..*" />
            <regex binding="Context.Request.ServerVariables['URL']" pattern="/signalr/.*" />
          </and>
        </or>
      </test>
    </errorFilter>
  </elmah>

@anurse

This comment has been minimized.

Copy link
Contributor

@anurse anurse commented Oct 14, 2019

Glad to hear it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.