HttpRequestException on sending response

[greatvovan]

I use Swashbuckle to expose HTTP API in my service (self-hosted web application). From time to time I see in log errors like this:

2017-05-25 16:19:41 [17] ERROR - Unhandled exception. Request details: 
Method: POST, RequestUri: 'http://localhost:8082/extract-info', Version: 1.1, Content: System.Net.Http.StreamContent, Headers:
{
  Connection: close
  Accept: application/json
  Accept-Encoding: gzip
  Accept-Encoding: deflate
  Host: localhost:8082
  User-Agent: Python-urllib/3.5
  Content-Length: 3980
  Content-Type: application/json
}
System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException ---> System.Net.HttpListenerException: The I/O operation has been aborted because of either a thread exit or an application request
   at System.Net.HttpResponseStream.EndWrite(IAsyncResult asyncResult)
   at Microsoft.Owin.Host.HttpListener.RequestProcessing.ExceptionFilterStream.EndWrite(IAsyncResult asyncResult)
   --- End of inner exception stack trace ---
   at Microsoft.Owin.Host.HttpListener.RequestProcessing.ExceptionFilterStream.EndWrite(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
   --- End of inner exception stack trace ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Owin.HttpMessageHandlerAdapter.<SendResponseContentAsync>d__20.MoveNext()

It reports that IO operation was aborted, altough client receives response completely and does not notice any problems.

I am posting the issue here because Swashbuckle uses Owin under its hood and stack trace points to Owin's code.

Initially I came across this issue on a slow machine and was unable to reproduce it on my computer. Then I ran a program which uses 100% CPU (any CPU performance test would do) and managed to reproduce it on my machine. I assume it can be some a kind of race conditions.

I don't think the details of my code are really helpful here because I managed to reproduce this error in extremely easy console application.

namespace My
{
  public class MyController : ApiController
  {
    [Route("extract-info")]
    [HttpPost]
    public ResultType ExtractInfo([FromBody] InputType inputData, bool option = false)
    {
      Thread.Sleep(rnd.Next(1000, 5000)); // Simulate processing.
      return GenerateResultType();  // Generate random/constant data.
    }
  }
}

The ResultType is a C# class, instances of this class are serialized to JSON like this:

{
  "list": [
    {
      "a": "Some text",
      "b": 12759966,
      "c": false,
      "d": true,
      "e": "ORG",
      "f": 5,
      "g": 6,
      "h": -1000.5,
      "i": false
    },
    ...
  ],
  "dict1": {
    "802474": 1.1,
    "4005": 0.293677663774,
    ...
  },
  "dict2": {
    "1141": 0.8998012898171055,
    "24005": 0.993677663774,
    ...
  }
}

The client (located on the same machine) makes requests from 10 parallel threads, each thread issues requests continiously one after another. The size of request data is 1-10KB, the size of returned data is 150-300KB. Due to Sleep() in request processing method the total throughput is not so high, about 3-4 requests per second.

Does anybody have any ideas about these exceptions?

[Tratcher]

Can you capture a network trace of this using Message Analyzer or Wireshark? I wonder if the connection is getting closed before the data is ACKd back to the server.

[greatvovan]

OK Tratcher, will do it soon.

[lanierhall]

Comin' up on the one year anniversary of the commitment to do it soon 😄