Grpc-web experimental and UseHttpSys()?

[omaristalis]

Hi there,

As part of the grpc-web experiment, is HTTP.SYS supported from an app not running in Azure or IIS?

For a little background of how I got here, I successfully ran a server using Kestrel and grpc-web. However, I tried to convert that to use HTTP.SYS, and I get the following invalid operation exception:
"The response headers cannot be modified because the response has already started"

Before I spend more time on this, it would be good to know if it can even work!

Hosting in this way is a specific use case for what we want to do.

Thanks,

Darren

[omaristalis]

For reference, the stack is here:

fail: Grpc.AspNetCore.Server.ServerCallHandler[6]
      Error when executing service method 'Test'.
System.InvalidOperationException: The response headers cannot be modified because the response has already started.
   at Microsoft.AspNetCore.HttpSys.Internal.HeaderCollection.ThrowIfReadOnly()
   at Microsoft.AspNetCore.HttpSys.Internal.HeaderCollection.set_Item(String key, StringValues value)
   at Grpc.AspNetCore.Server.Internal.GrpcProtocolHelpers.SetStatus(IHeaderDictionary destination, Status status)
   at Grpc.AspNetCore.Server.Internal.HttpResponseExtensions.ConsolidateTrailers(HttpResponse httpResponse, HttpContextServerCallContext context)
   at Grpc.AspNetCore.Server.Internal.HttpContextServerCallContext.EndCallCore()
   at Grpc.AspNetCore.Server.Internal.HttpContextServerCallContext.EndCallAsync()
   at Grpc.AspNetCore.Server.Internal.CallHandlers.ServerCallHandlerBase`3.HandleCallAsync(HttpContext httpContext)
fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
      An unhandled exception has occurred while executing the request.
System.InvalidOperationException: The response headers cannot be modified because the response has already started.
   at Microsoft.AspNetCore.HttpSys.Internal.HeaderCollection.ThrowIfReadOnly()
   at Microsoft.AspNetCore.HttpSys.Internal.HeaderCollection.set_Item(String key, StringValues value)
   at Grpc.AspNetCore.Server.Internal.GrpcProtocolHelpers.SetStatus(IHeaderDictionary destination, Status status)
   at Grpc.AspNetCore.Server.Internal.HttpResponseExtensions.ConsolidateTrailers(HttpResponse httpResponse, HttpContextServerCallContext context)
   at Grpc.AspNetCore.Server.Internal.HttpContextServerCallContext.ProcessHandlerError(Exception ex, String method)
   at Grpc.AspNetCore.Server.Internal.HttpContextServerCallContext.ProcessHandlerErrorAsync(Exception ex, String method)
   at Grpc.AspNetCore.Server.Internal.CallHandlers.ServerCallHandlerBase`3.HandleCallAsync(HttpContext httpContext)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext)
--- End of stack trace from previous location where exception was thrown ---
   at Grpc.AspNetCore.Web.Internal.GrpcWebMiddleware.HandleGrpcWebRequest(HttpContext httpContext, ServerGrpcWebMode mode)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[3]
      An exception was thrown attempting to display the error page.
System.InvalidOperationException: The response headers cannot be modified because the response has already started.
   at Microsoft.AspNetCore.HttpSys.Internal.HeaderCollection.ThrowIfReadOnly()
   at Microsoft.AspNetCore.HttpSys.Internal.HeaderCollection.Clear()
   at Microsoft.AspNetCore.Http.ResponseExtensions.Clear(HttpResponse response)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
fail: Microsoft.AspNetCore.Server.HttpSys.MessagePump[0]
      ProcessRequestAsync
System.InvalidOperationException: The response headers cannot be modified because the response has already started.
   at Microsoft.AspNetCore.HttpSys.Internal.HeaderCollection.ThrowIfReadOnly()
   at Microsoft.AspNetCore.HttpSys.Internal.HeaderCollection.set_Item(String key, StringValues value)
   at Grpc.AspNetCore.Server.Internal.GrpcProtocolHelpers.SetStatus(IHeaderDictionary destination, Status status)
   at Grpc.AspNetCore.Server.Internal.HttpResponseExtensions.ConsolidateTrailers(HttpResponse httpResponse, HttpContextServerCallContext context)
   at Grpc.AspNetCore.Server.Internal.HttpContextServerCallContext.ProcessHandlerError(Exception ex, String method)
   at Grpc.AspNetCore.Server.Internal.HttpContextServerCallContext.ProcessHandlerErrorAsync(Exception ex, String method)
   at Grpc.AspNetCore.Server.Internal.CallHandlers.ServerCallHandlerBase`3.HandleCallAsync(HttpContext httpContext)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext)
--- End of stack trace from previous location where exception was thrown ---
   at Grpc.AspNetCore.Web.Internal.GrpcWebMiddleware.HandleGrpcWebRequest(HttpContext httpContext, ServerGrpcWebMode mode)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Server.HttpSys.MessagePump.ProcessRequestAsync(Object requestContextObj)
   at Microsoft.AspNetCore.Server.HttpSys.MessagePump.ProcessRequestAsync(Object requestContextObj)

[analogrelay]

Triage: @JamesNK will take a look. The main issue here is that things like the status code and headers can't be modified after the server starts to send the response. gRPC-Web shoudn't be causing this though.

@omaristalis can you post a runnable sample that reproduces the problem? That would help us understand the full context.

[JamesNK]

The problem is HttpSys in .NET Core 3.1 doesn't implement a feature required by gRPC. That is already fixed the next release, 5.0, but a work-around is required in 3.1.

5.0 issue: dotnet/aspnetcore#16987

Add this class to your web app:

public class HttpSysWorkaroundHttpResponseFeature : IHttpResponseFeature
{
    private readonly IHttpResponseFeature _inner;

    public HttpSysWorkaroundHttpResponseFeature(IHttpResponseFeature inner)
    {
        _inner = inner;
        _inner.OnStarting(o =>
        {
            HasStarted = true;
            return Task.CompletedTask;
        }, null);
    }

    [Obsolete]
    public Stream Body
    {
        get => _inner.Body;
        set { _inner.Body = value; }
    }
    public bool HasStarted { get; private set; }
    public IHeaderDictionary Headers
    {
        get => _inner.Headers;
        set { _inner.Headers = value; }
    }
    public string ReasonPhrase
    {
        get => _inner.ReasonPhrase;
        set { _inner.ReasonPhrase = value; }
    }
    public int StatusCode
    {
        get => _inner.StatusCode;
        set { _inner.StatusCode = value; }
    }

    public void OnCompleted(Func<object, Task> callback, object state)
    {
        _inner.OnCompleted(callback, state);
    }

    public void OnStarting(Func<object, Task> callback, object state)
    {
        _inner.OnStarting(callback, state);
    }
}

And register this middleware delegate after UseGrpcWeb:

app.UseGrpcWeb();

// Add after existing UseGrpcWeb
app.Use((c, next) =>
{
    if (c.Request.ContentType == "application/grpc")
    {
        var current = c.Features.Get<IHttpResponseFeature>();
        c.Features.Set<IHttpResponseFeature>(new HttpSysWorkaroundHttpResponseFeature(current));
    }
    return next();
});

gRPC-Web should now work on HttpSys. After you upgrade to .NET 5.0 you can remove this code.

[omaristalis]

Perfect! That solves it for me. Thank you so much for the speedy response!