-
Notifications
You must be signed in to change notification settings - Fork 883
Description
Describe the bug
I'm not 100% sure it's a bug, but either it is, or something is unclear to me.
Occasionally* I'm seeing that ResponseTransformContext.ProxyResponse
is null
inside a delegate I pass to TransformBuilderContext.AddResponseTransform
, and I don't understand how that's possible. I see that the summary of ResponseTransformContext.ProxyResponse
says it "can be null if the destination did not respond", but I see from the logs of the proxied service that it did send a response (with Status code 401- Unauthorized, though I'm not sure it's related). Even if for some reason that response message was lost, then I would expect to have a way to get the error of the root cause (e.g. timeout, DNS issue, network disconnected, etc.). but I don't find any property that return that information. Note that I don't think that the problem is timeout because from the logs I see that I got the callback immediately after the proxied service responded.
*I saw it only twice out of 632 responses from the same endpoint (over a week period), and out of 9635 responses from all (3) endpoints. All of the responses from that endpoint return status 401. In most cases ResponseTransformContext.ProxyResponse
is not null
, but in those two cases out of 632 it is.
To Reproduce
I cannot reproduce it deterministically (and the chances to hit are very low as I mentioned above), but I can outline the structure of my code in regard to YARP. (It may be possible to reproduce it deterministically using load testing, but I don't have the time and resources for it now)
// In Bootstrapper:
private void ConfigureReverseProxy(IServiceCollection services)
{
services.AddReverseProxy().LoadFromMemory(new[]
{
new RouteConfig
{
RouteId = "route1",
ClusterId = "cluster1",
Match = new RouteMatch
{
Path = "{**catch-all}"
}
}
},
new[]
{
new ClusterConfig
{
ClusterId = "cluster1",
Destinations = new Dictionary<string, DestinationConfig>
{
["destination1"] = new() { Address = ConfigurationProvider.StsUrl }
}
}
})
.AddTransforms<StsProxyTransforms>();
}
internal class MyProxyTransforms : ITransformProvider
{
void ITransformProvider.ValidateRoute(TransformRouteValidationContext context)
{
}
void ITransformProvider.ValidateCluster(TransformClusterValidationContext context)
{
}
public void Apply(TransformBuilderContext context)
{
context.AddRequestTransform(async requestContext =>
{
// Here we examine the request and do some stuff, but we don't modify the request
// ...
});
context.AddResponseTransform(async responseContext =>
{
// THE FOLLOWING LINE OCCASIONALLY THROWS NullReferenceException BECAUSE responseContext.ProxyResponse is null...
if (!responseContext.ProxyResponse.IsSuccessStatusCode)
// ...
}
}
}
Further technical details
- Package: Yarp.ReverseProxy (2.1.0)
- The platform where the error occurs: .Net 6 on Linux
Activity
MihaZupan commentedon Aug 2, 2024
You can access the error via
httpContext.Features.Get<IForwarderErrorFeature>()
.It'll be set by the time you see a
null
response in a response transform.arnonax-tr commentedon Aug 4, 2024
Thanks @MihaZupan , I'll try that. I think it's worth mentioning in the docs of
ResponseTransformContext.ProxyResponse
arnonax-tr commentedon Aug 8, 2024
I tried it and now I see (in another endpoint BTW) the following information:
Does it give you any hint about the root cause? To me it still doesn't mean anything useful...
MihaZupan commentedon Aug 13, 2024
https://github.com/microsoft/reverse-proxy/blob/b1a1d8b20a2ff599517705771ce87614799e8ceb/src/ReverseProxy/Forwarder/ForwarderError.cs#L26-L29
RequestCanceled
generally indicates that the client (e.g. browser) disconnected/canceled the request before YARP was able to get a response from the backend server.This may be as innocent as the user navigating away to a different web page.
Updating the docs on ProxyResponse to hint at using
IForwarderErrorFeature
makes sense.arnonax-tr commentedon Aug 14, 2024
Thanks!