Custom Aggregator does not receive downstream responses.

[MJeorrett]

Expected Behavior

The content of the responses is passed to the custom aggregator with 200 status codes.

Actual Behavior

Responses passed to implementation of the IDefinedAggregator are 404 and their bodies are empty (despite the actual downstream requests apparently succeeding). Perhaps I am missing something blatantly obvious but I am stumped. Could be that the documentation just hasn't quite caught up yet as this extension point seems to have drastically changed recently as noted in this issue: #1069.

Steps to Reproduce the Problem

I have created a minimal repro here: https://github.com/MJeorrett/OcelotAggregationIssue. It contains 2 self hosted aspnet core servers, running on localhost:5201 and localhost:5202, and an server running ocelot (including an aggregated edpoint) on localhost:5200.

Note: *If i downgrade the project to version 14.1.3 of Ocelot, the custom aggregator is passed the successful responses as expected, see this branch in the linked repo.

ocelot.json configures aggregate endpoint like so:

{
  "Routes": [
    {
      "DownstreamPathTemplate": "/downstream1",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "localhost",
          "Port": 5201
        }
      ],
      "UpstreamPathTemplate": "/api/downstream1",
      "UpstreamHttpMethod": [ "Get" ],
      "Key": "downstream1"
    },
    {
      "DownstreamPathTemplate": "/downstream2",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "localhost",
          "Port": 5202
        }
      ],
      "UpstreamPathTemplate": "/api/downstream2",
      "UpstreamHttpMethod": [ "Get" ],
      "Key": "downstream2"
    }
  ],
  "Aggregates": [
    {
      "RouteKeys": [
        "downstream1",
        "downstream2"
      ],
      "UpstreamPathTemplate": "/api/aggregate",
      "Aggregator": "MyAggregator"
    }
  ],
  "GlobalConfiguration": {
    "BaseUrl": "http://localhost:5200"
  }
}

MyAggregator is implemented like so:

public class MyAggregator : IDefinedAggregator
    {
        // Inject _logger in constructor

        public async Task<DownstreamResponse> Aggregate(List<HttpContext> responses)
        {
            var responseBodies = responses.Select(response =>
            {
                _logger.LogDebug($"Status code {response.Response.StatusCode}.");

                using var responseReader = new StreamReader(response.Response.Body);
                return responseReader.ReadToEnd();
            }).ToList();

            return new DownstreamResponse(
                new StringContent(JsonConvert.SerializeObject(responseBodies)),
                HttpStatusCode.OK,
                new List<Header>(),
                "OK");
        }
    }

If you look at the logs from the OcelotAggregation project it seems that the downstream requests succeed but the response passed to my custom aggregator are 404:

dbug: Ocelot.DownstreamUrlCreator.Middleware.DownstreamUrlCreatorMiddleware[0]
      requestId: 0HM177E53QAI3:00000001, previousRequestId: no previous request id, message: Downstream url is http://localhost:5202/downstream2
info: Ocelot.Requester.Middleware.HttpRequesterMiddleware[0]
      requestId: 0HM177E53QAI3:00000001, previousRequestId: no previous request id, message: 200 (OK) status code, request uri: http://localhost:5202/downstream2
dbug: Ocelot.Requester.Middleware.HttpRequesterMiddleware[0]
      requestId: 0HM177E53QAI3:00000001, previousRequestId: no previous request id, message: setting http response message
info: Ocelot.Requester.Middleware.HttpRequesterMiddleware[0]
      requestId: 0HM177E53QAI3:00000001, previousRequestId: no previous request id, message: 200 (OK) status code, request uri: http://localhost:5201/downstream1
dbug: Ocelot.Requester.Middleware.HttpRequesterMiddleware[0]
      requestId: 0HM177E53QAI3:00000001, previousRequestId: no previous request id, message: setting http response message
dbug: OcelotAggregation.MyAggregator[0]
      Status code 404.
dbug: OcelotAggregation.MyAggregator[0]
      Status code 404.

Specifications

• Version: Ocelot v16.0.1; netcoreapp3.1
• Platform: AspNet Core (self hosted) running on Windows.
• Subsystem:

[jlukawska]

Hello,
I've looked at the Ocelot tests and I've found that the responses in the custom aggregator are accessed differently.

var one = await responses[0].Items.DownstreamResponse().Content.ReadAsStringAsync();
var two = await responses[1].Items.DownstreamResponse().Content.ReadAsStringAsync();

And it actually works, both in the acceptance tests and in my sample program.

I think the docs should be updated with the sample Custom Aggregator. I'll try to make a PR soon.

[raman-m]

The issue has been accepted due to open PR #1330 by @jlukawska :

• #1287 Docs: Example of custom aggregator #1330

[raman-m]

@MJeorrett commented on Jul 13, 2020

Hi Mett!
Thanks for your interest in Ocelot!

I would say that your description is perfect, the best I've seen in backlog. 🥇

---

I have created a minimal repro here: https://github.com/MJeorrett/OcelotAggregationIssue. It contains 2 self hosted aspnet core servers, running on localhost:5201 and localhost:5202, and an server running ocelot (including an aggregated endpoint) on localhost:5200.

This is very important to create such mini-repos to be able to reproduce a bug easily.
I always ask the authors to upload the code to GitHub for open bugs.

---

Matt,
Could you review current open the PR #1330 please?
Thanks!