Improving format_response performance

[xuorig]

format_response is a procedure that takes in the original GraphQL query as well as the federated response, and filters and reorders the response based on the input query. It does two main operations, filtering out unrequested fields, and ordering response fields based on where they were requested in the original query, as written in the GraphQL-Spec (https://spec.graphql.org/October2021/#sec-Objects.Field-Ordering).

In our production system, this procedure accounts for about 16% of the CPU time. Down from about 19-20% since this change. JSON ser/de and format_response are the bulk of what the router is doing, which is good news!

CPU Flamegraph

Most of the CPU time is spent allocating memory and spending time in various parts of IndexMap. GraphQL responses can grow very large, and a new solution would probably want to avoid allocating a brand new index map for the formatted response. Opening this issue to talk through potential solutions.

Throwing ideas in this issue to kickstart a discussion.

• A better solution here would of course to remove the need of format_response entirely. Is there a way to build the response during plan execution so that it require minimal, or no post processing? The problem is that at this point we don't currently have the sorting information available. But what if query planning gave us that information? It could provide information to the executor about fields that should be omitted, the index at which fields should be inserted, etc.
• A potentially shorter term solution is to sort and modify the response in place instead of creating a brand new response. A higher overhead than inserting elements in the right order in the first place, but involves less complexity and different components, the fix can be scoped down to this procedure only.

Thoughts? Other ideas?