AEP-231: Batch Get easily hit max URL lenght

[kindermoumoute]

AEP-231 recommends to use the get verb for BatchGet methods. Using the get verb here means all request fields will be passed as query parameters. Passing a list of path in a query parameter will quickly hit the max URL lenght, which is 2048 characters.

Suggestion: use the post verb for BatchGet methods.

[thegagne]

Is there a reason you can't use the list endpoint with a filter? I guess the filter goes in query params as well.

[kindermoumoute]

Is there a reason you can't use the list endpoint with a filter? I guess the filter goes in query params as well.

TBH we use the same endpoint for listing and batch get. This issue can be generalized to any "big" filter in the request of a GET.

[toumorokoshi]

Although IIUC there is no hard limit to URL length (that's implementation dependent), I think it's a valid point that those URLs will stop working for in some clients, as well as practically unwieldy.

The issue with moving the payload into the POST is that the request is no longer cacheable - But I'm thinking we could add a clause that says you MAY support POST to help enable a much larger batch-get. This would allow the vast majority of requests to be cacheable but enable outliers.

Thoughts?

[thegagne]

Ideally we would support the QUERY method for these, that acts like cacheable posts, but I know support for QUERY is still a little spotty.

[toumorokoshi]

Oh that's great! Seems like exactly what we need.

When you say the support is spotty, what do you mean?

[thegagne]

I mean that not all api gateways, clients, or http libraries support QUERY. Even browsers do not support it yet. The proposal is a few years old but it has yet to be made into an approved RFC. Support is growing, and I think it is the proper solution, but it's not quite ready yet.

Also it's notable that some caching solutions do support custom caching on post requests, and I would maybe suggest we use a custom method, like resource:query with POST, until QUERY becomes supported. If a caching middleware supports caching POST by path, this might work.