Python API can send metrics (grid) data which is too big #906
Description
In the FDS connector we've seen an example with over 120 grids. It seems that the connector & hence Python API tries to send all this data to the server as a single huge API request. This is bigger than 50 MB, which was originally the max size set in nginx.
At the moment if someone had e.g. 1000 grids it would be far, far worse :-)
Maybe the Python API could check the approximate size of the request before actually doing it, and if necessary, split it into multiple smaller requests? 10 MB is probably a reasonable limit per request.
Doing this in the Python API would mean all connectors (and users) would not need to worry about doing this themselves.
Our current infrastructure is not really designed to enable the REST API to support such large amounts of data, especially if it came at a high rate.