Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: adding POST support for query many locations. #49

Closed
iisri-vu opened this issue Sep 3, 2021 · 4 comments
Closed

Feature request: adding POST support for query many locations. #49

iisri-vu opened this issue Sep 3, 2021 · 4 comments

Comments

@iisri-vu
Copy link

iisri-vu commented Sep 3, 2021

The GET request has a limitation with uri length.
We need a POST to increase this limitation.

@ajnisbet
Copy link
Owner

ajnisbet commented Sep 4, 2021

Thanks for raising this, I've held off because it doesn't align with http semantics, but I could see it being useful. I'll have a look at what other APIs do.

I have some questions that would help me prioritise this issue:

  • How many locations are you requesting?
  • What error do you get?
  • If you're running into issues with the software you're using (rather than the opentopodata server) what software is that?

In the meantime there's a couple of things you can do:

  • Use polyline encoding.
  • Round your coordinates: check out this table for a rough sense of the precision of lat/lon coordinates.

@iisri-vu
Copy link
Author

iisri-vu commented Sep 4, 2021

Thanks Andrew,

I'm currently running my Flask server with blue print tech for GET and POST to stream road information.

I'm trying to get elevation of roads geometry, which may need a fair number of points.

I would like to host your API on our server to serve large number of locations query.

What you could do is have option on the POST query to limit the number of points. If we could set this limit value on our own build. This way it won't change your previous implementation or overload your server.

I have used open elevation. But it takes 20GB of rams for just one map and runs very slow on my high end laptop, running the docker service.

I'm currently evaluate your API. I saw it uses smaller map junks, which is great for tree search optimization and memory.

I'm currently downloading the 90m files. This seems to take for ever.

@iisri-vu
Copy link
Author

iisri-vu commented Sep 14, 2021

This is a minimal example of POST, without changing to the existing code.
Sorry I did not check for the exact error in the code.

the method accept: json: {"dataset_name":"srtm90m", "locations": latlngs, "interpolation":"cubic"}
latlngs = [[lat, lon], [lat, lon], [lat,lon],...]

By the way, I think returning a dataset name in the response is a waste of bandwidth. Maybe we can remove that.
Thanks

@app.route("/v1/elevation",  methods=["POST"])
def post_elevation():
    """Calculate the elevation for the given locations.
    Returns:
        Response.
    """
    try:
        if(request.method == 'POST'):
            obj = request.get_json()
            dataset_name = obj['dataset_name']
            latlons = obj['locations']
            if (obj['interpolation'] == None):
                interpolation = DEFAULT_INTERPOLATION_METHOD 
            else: 
                interpolation = obj['interpolation']

            if(len(latlons) > _load_config()["max_locations_per_request"]):
                return (
                    jsonify({"status": "SERVER_ERROR", "error": "Config Error: {}".format("too many location specified")}),
                    400,)

            #Convert list of (lat, lon) tuples to lat and lon lists.
            lats = [p[0] for p in latlons]
            lons = [p[1] for p in latlons]
            nodata_value = None

            # Get the z values.
            datasets = _get_datasets(dataset_name)
            elevations, dataset_names = backend.get_elevation(
                lats, lons, datasets, interpolation, nodata_value
            )
            
            # Build response.
            results = []
            for z, dataset_name, lat, lon in zip(elevations, dataset_names, lats, lons):
                results.append(
                    {
                        "elevation": z,
                        "dataset": dataset_name,
                        "location": {"lat": lat, "lng": lon},
                    }
                )
  
            data = {"status": "OK", "results": results}
            return jsonify(data)

    except (ClientError, backend.InputError) as e:
        return jsonify({"status": "INVALID_POST", "error": str(e)}), 400
    except config.ConfigError as e:
        return (
            jsonify({"status": "SERVER_ERROR", "error": "Config Error: {}".format(e)}),
            500,
        )
    except Exception as e:
        if app.debug:
            raise e
        app.logger.error(e)
        msg = "Unhandled server error, see server logs for details."
        return jsonify({"status": "SERVER_ERROR", "error": msg}), 500

@ajnisbet
Copy link
Owner

ajnisbet commented Oct 7, 2021

Added in 920f110

@ajnisbet ajnisbet closed this as completed Oct 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants