UI option to specify a Google Maps API key

[geopast]

It you have a CSV with many rows that need geocoding, you quickly hit the rate limit set by Google for public API calls. Specifying an API key in the request should solve the rate limit.

[piotr-cz]

Hi, at the moment there is implemented a delay trottle of 250ms which should be enough to work properly. But my sample data database consists of only 20 records.

Before I prepare an UI for specyfing Google Maps API key, you may try a hack in cockpit/modules/addons/Import/assets/js/filters.js:

1. Add API Key here:

  google.load('maps', '3', {other_params: 'libraries=places&language=' + locale + '&key=YOUR_API_KEY', callback: function() {

2. Change throttling delay here to 0:

  locationCockpitFilter.delay = 0;

I'm not able to try it out, because I don't own API key myself and I'm not sure this hack will work, because I'm using undocumented method to load an Google Maps API asynchronously.

Please let me know if hack worked and I'll prepare an UI option for that. Besides I'd like to store options in cockpit datastore so users don't have to specify them every time they want to use import.

[geopast]

Hi, thanks for the quick update. I tried it out and it starts geocoding but errors after 25 lookups. This appears to be the same as when the the key is not provided. I also put the throttling back with the key still in place to see if that made a difference - it starting importing slower (as I expected) but then errored (see below). The error occurs consistently at 25 entries of import with or without the key being specified. I've also tried different data and received the same result.

[piotr-cz]

It seems that we are not passing the API key properly - you should not be receiving that error. I'll investigate it later.

You may try to increase the delay to let's say 2000ms but of course it will result in slower processing.
I've been able to import 22 entries with delay of 250ms. Documentation specifies 10 queries per second (delay 100ms) but I've been receiving same OVER_QUERY_LIMIT after ~15 entries.

Another solution I see is to refactor geolocation to try again when got OVER_QUERY_LIMIT error until success as described here.

We could also use backend gecoding API, which has much more relaxed limits.

[piotr-cz]

Hold on, documentation, Premium plan usage limits says that even Premium plan has quota of 50 requests per second (throttling delay 20ms) so some minimum delay is always required.

[piotr-cz]

I've added an option to specify Google Maps API Key in Cockpit master branch in this PR: agentejo/cockpit#423 because since 22/06/2016 any usage of Google Maps requires using a key.

Key is now picked by the import module since v1.1.2.

However, I'm still getting OVER_QUERY_LIMIT error when geolocating more results (probably over 255) so unfortunately this hasn't changed. There is a chance this happens because I've been testing it in development environment on localhost.

Anyway feature is added.

[piotr-cz]

I noticed that when I reach OVER_QUERY_LIMIT error (even on paid plan) module tends to receive same errors when run again after ~10 requests.

I think that Google JS Geocoding API is primarily targeted to use by humans and so throttles batch usage.
Reloading module page helps here, maybe Geocoding API opens up new session with fresh quota that is bound to new instance.

Quote from Google Maps Platform > Usage Limits for Google Maps Platform Web Services:

Note: different limits apply to client-side APIs. Maps JavaScript API are rate limited per map session, so that requests are distributed across users. ...