valhalla_build_transit and transit.land limits

[molind]

I've tried to download transit information from transit.land. And now seems i've been blacklisted for overuse without api key. :( Is there a way to download and build transit information without heavy load on transit.land services? May be dump of transit information is available somewhere?

'''
evgen@vmap1:~/data$ valhalla_build_transit valhalla.json http://transit.land/ 100 ./transit_tiles
2017/08/07 13:16:28.346124 [INFO] Fetching transit feeds
2017/08/07 13:16:34.317941 [WARN] 429'd retrying http://transit.land//api/v1/feeds.geojson?per_page=false
'''

[drewda]

@molind thanks for your interest in Valhalla and in Transitland!

We are intending to release static data extracts from Transitland in the future -- that is the ideal way to support a wide range of users. In the meantime, we do need to meter the use of the Transitland API so that it's equally available to all. Users should not be blacklisted from the API, only throttled. Please try spacing out your requests so they are no greater than 8 per second and 12,000 per day.

We'll look forward to sharing more information in the future about static data extracts.

[molind]

@drewda thanks for your reply. I'm not trying to run something my own. Just trying to build transit data for valhalla using provided app from this repo (valhalla_build_transit). Without changing code i can't make it work as you said. And I'm not sure if it's even possible to download everything it needs in 12k queries. I agree that there is should be throttling to make service equally available to all. But from the other side may be there should be some mirror with generated data that's easy to fetch. Otherwise valhalla_build_transit makes no sense.

[drewda]

@molind for the time being, you may only be able to download data for a smaller geographic extent.

If you need immediate worldwide transit coverage, another alternative to consider: Mapzen hosts an instance of Valhalla under the name Turn-by-Turn: https://mapzen.com/products/turn-by-turn/ Mapzen Turn-by-Turn includes all transit service provided by Transitland.

[mzagaja]

@drewda Is there documentation for running valhalla_build_transit constrained to the appropriate API limits? I landed on this issue page trying to figure out how to add transit to my valhalla instance and attempted the following:

ubuntu@ip-10-79-196-140:~/valhalla$ valhalla_build_transit
Usage: valhalla_build_transit valhalla_config transit_land_url per_page [target_directory] [transit_land_api_key]
Sample: valhalla_build_transit conf/valhalla.json http://transit.land/ 1000 ./transit_tiles transitland-YOUR_KEY_SUFFIX
ubuntu@ip-10-79-196-140:~/valhalla$ valhalla_build_transit valhalla.json http://transit.land/ 1000 ./transit_tiles
2017/08/24 20:58:42.331457 [INFO] Fetching transit feeds
2017/08/24 20:58:55.219007 [WARN] 500'd retrying http://transit.land//api/v1/feeds.geojson?per_page=false
2017/08/24 20:59:09.735712 [WARN] 500'd retrying http://transit.land//api/v1/feeds.geojson?per_page=false

[mzagaja]

I discovered that part of the issue is that the transit_land_url needs to not end with a trailing slash. It started to work but is now giving me 429 rate limit errors. It doesn't seem to have any logic or documentation around the rate limits.

[arminakvn]

@drewda you mentioned: " ... you may only be able to download data for a smaller geographic extent", which is in fact what I want to do. How could I do this when building transit tiles? I just wanted to build transit tiles for one metropolitan area, say metro Boston.

[val-gh]

@drewda are there any updates on static data extracts from Transitland?
I've tried running valhalla_build_transit with a bounding box for the CONUS but I'm getting 429 rate limit errors.

[toton6868]

@drewda are there any updates on static data extracts from Transitland?
@drewda are there any updates on static data extracts from Transitland?I've tried running valhalla_build_transit with a bounding box for the CONUS but I'm getting 429 rate limit errors.

Probably Transitland drops their previous plan of Static data extract from Transitland. I am from a country where there is no Transit data exist openly. I paid and develop a Transit data. For certain time our company do not want to reveal the data to open world. So there is no way to use my data with Transitland and Valhalla. Probably our choice was wrong about valhalla. We are thinking about shifting to GraphHopper.

[gekpp]

Hi everybody. Is there any updates on static data extracts? I tried today, got 429 😔

[YashowardhanTripathi]

Any updates related to this issue.....?

Things we tried so far are:-

• We tried with different size and location bounding boxes (Hoping it will reduce the number of queries on transit-land server).
• We tried inserting sleep function for reducing number of hits.
• We tried this method also transitland_issue.

Above all done hence we assumed that transit-land has variable (not fixed) rate limit.

Kindly, Can you provide relevant code/script changes for this issue?