Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retryer #35

Merged
merged 3 commits into from
Aug 15, 2018
Merged

Retryer #35

merged 3 commits into from
Aug 15, 2018

Conversation

ljwolf
Copy link
Member

@ljwolf ljwolf commented Jul 19, 2018

This adds functionality to wait/retry between web requests.

This is necessary for some map APIs, especially when downloading large/high-detail tilesets.

This makes contextily kind of like a general purpose tile downloader for any API in the tile-url format, and I've found it extremely handy to work with rate-limited scientific APIs.

@ljwolf
Copy link
Member Author

ljwolf commented Jul 19, 2018

Hmm, strange; works locally, but I removed six.BytesIO for io.BytesIO, a three-only construct.

I think this may have something to do with using six.BytesIO rather than the python3 builtin io.BytesIO. Maybe six doesn't implement context handling.

@darribas
Copy link
Collaborator

darribas commented Aug 5, 2018

I've finally had a look at the functionality, this is great. If you add (passing) tests, I'll merge this. Hopefully it can make it to the upcoming rc!

@ljwolf ljwolf reopened this Aug 6, 2018
@ljwolf
Copy link
Member Author

ljwolf commented Aug 6, 2018

Not sure why this closed, but I had to reopen this to get the merge commit to show.

Also, not sure how to mock this functionality in a unittest.

@coveralls
Copy link

coveralls commented Aug 6, 2018

Coverage Status

Coverage decreased (-3.5%) to 90.0% when pulling 508da93 on ljwolf:retryer into 8e5039b on darribas:master.

@darribas
Copy link
Collaborator

darribas commented Aug 7, 2018

OK fair enough. I guess it is tested it at least works in any other method that pulls down tiles. One more thing, should we then add requests to the requirements.txt? It currently works so I imagine it's required by other libraries (geopy?) but it'd be good to have the explicit requirement. I'll merge afterwards.

@ljwolf
Copy link
Member Author

ljwolf commented Aug 9, 2018

I don't think requests is required by geopy, but it's used in a large amount of web-touching code. I also use it in cenpy.

I mainly like it for the raise_for_status() functionality.

I've added it to the requirements.

@darribas
Copy link
Collaborator

Cool!

@darribas darribas merged commit 8d787da into geopandas:master Aug 15, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants