Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add exception handling to download of json datasets #8

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

eUgEntOptIc44
Copy link
Contributor

As mentioned in issue #7 the GitHub action updating the json data from the Factbook source aborts on downloads from time to time. While the reason for the return HTTP error code 404 remains unclear integrating exception handling does seem reasonable to me. To my understanding it's the main objective of this repo to archive/mirror the contents of the Factbook source. So downloading as much as technically possible does serve this purpose. At least from my own point of view.

I'd like to point out that I lack any experience of coding in ruby so please double check the code for any typos.

@geraldb
Copy link
Member

geraldb commented Mar 25, 2022

Thanks for trying to start with adding more error handling. Sorry - from my point of view it is better to retry instead of skipping if a download fails. One idea is to add requests that fail to a queue and than retry after the first round is through / done - that should avoid trying to request the same page within a couple of seconds. That's my current thinking on how to proceed. Cheers. Prost.

PS: Just curious - what's your usage of the country datasets? Are you only interested in some countries? Or continents?

@eUgEntOptIc44
Copy link
Contributor Author

eUgEntOptIc44 commented Mar 25, 2022

Thanks for trying to start with adding more error handling. Sorry - from my point of view it is better to retry instead of skipping if a download fails. One idea is to add requests that fail to a queue and than retry after the first round is through / done - that should avoid trying to request the same page within a couple of seconds. That's my current thinking on how to proceed. Cheers. Prost.

ok. no worries. I suggest leaving this PR open till the underlaying issue is fixed.

PS: Just curious - what's your usage of the country datasets? Are you only interested in some countries? Or continents?

I use the datasets as a base for a info dashboard of my own witch merges data from multiple sources to a comprehensive overview per Country. To me it's important to have more than one point of view on important matters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants