Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a fallback data source when https://download.geonames.org/export/zip/{country}.zip fails #41

Closed
purusmahe opened this issue May 8, 2020 · 3 comments · Fixed by #46

Comments

@purusmahe
Copy link

With issues #40 and #34 , we see that genonames data source that we rely on seems to have intermittent availability issues. To protect against such intermittent failures can we think of adding a fallback data source or a cache. Both the issues were short lived and resolved by itself.

In Issue #40 @richunger pointed out that the data was available at http://download.geonames.org/export/dump/ when we had the issue, can we rely on that as a backup data source.

@rth
Copy link
Member

rth commented May 8, 2020

Thanks for opening this issue. The issue with http://download.geonames.org/export/dump is that last time I checked it didn't actually have files with exactly the same format as https://download.geonames.org/export/zip/ so we can't use it as a cache.

I think a better solution would be to store these files in a separate github repo, and then serve them via Github Pages as an alternate location. This would have the added advantage,

  • that we can do proper versioning of postal code date between different geodes updates (to see what changed)
  • potentially add patch files with missing or incorrect postal codes in the same repo (.e.g. Wrong mapping of Postal codes #23)

I have actually started https://github.com/symerio/postal-codes-data last time this happened but I think it may have files from the wrong URL. Also GB_full is too large to be added on Github unfortunately.

So what we would need is,

  1. a Github action on https://github.com/symerio/postal-codes-data to periodically download data from geonames.org and put it into this repo
  2. a mechanism in pgeocode to specify alternate URL locations.

Would anyone be interested in looking into it?

@dshinzie
Copy link

+1 on this. I am continually getting 404s for this package. Is this issue being looked at?

@rth
Copy link
Member

rth commented Oct 23, 2020

Some of the points from my above message were implemented in https://github.com/symerio/postal-codes-data. The fallback URL should then be,

https://symerio.github.io/postal-codes-data/data/geonames/<country-code>.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants