Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Geo lookup is broken #419

Closed
3 tasks done
acka47 opened this issue Apr 15, 2019 · 6 comments
Closed
3 tasks done

Geo lookup is broken #419

acka47 opened this issue Apr 15, 2019 · 6 comments
Assignees
Labels

Comments

@acka47
Copy link
Contributor

acka47 commented Apr 15, 2019

For example, all entries that come from the DBS have no geo coordinates, see https://lobid.org/organisations/search?q=_exists_%3AdbsID+AND+NOT+_exists_%3Aisil+AND+_exists_%3Alocation.geo

The problem is that the pelias index is missing from aither, see hbz/nwbib#463 (comment).

To do:

  • Fix index
  • adjust API URL in lobid-organisations if necessary
  • add tests for geo lookup so that we from now on learn of such a problem with the pelias index directly
@acka47 acka47 added the bug label Apr 15, 2019
@dr0i
Copy link
Member

dr0i commented Apr 23, 2019

The old installation is way too outdated and thus I decided not to fix the old one but to install a new one based on docker. Experienced an issue and opened a PR.

@dr0i
Copy link
Member

dr0i commented May 20, 2019

Set up Pelias as docker containers at lod@gaia:pelias/projects/lobid_DACH. Test like http://gaia.hbz-nrw.de:4000/v1/search?text=berlin . Not sure if all the data/functionality is good, though. Going to reconfigure and restart organisations-test to see the outcome.

@dr0i
Copy link
Member

dr0i commented May 20, 2019

https://stage.lobid.org/organisations/search?q=_exists_%3AdbsID+AND+NOT+_exists_%3Aisil+AND+_exists_%3Alocation.geo now result in ~700. But is this suffcient? Querying missing geo location data result in around 10 times more documents.

@dr0i dr0i assigned acka47 and unassigned dr0i May 20, 2019
@acka47
Copy link
Contributor Author

acka47 commented May 21, 2019

No, that is not sufficient. Although I don't know the numbers from the last time Pelias was running, I am quite sure there must be a problem somewhere.

@dr0i
Copy link
Member

dr0i commented May 23, 2019

There was the polylines data missing. Also, the whosonfirst filenames of the data had to be renamed to be taken into account. After pelias compose down; pelias compose up and reindexing lobid-organisations there are ~5k dbsIDs with a geo location, but still 3k without. Still have to dig a bit deeper to get the proper data I guess.

@dr0i
Copy link
Member

dr0i commented May 23, 2019

In #89 it is stated that ~1.2k are without geo data. So now we have 1.8k less than before. Discussed with @acka47: we will leave it as it is until someone is asking for improvement.

@dr0i dr0i closed this as completed May 23, 2019
dr0i added a commit that referenced this issue May 23, 2019
The new pelias instance is used for geo lookups.
Also, after transformations a curl is done do check if enough there is
enough geo data for the dbs IDs. If not, a mail is send.

See #419.
dr0i added a commit that referenced this issue May 23, 2019
Fixes the failing build of travis.

See #419.
dr0i added a commit to hbz/pelias-docker that referenced this issue May 23, 2019
dr0i added a commit to hbz/pelias-docker that referenced this issue May 23, 2019
dr0i added a commit to hbz/pelias-docker that referenced this issue Jun 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants