Skip to content

Update wikidata reconciliation endpoint#4206

Merged
wetneb merged 1 commit into
OpenRefine:masterfrom
PabloCastellano:wikidata-reconciliation-endpoint
Oct 20, 2021
Merged

Update wikidata reconciliation endpoint#4206
wetneb merged 1 commit into
OpenRefine:masterfrom
PabloCastellano:wikidata-reconciliation-endpoint

Conversation

@PabloCastellano
Copy link
Copy Markdown
Contributor

Fixes #4205

Changes proposed in this pull request:

  • Change to new wikidata reconciliation endpoint

@probot-autolabeler probot-autolabeler Bot added the Type: Documentation Issues related to improving project documentation or tutorials. label Oct 12, 2021
@github-actions github-actions Bot added the Type: Bug Issues related to software defects or unexpected behavior, which require resolution. label Oct 12, 2021
@wetneb
Copy link
Copy Markdown
Member

wetneb commented Oct 12, 2021

Hi @PabloCastellano, thanks a lot for this PR!
I agree with you, I think this is overdue. I had proposed this change a while ago (#3081) but we did not end up doing it because some (including me actually) had concerns about the fact that this endpoint is run by me on my own, instead of the Toolforge which offers some minimal user data protection (the fact that the service cannot see the IP addresses of the users).

In the meantime, it is true that the wdreconcile.toolforge.org has drifted away even more, since I do not fix any bugs there anymore.

Unless @tfmorris has concerns with this, I would merge this PR soon.

@PabloCastellano
Copy link
Copy Markdown
Contributor Author

PabloCastellano commented Oct 12, 2021

Hi @wetneb.

I have just read all the discussion about hosting the reconciliation service. Here are my two cents:

  • I'm fine with you seeing my IP and queries as the owner of the service. Maybe we can improve the situation by adding a disclaimer when you want to use this service in OpenRefine + a link to the Phab ticket.
  • I would merge this PR to keep reconciliation working, otherwise I would un-bundle the wikidata reconciliation service. In my case I wasted more time to figure out why reconciliation was working weirdly and replacing the endpoint than it would had taken adding the correct endpoint from the beginning
  • I think Wikimedia will eventually host this API. You have already raised awareness of a crucial service that looked invisible

Also, do you have a howto to replicate the service you are operating? What are the hardware requirements? Maybe more people are willing to operate more reconciliation services (or even running their own locally?).

@wetneb
Copy link
Copy Markdown
Member

wetneb commented Oct 12, 2021

Yes of course, the service's source code is available here: https://github.com/wetneb/openrefine-wikibase
This service is already being run by various organizations for various other Wikibase instances but I am not aware of an alternate public endpoint for Wikidata.

Copy link
Copy Markdown
Member

@wetneb wetneb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging soon, with the plan of releasing this in 3.5 unless concerns are brought up.

@tfmorris
Copy link
Copy Markdown
Member

I don't have a strong opinion on this.

@wetneb wetneb merged commit 3bca463 into OpenRefine:master Oct 20, 2021
wetneb pushed a commit that referenced this pull request Oct 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Type: Bug Issues related to software defects or unexpected behavior, which require resolution. Type: Documentation Issues related to improving project documentation or tutorials.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wikidata reconciliation endpoint is unreliable

3 participants