Support Wikidata

[lorinczz]

Could you add support for Wikidata?
Related links:

• http://www.mediawiki.org/wiki/Extension:Wikibase_Repository/API
• http://www.mediawiki.org/wiki/Manual:Pywikibot/Wikidata Python implementation
• http://www.wikidata.org/wiki/Wikidata:Creating_a_bot other examples

[eldur]

If you suggest an java api example ...

How do you want to get/set what data via jwbf?

For e.g. how to get the backlinks of "Mainpage"

for (String backlinkArticleName : new BacklinkTitles(bot, "Mainpage")) {
  println(backlinkArticleName);
}

[lorinczz]

The main difference is that you get JSON objects like wbgetclaims which should be mapped to Java objects based on the Wikidata data model. New data should be posted back in the same way. There are already implementations in Python and .Net. I can assist you with further information, if you need.
There is also a note “We are planning to migrate to JSON-only API, removing all other formats like XML, YAML, and PHP.” on API:Client code.

[eldur]

Interesting ... a json only API, should be no problem. Are you agreeing with the following acceptance criteria for this issue:

• each request to api.php should use format=json instead of format=xml

... I'm not sure .. is this datamodel already part of the official mediawiki installation (I ask, because, I need a testinstallation to verify my changes)

[lorinczz]

Wikidata uses JSON format in all examples. I don't know if it works other way. Is official, I already use it on huwiki via Scribunto modules. I don't know about the installation.

[eldur]

Hmm, but how do we test that our change will work?

Do you have a simple testcase - what (actually) not existent function do
you want to use?

[lorinczz]

There is a sandbox where we can test: Sandbox. The test case would be: retrieve the entity and add a claim, for example. Later we need to add qualifiers and references. Might be possible to do it all in one with wbeditentity, I don't know yet.

[lorinczz]

I would try it myself, if you write a generic method to call the Wikidata APIs, with value pairs for example.

[eldur]

I started a new branch for these topic ... what do you think?

[eldur]

.. sorry my trackpad, clicked where it don't have to ;-)

[lorinczz]

It's great to have a branch for this. How can I help? Does it help you if I look after the Python implementation? I would start from there because it's a working system. Or do you want to build it based only on basic information?

[eldur]

I've to admit, that I've no concrete idea/use case how to to use the wikidata requests in a real world app.

But the wikidata api extensions, seems to be an interesting case for json request handling and api mock tests (where I would offer some lines). I assume you have a concrete use case - so I would suggest, write what fits your needs, and we talk about it - if you interested in.

What do you think about this division of work?

[lorinczz]

The task is not small as I see it page.py, site.py. The goal would be to get an entity with wbgetentities, add or modify claims with qualifiers and references, and send it back with wbeditentity if it's possible, in order to have one edit per entity.

[lorinczz]

I was able to retrieve data with the help of your new module. What is your plan? How can I help?

[eldur]

Could you write the datamapping(json2java, java2json) or select a mapper that fits your needs? Maybe as new repository to try different approaches (and you don't have to wait on my response :-) )?

[eldur]

I see you found the wikidata-toolkit with the goal to write the datamapping and anything else ;-)

I would suggest to close this issue, okay?

[lorinczz]

Thank you for your help.

[lorinczz]

I'm stucked with jwbf. Can I ask a little help?

2014-02-25 21:41 GMT+01:00 Loki notifications@github.com:

I see you found the wikidata-toolkithttps://github.com/Wikidata/Wikidata-Toolkitwith the goal to write the datamapping and anything else ;-)

I would suggest to close this issue, okay?

Reply to this email directly or view it on GitHubhttps://github.com//issues/9#issuecomment-36055441
.

[eldur]

sure, open a new question

[lorinczz]

You better implement wbgetentities. See here:
https://www.wikidata.org/wiki/Special:ApiSandbox#action=wbgetentities&format=json&sites=dewiki&titles=Hamburg
It's more useful and is flexible, you can set to get only the claims, the response could fit to the converter.

[fer-rum]

We aim to become this flexible too :)
But honestly there is still a lot of WIP.

[lorinczz]

I was referring to the source of the json, a different API module, not to the library used.

[fer-rum]

This indeed is external Json. I will notify you once I have something useful.

[fer-rum]

We released the Wikidata Toolkit 0.3.0

It now fully supports the external Json (which has become the mainly used Json in the meantime) and parsing of the new Json dumps provided by Wikidata.

For detailed Information see the release notes

Technical side notes

One needs to inject the site IRI information into the objects after parsing them (since the Json does not have this), like so:

JacksonItemDocument document = mapper.readValue(mwRevision.getText(), JacksonItemDocument.class);
document.setSiteIri(Datamodel.SITE_WIKIDATA);

When deserializing individual Statements, Snaks, or ItemIdValues, a reference to the parent JacksonItemDocument needs to be injected (they will get their site IRI from there; this is convenient in normal parsing since we know the parent document when creating snaks, but not the site IRI yet). If you really need standalone Statements etc. from Jackson, then some minimal dummy document is needed for providing the site IRI (see JacksonObjectFactory on how to do this).

The normal way of creating Jackson objects from scratch is to use the JacksonObjectFactory. This can be used with DataModelConverter to make Jackson copies of other datamodel implementations.