Add Lexeme editing

[wetneb]

We now have support for Lexeme entities in the datamodel.
We could also support editing these in wdtk-wikibaseapi.

[Tpt]

https://phabricator.wikimedia.org/T202725 and https://phabricator.wikimedia.org/T199896 will make the implementation a bit cumbersome. Fixing these limitations properly in WikibaseLexeme is probably going to be as hard as writing a circumvention in WikidataToolkit

[wetneb]

Interesting!

For the first one (T202725) this is related to #376: we should not assume that wbeditentity returns the full entity as this is only the case for Items apparently (not even properties).

For the second one I guess this means we need to implement other API actions, which are not going to be atomic indeed. I think we need to rethink the architecture of this module - I wanted to do that for a long time (#403) but haven't got round to it yet. This is related to my second bullet point in that ticket.

[Tpt]

Indeed, having support for other API actions would be great. But I believe these limitation should also be removed on the WikibaseLexeme side.

[thadguidry]

Also related: T249206 - Serialized statements of Forms and Senses are missing data type fields

datatype field for each Snak was missing previously for statements within Senses and Forms of a Lexeme.
Starting on 26 August 2020 the datatype fields will be present on them. [Reference: Wikidata Project Chat]

[62mkv]

hi team! sorry for a noob question, but.. how does one create a lexeme (with forms) using WDTK? I see that "datamodel" artifact has quite extensive support for Lexemes, but could not find anything in the WikibaseDataEditor. Is it because of this issue? if so, what would be the suggested workaround? Thanks in advance.

[wetneb]

Hi @62mkv, that's correct: editing and creating lexemes is not supported yet in WDTK.

[62mkv]

thanks, @wetneb! I am looking into it now, seeing what is possible as a quick hack to be able to just a) create lexemes with forms or b) add forms to existing lexeme. Currently the stumbling point for me is an apparent inability to create a FormDocument as a new Form (with "null" id) for a given Lexeme.

@Tpt I see you've added those types, what would be your suggestion on how to resolve it?

PS: there seem to be tools that are capable of what I need, in particular LexData (although they seem to be using different action, "wbladdform") and https://github.com/lucaswerkmeister/tool-lexeme-forms/, but I'd really not want to abandon Java for this... TIA!

[Tpt]

Hi @62mkv!

To create a new Form with the WDTK datamodel, you can use the LexemeDocument.createForm method.
This will properly generate a new form identifier and add the form to the lexeme object.

Then, we need to implement lexemes/forms and senses saving. The wbeditentity API action we use for forms and senses is a bit limited (c.f. the upper discussions). If you are familiar with PHP, the easiest way to go is probably to just fix the MediaWiki WikibaseLexeme extension. If not, maybe some hacks with the existing API actions wbaddform... might do the job.

[62mkv]

thanks @Tpt ! I guess that would cover my use-case №1 (create and add forms) but how do I get LexemeDocument, if I have an L-id already?

I see that I can use WbGetEntitiesAction to get an EntityDocument but how to obtain a proper LexemeDocument out of that?

[62mkv]

This will properly generate a new form identifier and add the form to the lexeme object.

by the way, javadoc on that method says

    /**
     * Creates a new {@link FormDocument} for this lexeme.
     * The form is not added to the {@link LexemeDocument} object,
     * it should be done with {@link LexemeDocument#withForm}.
     */

[Tpt]

I see that I can use WbGetEntitiesAction to get an EntityDocument but how to obtain a proper LexemeDocument out of that?

You could just cast using the usual (LexemeDocument).

by the way, javadoc on that method says

Indeed, my bad.

[62mkv]

Cool! and by the way, if I try to call LexemeDocument.createForm on a not-yet added lexeme, it throws an exception

java.lang.IllegalArgumentException: The string L0-F1 is not a valid form id

	at org.wikidata.wdtk.datamodel.implementation.FormIdValueImpl.<init>(FormIdValueImpl.java:65)

so, it seems like there's no easy way to create lexeme AND add forms in a single wbeditaction hop... I'll try with WbGet now

[62mkv]

so, with this code:

        LexemeDocument existingLexeme = (LexemeDocument) wikibaseDataFetcher.getEntityDocument("L1358");
        FormDocument formDocument = existingLexeme.createForm(
                Collections.singletonList(Datamodel.makeMonolingualTextValue("aprils", LANGUAGE_CODE)),
                Collections.singletonList(getItemIdForTestWikidata("Q42"))
        );

        LexemeDocument withForm = existingLexeme.withForm(formDocument);

        LexemeDocument result = wikibaseDataEditor
                .createLexemeDocument(withForm, "Adding form to existing lexeme", null);

i am getting this request string:

summary=Adding form to existing lexeme&new=lexeme&maxlag=5&data={"type":"lexeme","id":"L1358","lexicalCategory":"Q212131","language":"Q208912","lemmas":{"en":{"language":"en","value":"april"}},"claims":{},"forms":[{"id":"L1358-F1","representations":{"en":{"language":"en","value":"aprils"}},"grammaticalFeatures":["Q42"],"claims":{},"lastrevid":533196,"type":"form"}],"senses":[],"lastrevid":533196}&bot=&assert=user&format=json&action=wbeditentity&token

and this MediaWikiException:

org.wikidata.wdtk.wikibaseapi.apierrors.MediaWikiApiErrorException: [param-invalid] Invalid field used in call: "id", must match id parameter

is it problem with my code, the WDTK unreadiness, or Wikidata API problem? I can't tell :( to me, request content looks legit. it correctly shows lemma, lexeme id, form with features..

UPD: aha, so, looking at the documentation for wbeditaction, (https://www.wikidata.org/w/api.php?action=help&modules=wbeditentity) it seems as though id parameter is missing. will look as to why that might happen