Feature request: add a bit of extra metadata in the Wikidata reconciliation 'hovercards' to make matching easier

[trnstlntk]

Feature request, possibly relevant for @wetneb ?

Would it be possible to extend the 'hovercards' that appear when clicked on a Wikidata reconciliation suggestion? (I use OpenRefine 2.8 but this feature has been present in any OR version in which I have used Wikidata reconciliation).

• The extra 'explanation' that is now shown, is the Wikidata label. However, this is sometimes absent and most often quite minimal, not offering enough precise data for good disambiguation, so a clickthrough to the item itself is often needed (which takes a lot of time, especially when reconciling large datasets).

• I would suggest to look at Magnus Manske's Autodesc (Automated description API) for a replacement: - it gives much more detailed, auto-generated descriptions based on the Wikidata statements that are available on the item, also when a description on Wikidata proper is lacking.

• If this is not a viable option, just including birth date and death date (P569 and P570) for humans (Q5) would also help tremendously.

Thanks for the good work, from an enthusiastic user and evangelist of OpenRefine :-)

[thadguidry]

@fokky I think a much better way to handle this is not actually provide Hover Cards because they acutally limit us quite a bit, but instead to have a complete Recon Panel that can be docked on the right/left side of OpenRefine so that you can have a fully scroll-able panel for showing complete information from Wikidata or another recon service. That's basically our longterm plan for this UI feature.

[wetneb]

hi @fokky - thanks for this report, I couldn't agree more!!!

The reconciliation service does use Magnus' autodesc, but currently only on items that do not have a description yet. And I think the feature might even be broken (maybe by a change in autodesc's API? I have to investigate).

Overall I think these overviews could be improved a lot. The question is: beyond descriptions, how should we pick the "interesting" properties that users will find useful to disambiguate items? It's possible to pick birth dates manually for humans - but what about other domains? I would ideally like to have a generic approach that works everywhere.

[ostephens]

@wetneb @fokky thinking out loud, but would it make sense to use the values of the P1963 property (properties for this type) from the type chosen to reconcile against?

I guess we'd need to define this somewhere in the existing API as something like: 'disambiguation properties' or similar to allow this to work across multiple sources....

[thadguidry]

@ostephens P1963 is a good 1st pass for disambiguation properties in Wikidata. Unfortunately, often there are not enough P1963 statements on Types in Wikidata, in fact, I spend too much of my minutes of spare time often filling them in... like this one just now https://www.wikidata.org/wiki/Q12518 But yeah, calling properties that help describe uniqueness for entities... "disambiguating properties" is fine.

[wetneb]

yeah I already use P1963 to suggest properties to fetch in the data extension dialog so that would make sense… but sometimes there are a lot of them! so the preview dialog should probably be extended to fit a bunch of them.

[wetneb]

@fokky I have fixed a bug in the call to autodescribe - you should now see more of them in OpenRefine (no need to update anything on your side).

@thadguidry I still don't see exactly what you mean by your "complete Recon Panel" - could you describe that a bit more? Maybe with a drawing?

[thadguidry]

@wetneb Oh, its just a panel that we can expose for an IFRAME for the Recon Entity endpoint that the Recon service provides. It would save the user from having to buy another computer monitor, or installing a Smart Split Tab extension in their browser, just to see the details of a particular Recon Entity endpoint ... allowing them to stay focused on one browser tab in OpenRefine as they work down their column of entities.

[wetneb]

@thadguidry I kind of get the broad idea but it's still fuzzy, sorry - I guess we can discuss that directly at the next meeting.

[trnstlntk]

Thanks, everyone, for your prompt and enthusiastic responses.

Although I'm kind of a Wikidata veteran editor, I did not know about P1963 - I will surely have a look at that more often.

Re: including the most basic metadata, I think nothing at the moment can beat Autodesc, so I think supporting this is a good move for the time being.

I also applaud all other efforts that make disambiguation easier (i.e. needing less clicks and eliminating browser tab switches).

[ettorerizza]

It would probably be a little slower to load, but what about an iframe that would display the mobile version of the Wikidata page in question? A bit like this, but with two scroll bars to move the mini-web page.

[thadguidry]

@ettorerizza yes, that's the idea Ettore, however, not a popup dialog as we have now, but actually 2 panels, one where a constant View Recon Entity panel shown on the right of the browser window, that can be toggled to display or not, and who's content simply refreshes when an entity is hovered mouse on in the recon column and perhaps that can be data limited to the user's chosen set of disambiguating properties. This is basically what the Freebase Recon UI had that was hosted on Freebase.com long ago for Judgements (somewhat similar to Primary Statements), but we never got around to moving that panel functionality into OpenRefine itself.

The View Recon Entity panel itself really should be dockable, sizeable, and showing a user's chosen set of disambiguating properties by a user to customize their recon service workflow. Some Recon services show a lot of info, some not so much. We want to give choice to the user themselves, to show more or less data for each entity, and also our new interface should be much more streamlined to allow quicker judgements by perhaps just hover mouse over entities where the View Recon Entity panel just auto refreshes to show details of the entity and then on the Match Recon Entity panel, you click on much smaller buttons in the column when you want to match. Less clicking, more Auto Show magic, and customizable views for the View Recon Entity panel. The Match Recon Entity panel can thus be very small, with only buttons for Match, Match All, Cancel.

Pretend, we're Microsoft or Ubuntu, designing a brand new style and flow of Recon matching.

[wetneb]

@ettorerizza really nice idea! and it should be fairly simple to implement too. I'll give it a try.

@thadguidry okay, that's much clearer, thanks!

[ettorerizza]

Tanks @wetneb , but I just have a good memory (I think it was the system used in LODrefine). ;)

That said, it would not reduce much the number of clicks in my example with Amsterdam. I know that the first one has the smallest Qid and it's probably the capital of the Netherlands, but what if I'm looking for Jacques Brel's song? Thad's system also involves hovering each candidates and I guess it will take a second or two to load the information into the dock panel.

Ideally, each candidate label should contain a descriptive element to spot the right one at a glance, like Wikipedia's qualifiers in brackets. eg, Amsterdam (City in The Netherlands), Amsterdam (Jacques Brel 'song)... To avoid pointless API calls, we could reserve the operation to candidates tied who have the maximum score and the closest spelling to the original value. If this increases the reconciliation time too much, we can make the operation optional. At the start of the reconciliation, the user could choose between "simple reconciliation" or "reconciliation with qualifiers".

But I think aloud, I completely ignore if it's implementable.

[wetneb]

I quite like the idea of showing more metadata without any click or hover - I guess just including descriptions (Wikidata's native ones or Magnus' auto-generated ones) along each match candidate would already help a lot.

[thadguidry]

@wetneb Again, I say let the users make the choice of what data and how much or little to see rather than hardcoding a Recon workflow that might not work for them. Let's just give them the power tools to work better with Recon.

@ettorerizza The hover is a different mode than your thinking. My idea is that it also has background cell highlighting for easier visibility with 3 buttons (Checkmark, 2 Checkmarks, X). The set of entities shown on the current grid would be background loaded in batches, not loaded upon each hover, probably using using something like HTML5 and React's iframe ref with callback

[wetneb]

@thadguidry yeah, but the simpler the better, both for the servers, the users, the reconciliation service providers, and the OpenRefine dev team!

I think we have identified a few cheap changes that will make a big difference - once this is in place we can think more about the more involved stuff.

[thadguidry]

@wetneb yes, absolutely baby steps. I just wanted to ensure that the team understands my opinion here, since you asked for clarity earlier. Agreed, those quick changes can help considerably for now.

[wetneb]

@fokky @ettorerizza Here is what you get when using m.wikidata.org to do the previews:

(you can try it yourself with this temporary reconciliation endpoint: http://ulminfo.fr:3893/en/api)

My gut feeling is that it is not condensed enough so I have made a version where the view is scaled down, but unfortunately it renders as blurry (at least for me):

(you can try this second version at http://ulminfo.fr:3894/en/api).

Let me know what you think…

[ettorerizza]

Really useful, thanks @wetneb ! The windows display need to be improved a bit, but the speed seems very good !

but unfortunately it renders as blurry (at least for me)

Yes, it's weird. The text appears very clear, then becomes a little blurry.

@fokky If you want to give a try, note that you have to wait a big minute with a spinning wheel before the new reconciliation service is available.

[wetneb]

@ettorerizza When you say that "the window display needs to be improved a bit", do you mean that the position of the preview window is not optimal? I'm not sure what we can do about this… I guess it will be solved once we have a dedicated reconciliation UI that does not rely on pop-ups.

Do you prefer the blurry zoomed out version or the un-zoomed clear one? Do you think it makes sense to replace the current preview with any of them?

Note that we can also change the dimensions easily (for now it's 500x400px).

[ettorerizza]

@wetneb Yes, the preview window sometimes pop up in the upper direction and no longer fit on the screen.

I've not tried the first version, of course, but from what I see, it could perfectly suit too. After all, we just need some clues to disambiguate, not a complete description of the item.

[ettorerizza]

I'm not sure what we can do about this… I guess it will be solved once we have a dedicated reconciliation UI that does not rely on pop-ups.

A quick fix might be to tweak data-table-view.less, but it's probably not a very good practice...

.data-table-cell-editor {
  border: 1px solid @chrome_primary;
  background: @chrome_secondary;
  padding: @padding_tight;
  .rounded_corners();
  }

/*separate this class from .data-table-cell-editor */
.data-table-topic-popup {
  border: 1px solid @chrome_primary;
  background: @chrome_secondary;
  padding: @padding_tight;
  .rounded_corners();
  top: 10%!important; /* element added */
  }

[wetneb]

I plan to work on this at the Wikimedia Hackathon 2018:
https://phabricator.wikimedia.org/T193875

[wetneb]

Migrating this issue to where it belongs: wetneb/openrefine-wikibase#61