« Auto-match candidates with high confidence » is deceitful and should not be pre-check

[antoine2711]

In the Reconciliation dialog, the checkbox « Auto-match candidates with high confidence » is pre-checked when entering the dialog.

Since the confidence from the Recon service is not always useful (it can say 100% to many items even when some should be defraded, user don't really know what the number means and exactly how it is given). Because of that, I think the checkbox should be un-cheked by default. A preference could be create to control the default value of that control.

Since this has a direct impact on data pushed to Wikidata, and I highly raise the possibility of bad data being pushed, I consider this a bug.

To Reproduce

Steps to reproduce the behavior:

1. Open the reconciliation dialog

Current Results

The checkbox « Auto-match candidates with high confidence » is pre-checked.

Expected Behavior

The checkbox « Auto-match candidates with high confidence » is unchecked when opening the dialog.

Screenshots

Versions

• Operating System: N/A
• OpenRefine: v3.5.x and below.

[wetneb]

We could totally consider leaving this disabled by default. I expect quite a few users will be surprised by the move and will come back to us complaining that the service does not work correctly anymore, somehow ^^

[tfmorris]

Deceitful? It sounds like you're talking about a broken reconciliation service which is returning high scores for low confidence matches. The best thing to do is to file a bug report with the broken service(s). You can't expect OpenRefine to provide reasonable behavior if it's being lied to.

[antoine2711]

Deceitful? It sounds like you're talking about a broken reconciliation service (…)

@tfmorris: I'm probably not choosing the good word here. Maybe « misleading » would be better?

Yes, I think the Recon Service could be improve regarding its confidence rating. But this would have to be address in that project, not OR. It has also been discussed in length in other issues here in OpenRefine's repo. (See #3139)

Here, in OR, I'd just like to always do the confidence check by myself, so I always have to uncheck this particular checkbox. If I forget, then I might not realised that some entities were reconcialied without my direct involvement. That's what I want to prevent.

Regards, Antoine

[thadguidry]

One thing we could offer is allowing a user to manually adjust the threshold. For instance, if a confidence score is returned with 100 then a 20% threshold would lower it to 80 and scores of 40 would be lowered to 32. This could be a new Recon Score Threshold Facet. Wait, can a user not manipulate the score now and store them in a new column? They can just use cell right to copy the contents and then manipulate the json further in the cells?

[sherrif10]

Hello @antoine2711 @wetneb i would like to work on this issue . Kindly assigne it to me. Thanks.

[wetneb]

@sherrif10 I do not think this is a suitable issue for a newcomer as the difficulty does not reside in the implementation - this is instead a design decision that ought to be solved collectively. Once there is a clear consensus for a solution, it can be implemented, but so far it looks a bit early for that.

[sherrif10]

Thanks for clarification @wetneb

[antoine2711]

(…) this is instead a design decision that ought to be solved collectively. Once there is a clear consensus for a solution, it can be implemented, but so far it looks a bit early for that.

@wetneb: I think the real problem is in the Recon Service. On OpenRefine, the only thing we can do is

• not check the checkbox
• add a prefenrence to control the default value of the checkbox

More than that, it's another issue. It's improving the confidence calculation, very complex issue. This one is really only to complain about the checkbox that I always have to change and that I sometime forget.

Regards, Antoine

[thadguidry]

I think we have a solution to address the immediate problem that @antoine2711 and others have, myself included, to have a preference setting for Auto-match. We will keep the default of this setting to true as @tfmorris and @wetneb suggested. Users that want to change the default of Auto-match to false can do so by setting the preference value. Our existing preferences documented here: https://docs.openrefine.org/manual/running#preferences

So, let's make this a preference implementation with key reconciliation.automatch where the users' preference Boolean value of true or false can be read and implemented here:
https://github.com/OpenRefine/OpenRefine/blob/master/main/webapp/modules/core/scripts/reconciliation/standard-service-panel.js#L335
optionally could also be added in the PreferenceStore tests but not necessary.

[ayushrai206]

hey everyone ,i would like to work on this please assign this to me.

[antoine2711]

@ayushrai206, this issue is yours to work.

For the preference name, I think ui.reconciliation.automatch could be good. Here's the list of the current preferences:
https://openrefine.org/docs/manual/running#preferences

Regards,
Antoine

[ayushrai206]

I think we have a solution to address the immediate problem that @antoine2711 and others have, myself included, to have a preference setting for Auto-match. We will keep the default of this setting to true as @tfmorris and @wetneb suggested. Users that want to change the default of Auto-match to false can do so by setting the preference value. Our existing preferences documented here: https://docs.openrefine.org/manual/running#preferences

So, let's make this a preference implementation with key reconciliation.automatch where the users' preference Boolean value of true or false can be read and implemented here: https://github.com/OpenRefine/OpenRefine/blob/master/main/webapp/modules/core/scripts/reconciliation/standard-service-panel.js#L335 optionally could also be added in the PreferenceStore tests but not necessary.

hey, can you please specify why does that particular line no. needs the change, as what i am planning here is simply implementing a function, like the function we used in setting the user preference function while previewing matched cells.

[antoine2711]

hey, can you please specify why does that particular line no. needs the change, as what i am planning here is simply implementing a function, like the function we used in setting the user preference function while previewing matched cells

I think the correct line to change would be:

OpenRefine/main/webapp/modules/core/scripts/reconciliation/standard-service-panel.js

Line 340 in 5273a3d

autoMatch: this._elmts.automatchCheck[0].checked,

Here an example of reading a preference:

OpenRefine/main/webapp/modules/core/scripts/project.js

Line 52 in 5273a3d

var leftPanelWidth = JSON.parse(Refine.getPreference("ui.browsing.facetsHistoryPanelWidth", 300));

I think the PR for this Issue can be simple.
There are also the changes to the documentation.

Regards,
Antoine

[antoine2711]

@ayushrai206 : I think my previous answer was just partially correct. The line I'm pointing you is not the good place, from what I understand to set the defautl value; this is just before sending the request to the API.

We need to change the default value of the checkbox at interface creation, which is elsewhere.

Probably around here:

OpenRefine/main/webapp/modules/core/scripts/reconciliation/standard-service-panel.js

Line 96 in 5273a3d

this._elmts.or_proc_autoMatch.html($.i18n('core-recon/auto-match'));

Regards,
Antoine

[ayushrai206]

@Abbe98 @antoine2711 i think i did it, also sorry for the mess, won't happen again.