"Add columns from reconciled values" sometimes returning lagged or cached values for some rows

[Jheald]

I was working through some reconciliations suggested by OpenRefine, manually adding a P10873 (Mapping Museums ID) on Wikidata if the reconciliation was correct, or creating a new item there & updating the reconciliation on my OR sheet if it wasn't.

To confirm that I had indeed updated everything correctly, I then used "add columns from reconciled values" to generate an updated column of the MM ids from wikidata on my OR sheet.

In most cases this was successful, and the MM id that I had added was successfully returned. However for six rows no MM id was returned, even though it was now present on Wikidata.

Additionally, when I tried to return P31 values for the six, one edit I had made 24 hours previously was not reflected, also another I had made 9 hours earlier.

I posted an account of this to the Open Refine+Wikidata telegram channel, and somebody there tried to reproduce the issue, but got completely up-to-date values.

I tried again this morning, and found two rows now returned up-to-date values (including one of the P31 changes that hadn't previously been reflected); but not the other 4 rows, which still returned no MM id, nor the other P31 change.

Re-starting OpenRefine and re-reconciling the six rows as a new project did not change this.

To Reproduce

Steps to reproduce the behavior:

1. Load data-file below as a new project
2. Reconcile on Q-ids vs Wikidata
3. Retrieve P10873 and P31 from Wikidata using "add columns from reconciled values"

Current Results

As described above, the first two rows now work for me; however the remaining 4 rows do not retrieve the MM id, and "Wallace monument" does not retrieve the P31 = "monument"

Expected Behavior

All 6 rows should retrieve MM ids, and the P31 values shown in the screenshot immediately below

Screenshots

Success from New Zealand:

Continued failure in Scotland:

Versions

• Operating System: Windows 10
• Browser Version: Chrome version 104.0.5112.81
• JRE or JDK Version: openjdk version "17.0.4" 2022-07-19 ; OpenJDK Runtime Environment Temurin-17.0.4+8 (build 17.0.4+8) ; OpenJDK 64-Bit Server VM Temurin-17.0.4+8 (build 17.0.4+8, mixed mode, [sharing)]
• OpenRefine: OpenRefine 3.6.0

Datasets

Additional context

[Jheald]

Dataset:

c6649e3c.txt

[Jheald]

(Note: slight difference between the screenshots above, because the first one had used "use values as identifiers" in the Qid reconciliations, whereas I had just done "reconcile" -- but outcome is the same for me either way: just the first two MM ids then retrieved, not the remaining four).

[Jheald]

Update: just tried again (a further 7 hours later), and now it is picking up the values expected.
Looks like a cache may finally have cleared -- but 24 hours is rather a long time for it to have hung around. (On the telegram channel Antonin says the cache is supposed to only live for an hour).

[thadguidry]

The cache usually refreshes after an hour (but he and others have seen cache issues like this before). Ideally, we'd get better support all around from Wikimedia foundation for the reconciliation endpoint. But that's for another day.

Glad you got it working! We'll close this issue.

[wetneb]

This could also be worked on in the current reconciliation service:
wetneb/openrefine-wikibase#146