Handle Case Insensitive Identifiers

[derenrich]

Twitter usernames (for example) are case insensitive. If I tell the plugin to link a twitter handle to an item and the same handle is already present (but in a different case) it should not add the redundant case. You can detect whether a property is sensitive to case by checking for "has quality" "case insensitivity".

[fuddl]

Twitter username → has quality → case insensitivity so
twitter.com/DerenRich should resolve to Q108267155 (actual value derenrich).

Alright, I'll see what I can do.

[fuddl]

@derenrich do you have any clue about how to make a case insensitive sparql query?

`
  SELECT ?item
  WHERE {
    ?item wdt:${ prop } "${ id }".
  }
`

should I just assume the string in the database is lower case?

`
SELECT ?item
WHERE {
  ?item wdt:${ prop } "${ ci ? id.toLowerCase() : id }".
}
`

[derenrich]

i think you have to rewrite the query with a filter and downcasing both the search and target strings (or a regex). probably less efficient but it's the only way to be sure.

[fuddl]

I'm starting to wonder why we are doing this. Does any part of the twitter api downcase the usernames? Is there a url scheme that only serves the usernames lowercased? If not, why don't we store usernames how they occur? What converts them to lowercase, and why?

[camelCaseNick]

We are not doing that at all. Nothing is converted to lower case. On our side, P2002 is supposed to store the preferred variant and not the lower case one.

For me it would be "camelCaseNick" not "camelcasenick" nd if you clicked from within Twitter on my profile you'd get https://twitter.com/camelCaseNick. However if you manually navigated to https://twitter.com/camelcasenick Twitter would serve the same profile without a redirect. That is the problem here.

A functioning (though less efficient) query could be:

select ?item where {
  ?item wdt:${ prop } ?id.
  filter(lcase(?id) = "${ id.toLowerCase() }")
}

[fuddl]

However if you manually navigated to https://twitter.com/camelcasenick Twitter would serve the same profile without a redirect. That is the problem here.

I understand that. But how likely is it? Why should anybody do that?

I found https://twitter.com/camelCaseNick using twitters own search. However the page was linked with:

<link href="https://twitter.com/camelcasenick" rel="canonical">

So twitter seems to be uncertain, which username they want to show us. I would prefer to store the one that is not lowercased because it contains more information. 🤷

[derenrich]

This is a general problem not limited to Twitter

[fuddl]

@derenrich Version 0.189 should now resolve case insensitive identifiers case insensitively:

If you notice any problems, feel free to reopen this issue

[derenrich]

thanks!