Issue with special characters when upgrading Examine

[jemayn]

Hey Shannon,

I've found an issue with special characters when upgrading Examine on a v8 site. The site initially ran Examine 1.0.2 and Umbraco 8.10.2. We upgraded Examine to version 1.2.2 to try to get around another bug that others had mentioned to be fixed in the latest Examine version.

To get Search working with the Danish letters æ, ø & å we have for a long time had to replace them in search strings with non Danish characters, for that we have a method in Skybrud.Search: https://github.com/skybrud/Skybrud.Umbraco.Search/blob/v3/latest/src/Skybrud.Umbraco.Search/SearchHelper.cs#L347

However, after upgrading from 1.0.2 -> 1.2.2 the search stopped working if it included Danish characters, where it worked before.

The lucene query would be something like

nodeName_lci: stoj  

Where without the ReplaceDiacritics method it would instead be

nodeName_lci: støj  

I can't find anything in the milestone notes that would lead me to believe this has changed somehow, but I tried installing different versions of Examine, rebuilding the index and performing the search to figure out when it started happening, and it
Worked on 1.0.2
Worked on 1.0.6
Worked on 1.1.0
Worked on 1.2.0
Didn't work on 1.2.1 (& 1.2.2)

So it seem to have been introduced between 1.2.0 and 1.2.1.

If it just works without our own method to replace special characters from now on then that is fine, just surprised to find this without any note anywhere.

[Shazwazza]

I believe this is the same issue as #244

There is an in depth explanation there and then a further discussion on the Umbraco issue tracker here umbraco/Umbraco-CMS#11176

Let me know if that is your issue - which is essentially to do with wildcard queries because wildcard queries don't get processed via analyzers. The recap of the issue is here umbraco/Umbraco-CMS#11176 (comment)

There's work arounds - like using a different analyzer for your index or for specific fields.

[jemayn]

Sorry I never got back on this one. We add a dictionary on top that does indeed add some wildcard queries to the search, so sounds like that same.

Our work around for now has been to just not add our helper method https://github.com/skybrud/Skybrud.Umbraco.Search/blob/v3/latest/src/Skybrud.Umbraco.Search/SearchHelper.cs#L347 and it seems fine, so maybe this "issue" will just simplify things.

We can close this issue for my sake, it is somewhat resolved 🙂