Problem with non-latin characters in synonyms.txt

[pblasone]

Thank you Nolan for making this really cool plugin for better synonym handling. I installed it on my Solr 4.0 and had no problems getting it running.

However I seem to have stumbled across a problem with non-latin characters. I'm not completely sure that my problem is due to non-latin characters in the synonyms file, but I believe that my tests as described indicates it:

The contents of my synomym.txt file (file saved in UTF-8):

  brystforstørrelse, brystoperation, silikonebryster, bfo, brystforstørrende operation
  mop, abdominalplastik, maveplastik, maveopstramning

Notice that the first line of the synonym file contains some terms with the unicode character ø. The second line contains no special characters.

Upon testing the real life scenario, where sombody makes a search for bfo (a common abbreviation), I get this parsed result (notice how some of the tokens are missing a few characters near the end):

+((title:bfo) ((((title:brystforstør) (title:operation))~2) (title:brystoperation) (title:silikonebryst) (title:brystforstør))) () ()

To test my suspicion, that this has something to do with UTF-8 and character positions/offsets, I try to search for mop, invoking the second line in the synonym file.

This search returned this parsed result, with all words having the correct length, which further strengthens my suspicion:

+((title:mop) ((title:abdominalplastik) (title:maveopstramning) (title:maveplastik))) () ()

Could there be something to this, or am I doing something else wrong?

Thanks!