Searching titles and artists with accents and diacritics doesn't work

[mixxxbot]

Reported by: pcantare
Date: 2022-07-07T11:09:04Z
Status: Fix Committed
Importance: Medium
Launchpad Issue: lp1980944
Tags: library, macos

---

Mixxx version: 2.3.3
OS version: MacOS 12.4

I have a lot of files with accents and diacritics (Spanish songs) and this causes some problems when searching them (I don't have the same characters in my keyboard). What's really strange is that I can't even find a track copying and pasting the exact title it has in the mp3 tags.

A possible solution: each searchable field (title, artist, genre...) could be duplicated in the database with an ASCII only version, for example:

title: Buscándote
title_ascii: buscandote

The original field is used for display while the ASCII one is used for search.

To convert a string to its ASCII can be done normalizing the unicode characters to NFD (https://en.wikipedia.org/wiki/Unicode_equivalence#Normal_forms) and then strip away all marks and diacritics. The same should be done to the string entered in the search field.

Example in Java: https://stackoverflow.com/questions/2397804/java-string-searching-ignoring-accents/2397830#⁠2397830

[mixxxbot]

Commented by: ronso0
Date: 2022-07-10T20:20:10Z

---

I can't reproduce on Linux with Mixxx 2.3.4

I tried both with the file name and title tag and could find it both with "Buscándote" and "Buscandote".

So this is a macOS issue?

[mixxxbot]

Commented by: pcantare
Date: 2022-07-11T12:27:12Z

---

I have Mixxx 2.3.3. Not sure if in 2.3.4 it might have been fixed.

As for MacOS, I'm pretty sure the file system uses NFD normalization format rather than NFC, so searching the file name might get wrong result: https://medium.com/@sthadewald/the-utf-8-hell-of-mac-osx-feef5ea42407

I find it strange anyway that searching the title in the mp3 tags gets different results in different OS's. The tags in my mp3s are normalized as NFC.

[mixxxbot]

Commented by: ronso0
Date: 2022-07-12T09:33:34Z

---

related: file path issue with m3u on macOS
https://mixxx.discourse.group/t/import-m3u-without-importing-files/25210/11

[mixxxbot]

Commented by: daschuer
Date: 2022-07-14T11:32:17Z

---

We use a likeCompareLatinLow() function that replaces all special characters with their Latin replacements before compare. It sounds like this is not working on your device.

mixxx/src/util/db/dbconnection.cpp

Line 386 in a5906c1

int DbConnection::likeCompareLatinLow(

Did you use the official Mixxx build?

[mixxxbot]

Commented by: pcantare
Date: 2022-07-14T11:54:34Z

---

I use the official Mixxx build (2.3.3) and I also downloaded the source files and built it. This morning I was actually prying into that file.

[mixxxbot]

Commented by: pcantare
Date: 2022-07-14T12:14:56Z

---

Actually it looks like the function likeCompareLatinLow() is not called when the search is performed. I added a few couts in the funcion and nothing is printed. Anyway, I could see that makeLatinLow() is called instead, but I don't know from where.

[mixxxbot]

Commented by: daschuer
Date: 2022-07-15T04:37:23Z

---

I have confirmed here:
https://github.com/daschuer/mixxx/actions/runs/2670031391
That the code is skiped.
I can remember that we had issues to verify that Qt uses the same SQLite library than Mixxx would use. Because of crashers in case this is not the case we have disabled the code.

I assume everything is working with 2.4 alpha. So I suggest to use that, after some brief tests.
Backup your library, before installing 2.4 just in case ..

[mixxxbot]

Commented by: pcantare
Date: 2022-07-15T06:39:35Z

---

Thank you very much. I have the source code of 2.3.3. Is there a way I can enable this behaviour in this version as well?

[mixxxbot]

Commented by: daschuer
Date: 2022-07-15T09:12:59Z

---

You can enable LOCALAWARECOMPARE like described here:

mixxx/CMakeLists.txt

Line 2500 in 6dea2f3

# FIXME: It is difficult to get qmake to link Qt to a custom built SQLite on

This works only if Qt links to the same SQLite binary as Mixxx.

[mixxxbot]

Commented by: pcantare
Date: 2022-07-15T10:40:16Z

[mixxxbot]

Commented by: pcantare
Date: 2022-07-15T11:15:23Z

---

Thank you. It works!

[mixxxbot]

Issue closed with status Fix Committed.