Assertion failed error after Macos Update

[v1kn]

Macbook Air M2
MacOS Sequoia update to 15.4
locale set to en_US.UTF-8

Cannot start cmus.
error msg:

Assertion failed: (p->val == key), function lookup_substsearch, file collate.c, line 596.

cmus is installed via brew. reinstalling cmus with brew did not help.

[ttr3dp]

What worked for me is setting LC_COLLATE="C".
You can test it out with LC_COLLATE="C" cmus first.

[v1kn]

@ttr3dp You mean starting cmus with the LC_COLLATE="C" as a prefix?
No luck, same result.

[drewdiver]

Same issue this morning on same hardware (MacBook Air M2) and having just updated to macOS Sequoia 15.4. It worked yesterday on macOS 15.3.2.

Tried setting LC_COLLATE="C" with no luck.

[ttr3dp]

@v1kn Hm... 🤔
Here's more info, if it is of any use:

• I'm using master build, installed with brew install cmus --HEAD.
• Don't have any other LC_* vars exported in my env, just LANG=en_US.UTF-8 (not sure if this makes a difference)

[drewdiver]

Removed cmus and re-installed via brew install cmus --HEAD and attempted LC_COLLATE="C" cmus to no avail.. nothing other than the macOS upgrade has changed in my setup which was working.

ProductName:		macOS
ProductVersion:		15.4
BuildVersion:		24E248

[ttr3dp]

@drewdiver I also tried it with LC_ALL="C" cmus and it worked. Here's what locale outputs by default on my machine:

$ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=

[nnnell]

@ttr3dp Thanks for the tip — I've been experiencing the same issue after the 15.4 update, and LC_ALL="C" cmus did the trick.

It's not handling non-ASCII characters as shown here, but everything else seems to be working as normal.

My default locale environment variables:

$ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL="en_US.UTF-8"

[v1kn]

Unfortunately, half of my musical library is in Asian languages.
Not showing CJK characters would be a deal breaker for me.

[drewdiver]

@ttr3dp LC_ALL="C" cmus works. However, I'm with @v1kn, I have a lot of Japanese jazz which now displays incorrectly.

[mitchblank]

Backtrace:

    frame #3: 0x00000001900e5eec libsystem_c.dylib`__assert_rtn + 284
  * frame #4: 0x00000001900e6534 libsystem_c.dylib`lookup_substsearch.cold.1 + 44
    frame #5: 0x00000001900c312c libsystem_c.dylib`lookup_substsearch + 108
    frame #6: 0x00000001900c357c libsystem_c.dylib`__collate_lookup_which + 192
    frame #7: 0x00000001900c3740 libsystem_c.dylib`__collate_xfrm + 380
    frame #8: 0x00000001900d54b0 libsystem_c.dylib`strxfrm_l + 256
    frame #9: 0x000000010002b768 cmus`u_strcoll_key + 64
    frame #10: 0x000000010002b8bc cmus`u_strcasecoll_key0 + 28
    frame #11: 0x0000000100027098 cmus`track_info_set_comments + 724
    frame #12: 0x0000000100003c70 cmus`cache_init + 888
    frame #13: 0x0000000100004d94 cmus`cmus_init + 24
    frame #14: 0x000000010002df24 cmus`main + 472
    frame #15: 0x000000018fe3eb4c dyld`start + 6000

So crashing inside libSystem's strxfrm() implementation.

It seems that recent changes to that function's error handling have caught out other projects; for example python/cpython#130567 might be related.

What cmus is doing with...

size_t xfrm_len = strxfrm(NULL, str, 0);

...seems to be relying on some non-standard assumption that strxfrm() returns the needed size on failure.

If you look at the change that cpython made for their bug ( https://github.com/python/cpython/pull/130619/files#diff-68d4a50f88ce330d08190024a328c2d0fc25b94a3d864796e590a34849450cc0L418 ) it seems they used to do something similar to this cmus code but replaced it with a grow-and-retry loop.

[v1kn]

@mitchblank I will say thank you for your investigation, even though i don't understand its contents at all 😂
Hope it's possible to get patched in cmus as well!

[mitchblank]

@drewdiver I also tried it with LC_ALL="C" cmus and it worked. Here's what locale outputs by default on my machine:

cmus also has its own environment variable $CMUS_CHARSET that overrides this it seems. For me CMUS_CHARSET=C cmus works fine. Handy if you just want to set a single environment variable in your shell and not have it affect other programs.

But it still isn't great if you have a lot of unicode artist and song names, since anything non-ASCII will renter as ?

[mitchblank]

Digging a bit more with some good ol' printf debugging. Looking more and more like a MacOS bug to be honest.

For me, it gets through a number of strings correctly... including ones with non-ASCII characters. The first thing that causes a crash is while it's loading the last track on Björk– Telegram -- namely Headphones (Ø Remix). It seems that passing that null-set character into strxfrm() is enough for an assertion failure.

Apple does have libc open-sourced at https://github.com/apple-oss-distributions/Libc but as far as I can tell they haven't posted the version from MacOS 15.4 yet (I think there is usually some lag between OS release and the open-source bits being pushed publicly) so it's not possible yet to see what changed between 15.3 and 15.4

The assertion itself in lookup_substsearch() is not new -- it's in locale/FreeBSD/collate.c. As that name implies the code is originally sourced from FreeBSD although it has diverged a bit. In the FreeBSD source tree it's in lib/libc/locale/collate.c but it's just called substsearch(). The assert existed there, going back years.

Most likely the xlocale_collate structure (internal to the locale code) is corrupt in some way... which makes me thing there is just a regression in MacOS itself

[mitchblank]

I don't know much about the internals of the locale code, but ultimately the xlocale_collate structs get read in at runtime from /usr/share/locale/en_US.UTF-8/LC_COLLATE or similar, so it's possible that something there changed in 15.4. Unfortunately I don't have any machines still running 15.3 handy to compare them. (Plus they're binary files, so not super readable anyway)

It is possible to repoint the locale code to a different tree entirely by setting $PATH_LOCALE which replaces the default /usr/share/locale directory, so that might be something to experiment with.

[mitchblank]

Unfortunately my attempt to create a minimal repro did not work:

#include <string.h>
#include <locale.h>

int main()
{
        setlocale(LC_COLLATE, "");
        return strxfrm(NULL, "headphones (ø remix)", 0);
}

This runs fine, so it's not like strxfrm() is just allergic to that string or something.

ui_curses.c does that setlocale() call early on and it does effect the return value (strxfrm wants to create a larger transformed string) but even without that line it doesn't crash.

So maybe some other locale API is being used in a weird way someplace? Or maybe even a memory corruption bug? Or something that only fails in multithreading?

I'm feeling stuck.

One thing that does make the problem go away is to just not load the collation tables at all. i.e.

diff --git a/ui_curses.c b/ui_curses.c
index 6ff7fa6..00c431f 100644
--- a/ui_curses.c
+++ b/ui_curses.c
@@ -2611,7 +2611,7 @@ int main(int argc, char *argv[])
        }

        setlocale(LC_CTYPE, "");
-       setlocale(LC_COLLATE, "");
+//     setlocale(LC_COLLATE, "");
        charset = getenv("CMUS_CHARSET");
        if (!charset || !charset[0]) {
 #ifdef CODESET

Unicode characters still display OK but obviously sorting might be different.

Still, no reason to think that the setlocale() is wrong (that code hasn't changed since 2010: 311ec12 ) but at least it avoids the weird crash.