20.0-testing-r2181 / AMD64: included unrar gives core dump, leading to failed downloads

[sanderjo]

20.0-testing-r2181 on Ubuntu 18.04 on AMD64 aka x86_64: included unrar gives core dump, leading to failed downloads "Unpack for <blabla> failed."

See https://forum.nzbget.net/viewtopic.php?f=3&t=3212&p=20545#p20545

sander@witte:~/nzbget$ ./unrar 
unrar: loadlocale.c:129: _nl_intern_locale_data: Assertion `cnt < (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))' failed.
Aborted (core dumped)

sander@witte:~/nzbget$ ./nzbget --version
nzbget version: 20.0-testing-r2181
sander@witte:~/nzbget$ 

Proof the problem is in nzbget-20's unrar (and workaround): Replacing with unrar from nzbget-19.1 'solves' it.
Another workaround: in nzbget's settings, point to the unrar provided by Linux.
The unrar from the rarlab works too https://www.rarlab.com/rar/rarlinux-x64-5.5.0.tar.gz

EDIT:

FWIW:

If I set LC_ALL to C (hadolint/hadolint#173 (comment)), the faulty unrar works:

sander@witte:~/nzbget$ env LC_ALL=C ./unrar-crash-crash 

UNRAR 5.50 freeware      Copyright (c) 1993-2017 Alexander Roshal

Usage:     unrar <command> -<switch 1> -<switch N> <archive> <files...>
               <@listfiles...> <path_to_extract\>

EDIT 2: FWIW: the x86_32 unrar is ok:

sander@haring:~/weg11111/nzbget$ ./nzbget --version
nzbget version: 20.0-testing-r2181

sander@haring:~/weg11111/nzbget$ ./unrar  | head -2

UNRAR 5.50 freeware      Copyright (c) 1993-2017 Alexander Roshal


sander@haring:~/weg11111/nzbget$ ll unrar
-rwxr-xr-x 1 sander sander 537204 May 13 21:40 unrar*

EDIT 3: FWIW

On an old Ubuntu 12.04 (also x86_64), the included unrar does work. Strange. Is the problem glibc version dependent (as the loadlocale.c seems to be a part of glibc)?

The included unar works too on
Ubuntu 16.04, with GNU C Library (Ubuntu GLIBC 2.23-0ubuntu10) stable release version 2.23
Ubuntu 17.10 with GNU C Library (Ubuntu GLIBC 2.26-0ubuntu2.1) stable release version 2.26

(On Ubuntu 18.04: GNU C Library (Ubuntu GLIBC 2.27-3ubuntu1) stable release version 2.27, so if it's glibc, it's something that changed between 2.26 and 2.27)

[hugbug]

Unrar shipped with nzbget installer is statically linked; that's to make it compatible with many Linux distributions.

In v19 and early versions unrar was compiled using uClibc (just like nzbget itself). In v20 I'm using glibc for x86_64 and armhf; for other targets unrar is still compiled using uClibc. That explains why 32 bit version of unrar works - it's compiled with uClibc.

I think Ubuntu 18.04 has changes related to locales which probably require newer glibc or it must be configured differently.

Unrar from rarlab is dynamically linked against glibc and therefore uses the installed version of glibc. Although that makes it work with Ubuntu 18.04 the dynamic linking reduce compatibility with other systems. In particular it will not work on systems which don't use glibc (like embedded) or use older glibc versions than the one used when unrar was compiled.

Anyway I'm questioning why unrar needs locale support at all. The source code contains a call to function setlocale which initialises locale according to environment variables. I don't see any localisations in unrar though. For example the output of unrar l archive.rar uses hard coded format for time.

I removed the call to setlocale and now unrar seems to work. @sanderjo, I'll send you the new binary to try.

[sanderjo]

Your unrar works on Ubuntu 18.04 (see below). Nice.

So you removed setlocale from unrar itself and unrar still works? Cool ... worth a patch towards rarlab?

BTW: I thought unrar from rarlab was self-contained, but now I see "ldd unrar" indeed show libraries (see below)

sander@witte:~/Downloads$ ./unrar-x86_64-patched | head -4

UNRAR 5.50 freeware      Copyright (c) 1993-2017 Alexander Roshal

Usage:     unrar <command> -<switch 1> -<switch N> <archive> <files...>


sander@witte:~/Downloads$ ldd unrar-x86_64-patched 
	not a dynamic executable



sander@witte:~/Downloads$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04 LTS
Release:	18.04
Codename:	bionic

The rarlab's unrar ... with libraries indeed:

sander@witte:~/Downloads/rarlab-blabla/rar$ ldd unrar 
	linux-vdso.so.1 (0x00007fffabd6a000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f3dde3bd000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3dde01f000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f3ddde07000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3dddbe8000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3ddd7f7000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f3dde74b000)

[BartmanAbyss]

btw, same behavior with included 7za binary (Ubuntu 18.04)

[sanderjo]

btw, same behavior with included 7za binary (Ubuntu 18.04)

Oh, yes indeed:

sander@witte:~/nzbget$ ./7za 
7za: loadlocale.c:129: _nl_intern_locale_data: Assertion `cnt < (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))' failed.
Aborted (core dumped)

sander@witte:~/nzbget$ env LC_ALL=C ./7za | head -3

7-Zip (a) [64] 9.38 beta  Copyright (c) 1999-2014 Igor Pavlov  2015-01-03
p7zip Version 9.38.1 (locale=C,Utf16=off,HugeFiles=on,4 CPUs)

sander@witte:~/nzbget$ ./nzbget --version
nzbget version: 20.0-testing-r2181
sander@witte:~/nzbget$ 

[hugbug]

@BartmanAbyss thanks for mentioning this.
Reopening issue.

[hugbug]

Applied the same fix to 7zip.