Mono dependent plugins .e.g radarr and potentially sonarr no longer work #165

balbassam · 2021-02-17T18:39:13Z

It looks like to me that something broke mono in the latest updates.

Steps to reproduce on TrueNAS are as follows:

Install radarr (or sonarr) plugin using the gui. Find that the management interface doesn't work.

Running a shell in the plugins jail, I tried to run the binaries directly and find that there's an undefined symbol in mono.

For sonarr:

/usr/local/bin/mono /usr/local/share/sonarr/NzbDrone.exe --nobrowser --data=/usr/local/sonarr
ld-elf.so.1: /usr/local/bin/mono: Undefined symbol "pthread_setname_np@FBSD_1.6"

For radarr:

/usr/local/bin/mono /usr/local/share/radarr/Radarr.exe
ld-elf.so.1: /usr/local/bin/mono: Undefined symbol "pthread_setname_np@FBSD_1.6"

I have a sonarr plugin that I haven't updated that still works, while my radarr plugin that I did update recently broke.

The text was updated successfully, but these errors were encountered:

repvik · 2021-02-19T15:24:30Z

Can confirm this issue. Seeing the same missing symbol.

mikk150 · 2021-02-19T16:13:08Z

Emby has same issue as well

fulder · 2021-02-19T22:01:07Z

After searching and digging around a bit after the cause for this issue I think I finally managed to find something. It look like mono on 12.1 version is a bit different build compared to e.g. 12.2 version.

12.1:

Mono JIT compiler version 5.10.1.57 (5.10.1.57 Wed Feb 17 21:18:53 UTC 2021)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           __thread
	SIGSEGV:       altstack
	Notification:  kqueue
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug 
	Interpreter:   yes
	LLVM:          supported, not enabled.
	GC:            sgen (concurrent by default)

12.2:

Mono JIT compiler version 5.10.1.57 (5.10.1.57 Tue Feb 16 01:44:31 UTC 2021)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           __thread
	SIGSEGV:       altstack
	Notification:  kqueue
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug 
	Interpreter:   yes
	LLVM:          supported, not enabled.
	GC:            sgen (concurrent by default)

The version is actually the same, although the build time on 12.1 FreeBSD is 1 day newer. Furthermore the FSBD_1.6 seems to be a Symbol Version variable for FreeBSD 13 (see: https://wiki.freebsd.org/SymbolVersioning).

My theory is that the latest mono compile/build for 12.1 FreeBSD somehow managed to use the invalid FSBD_1.6 instead of FSBD_1.5 which could cause the problems we are seeing. A quickfix could be to simply update the affected plugin manifests to the newer (and working) FreeBSD version.

UPDATE: Version 12.1 of FreeBSD is actually already EOL so upgrading to 12.2 is probably a good idea after all.

Thefrank · 2021-02-20T18:43:47Z

mono need to be built under the (oldest) FreeBSD version they will be running under which is likely why you are seeing those symbol version errors. dotNET runs into this same problem (cross built under 12 will not run under 11)

edit: this usually should not be an issue as there is some compatibility inside the libraries to handle old APIs but I guess that is not the case here

altonius · 2021-02-27T05:34:21Z

I upgraded my mono in the jail to 6.8.0.105 and resolved the issue.

fulder · 2021-02-28T10:48:00Z

Another good explanation related to this issue: https://forums.FreeBSD.org/threads/python-undefined-symbol-close_range-fbsd_1-6.78983/post-495312. I will create a new PR shortly upgrading the affected plugins to the latest non-EoL FreeBSD version.

spacecabbie · 2021-02-28T18:06:48Z

I upgraded my mono in the jail to 6.8.0.105 and resolved the issue.

I am on 6 never had the issue. We should really step away from 5.

jryski · 2021-03-01T04:02:16Z

I upgraded my mono in the jail to 6.8.0.105 and resolved the issue.

I am on 6 never had the issue. We should really step away from 5.

I'm on the same version of Mono 6.8.0.105, on Truenas 12.2U2 and Radarr service will not launch. Not in the shared jail I had it running in, nor in a new jail created for testing. This stopped working in the last 48 hours for me. My truenas ui failed to present the login prompt, which led to me restarting the entire server, Radarr would not load after restart. I've tried reinstalling, restoring from backup, and updating every related pkg I could think of.

spacecabbie · 2021-03-01T06:53:22Z

Who maintains the plugin we have a dotnet version running with radarr it would be a great way to avoid this mono issues.,

tprelog · 2021-03-01T18:07:39Z

Who maintains the plugin

The easiest way to determine this, is by checking where the artifact repository point to, in the plugin's manifest.

https://github.com/ix-plugin-hub/iocage-plugin-index/blob/master/radarr.json#L4

radarr, looks like it may have been previously maintained by ix-systems, but has now been moved here, to the community plugins. I think two possible options are

Fork the artifact repo and apply your changes. Then create a PR back to upstream artifact repo.
Fork the artifact repo and apply your changes. Then create a PR, changing the artifact repo to use your fork, and have more control.

fulder · 2021-03-01T19:50:55Z

Who maintains the plugin

In radarr and sonarr cases this is a bit more complected than just the plugin maintainer as both are installed using a prebuilt FreeBSD package with mono dependencies. (I'm guessing we could change these plugins and install them into a more manual manner, but not sure that's the right way to go here, see section below)

IMO the correct way of updating mono for these specific plugins would be to contact the package maintainer and update the mono dependencies there. I don't really know the fastest process of requesting/making such changes, have just tried to mail the corresponding package maintainer asking for this update.

fulder · 2021-03-01T20:09:11Z

have just tried to mail the corresponding package maintainer asking for this update

Got a really fast response explaining mono 6.X have no good FreeBSD support for this time being with a link to a long issue/discussion about this: https://reviews.freebsd.org/D23300

spacecabbie · 2021-03-02T07:28:07Z

Alright fair but the lads at radarr have dotnet working its smooth sailing i am beta testing it now but I do not know how to update a plugin. Zo figured I would tag the maintainers but as I understand this there is none atm ?

repvik · 2021-03-02T12:35:15Z

Make an alternative plugin (eg. radarr-dotnet) and get that added to community plugins?

fulder · 2021-03-18T15:51:06Z

The PR upgrading the mono dependent plugins from FreeBSD version 12.1 to 12.2 (#179) should be merged and ported back to the 12.2-RELEASE branch (#195). I hope this should solve this issue and we could maybe close this one?

If you have some time, please refresh your community plugin index in the UI and verify if this fix works for you as well.

balbassam · 2021-03-18T16:04:35Z

Just installed a fresh radarr plugin on my truenas machine and it works! :)
Thanks for all the work!

applemang · 2021-03-20T13:57:54Z

The PR upgrading the mono dependent plugins from FreeBSD version 12.1 to 12.2 (#179) should be merged and ported back to the 12.2-RELEASE branch (#195). I hope this should solve this issue and we could maybe close this one?

If you have some time, please refresh your community plugin index in the UI and verify if this fix works for you as well.

Is a fresh install required here or should we able to update the community plugins and then do an 'Update' via the UI?

fulder · 2021-03-20T21:52:51Z

Is a fresh install required here or should we able to update the community plugins and then do an 'Update' via the UI?

@applemang I tried upgrading sonarr (using the UI) which worked just fine but I had troubles with radarr (not sure if there was something broken with my jail), ended up installing a fresh one of it instead. My tip would be to run and download a backup, try an upgrade and in worst case install a fresh plugin and restore the backup (this is what I ended up doing with radarr).

jryski · 2021-03-21T18:01:33Z

I was able to install via the GUI, but installing in a jail using "pkg install" still results in radarr service being unable to start.

…

On Sat, Mar 20, 2021 at 5:53 PM Michal Sadowski ***@***.***> wrote: Is a fresh install required here or should we able to update the community plugins and then do an 'Update' via the UI? I tried upgrading sonarr (using the UI) which worked just fine but I had troubles with radarr (not sure if there was something broken with my jail), ended up installing a fresh one of it instead. My tip would be to run and download a backup, try an upgrade and in worst case install a fresh plugin and restore the backup (this is what I ended up doing with radarr). — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#165 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQTI7SIQ7LUPQKY7RLWYFLTTEUKMDANCNFSM4XY33HGQ> .

fulder mentioned this issue Feb 23, 2021

Cirrus CI tasks for plugin installation tests #169

Merged

This was referenced Mar 3, 2021

UI portal install test support #178

Merged

UI portal fixes #179

Merged

balbassam closed this as completed Mar 18, 2021

fulder mentioned this issue Jun 11, 2021

Sonarr and Radarr Default to Outdated Mono-5.10 #238

Closed

morrell7 mentioned this issue Feb 20, 2022

new install TrueNAS-12.0-U8: radarr connection refused #297

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mono dependent plugins .e.g radarr and potentially sonarr no longer work #165

Mono dependent plugins .e.g radarr and potentially sonarr no longer work #165

balbassam commented Feb 17, 2021 •

edited

repvik commented Feb 19, 2021

mikk150 commented Feb 19, 2021

fulder commented Feb 19, 2021 •

edited

Thefrank commented Feb 20, 2021 •

edited

altonius commented Feb 27, 2021

fulder commented Feb 28, 2021

spacecabbie commented Feb 28, 2021

jryski commented Mar 1, 2021

spacecabbie commented Mar 1, 2021

tprelog commented Mar 1, 2021

fulder commented Mar 1, 2021

fulder commented Mar 1, 2021

spacecabbie commented Mar 2, 2021

repvik commented Mar 2, 2021

fulder commented Mar 18, 2021 •

edited

balbassam commented Mar 18, 2021

applemang commented Mar 20, 2021

fulder commented Mar 20, 2021 •

edited

jryski commented Mar 21, 2021 via email

Mono dependent plugins .e.g radarr and potentially sonarr no longer work #165

Mono dependent plugins .e.g radarr and potentially sonarr no longer work #165

Comments

balbassam commented Feb 17, 2021 • edited

repvik commented Feb 19, 2021

mikk150 commented Feb 19, 2021

fulder commented Feb 19, 2021 • edited

Thefrank commented Feb 20, 2021 • edited

altonius commented Feb 27, 2021

fulder commented Feb 28, 2021

spacecabbie commented Feb 28, 2021

jryski commented Mar 1, 2021

spacecabbie commented Mar 1, 2021

tprelog commented Mar 1, 2021

fulder commented Mar 1, 2021

fulder commented Mar 1, 2021

spacecabbie commented Mar 2, 2021

repvik commented Mar 2, 2021

fulder commented Mar 18, 2021 • edited

balbassam commented Mar 18, 2021

applemang commented Mar 20, 2021

fulder commented Mar 20, 2021 • edited

jryski commented Mar 21, 2021 via email

balbassam commented Feb 17, 2021 •

edited

fulder commented Feb 19, 2021 •

edited

Thefrank commented Feb 20, 2021 •

edited

fulder commented Mar 18, 2021 •

edited

fulder commented Mar 20, 2021 •

edited