Can not connect to the streaming server. Problem with Liquidsoap

[GFMradio]

OS : Ubuntu 18.04.6
PC : Z440 workstation

By January, LibreTime had been installed and used on three other Z440 workstations.
These three LibreTime work well so far.

However, installing a new LibreTime from last week will cause errors such as images.

< Can not connect to the streaming server
Problem with Liquidsoap...>

Is there any way to solve this?

[GFMradio]

I solve it.
https://discourse.libretime.org/t/libretime-not-on-air-getting-information-from-server/1038/6

[paddatrapper]

I can replicate this on a Ubuntu 20.04 Vagrant box. Looks like liquidsoap is having issues:

2022/02/16 08:13:49 [lang:3] timeout --signal=KILL 45 libretime-playout-notify stream '1' '1644999089.78' --error='could not write data to host: Broken pipe in write()' &                                        
2022/02/16 08:13:49 [LibreTime!:3] Closing connection...                                                                                                                                                          
2022/02/16 08:13:49 [LibreTime!:3] Will try to reconnect in 5.00 seconds

[paddatrapper]

Looks like liquidsoap can connect to Icecast and continues working until it needs to stream a file/media. Then it is unable to stream successfully and eventually resets the stream to try again. This loops indefinitely.

It also seems to be sending a lot of notify messages, that are then flushed on restart. My guess is that they are being queued faster than they can be sent. E.g. kicking of a music track generates 121 (and climbing) notify processes on my machine. Removing https://github.com/LibreTime/libretime/blob/main/playout/libretime_liquidsoap/1.4/ls_script.liq#L81 fixed the playout, but obviously is not the fix, as that removes all the communication back to the UI

[jooola]

@paddatrapper You are described an issue that have mostly nothing to do with initial one. Maybe we should retitle the issue ?

I did reproduce this, and I could workaround it by installing a recent version of liquidsoap (1.4.4).

I wonder if the ppa change did affect this ? Has this been working on focal already ?

I did some research on the liquidsoap side:

• Metadata Issue in 1.4.1 savonet/liquidsoap#1541
• Unexpected behaviour when inserting metadata in 1.4.1 savonet/liquidsoap#1115

[paddatrapper]

@jooola the result of the issue is the same as presented above - you get an Unable to connect to the streaming server error

I usually test PRs on focal, so it was working previously...

[paddatrapper]

I'm busy building a newer version of liquidsoap for Xenial at the moment. Will push it to the PPA and then we can enable and install from the PPA in the install script for Xenial only

[jooola]

Xenial ? Aren't you referring to focal or even bionic ??

[paddatrapper]

Yes, Bionic and focal - 20.04 and 18.04

[jooola]

@paddatrapper Could you report here where you are facing an issue ? I've been digging in your latest changes on salsa, and it seems the problem is ocaml-ogg ? Or are there other packages that are failing ? Should I step in and try to search for fixes ?

[paddatrapper]

It was a local configuration issue which I have managed to fix. Busy building now

[paddatrapper]

This issue doesn't seem to be a liquidsoap version, or at least there is a change that we have made somewhere that has caused us to run into an upstream bug. Running the same liquidsoap version and upgrading from alpha.10 to master caused the issue to appear on 18.04. Running alpha.10 worked without a problem

[jooola]

Wondering if we know if the problematic libretime-playout-notify call is failing, and called again on failure ?

[paddatrapper]

It doesn't appear to be failing, at least the exit code when called manually is 0

[paddatrapper]

Hmmm, interesting. Testing this morning on 18.04 and I still get the Can not connect to the streaming server. Problem with Liquidsoap..., however, playout is still happening correctly. Which is different behaviour from 20.04. This is less of a problem, as the playout still works, just doesn't seem to report back properly (maybe missing a notify call somewhere in the script for liquidsoap 1.1.1)

[paddatrapper]

Tested an upgrade on Ubuntu 18.04 from alpha.10 to current main (with #1707 merged) and playout is working fine. However it does seem something in the feedback loop is broken - the stream status and the on-air button indicate issues. There aren't any in the playout system though.

[jooola]

Is this fixed after a playout/liquidsoap restart ?

[paddatrapper]

Not in my experience

[jooola]

I'll investigate tomorrow, I won't have time for today.

[jooola]

Here is the problematic error message:

Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]: Traceback (most recent call last):
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:   File "/usr/local/bin/libretime-playout-notify", line 33, in <module>
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:     sys.exit(load_entry_point('libretime-playout==1.0', 'console_scripts', 'libretime-playout-notify')())
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:   File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1128, in __call__
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:     return self.main(*args, **kwargs)
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:   File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1034, in main
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:     _verify_python_env()
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:   File "/usr/local/lib/python3.6/dist-packages/click/_unicodefun.py", line 100, in _verify_python_env
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:     raise RuntimeError("\n\n".join(extra))
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]: RuntimeError: Click will abort further execution because Python was configured to use ASCII as encoding for the environment. Consult https://click.palletsprojects.com/unicode-support/ for mitigation steps.
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]: This system supports the C.UTF-8 locale which is recommended. You might be able to resolve your issue by exporting the following environment variables:
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:     export LC_ALL=C.UTF-8
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:     export LANG=C.UTF-8
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]: Traceback (most recent call last):
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:   File "/usr/local/bin/libretime-playout-notify", line 33, in <module>
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:     sys.exit(load_entry_point('libretime-playout==1.0', 'console_scripts', 'libretime-playout-notify')())
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:   File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1128, in __call__
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:     return self.main(*args, **kwargs)
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:   File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1034, in main
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:     _verify_python_env()
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:   File "/usr/local/lib/python3.6/dist-packages/click/_unicodefun.py", line 100, in _verify_python_env
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:     raise RuntimeError("\n\n".join(extra))
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]: RuntimeError: Click will abort further execution because Python was configured to use ASCII as encoding for the environment. Consult https://click.palletsprojects.com/unicode-support/ for mitigation steps.
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]: This system supports the C.UTF-8 locale which is recommended. You might be able to resolve your issue by exporting the following environment variables:
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:     export LC_ALL=C.UTF-8
Mar 29 10:22:01 vagrant libretime-liquidsoap[10038]:     export LANG=C.UTF-8

According to https://click.palletsprojects.com/en/8.1.x/unicode-support/ we need to export some of these locales info. This is only required for python 3.6, python3.7 works fine.