Liquidsoap crashes when metadata is pushed after input.harbor connection in 2.2.2 #3574
Comments
|
Hi @SkillenUK, I have tried to reproduce the issue but have not been able to succeed. ChangesI need to modify the script to work in my environment. --- a/main.liq
+++ b/main.liq
@@ -3,26 +3,26 @@ log.level.set(4)
# Set the ssl certs for the harbor
cert = http.transport.ssl(
- certificate="/etc/ssl/certs/host_net.crt",
- key="/etc/ssl/private/host_net.key")
+ certificate="/tmp/test/cert.pem",
+ key="/tmp/test/key.pem")
# Define a function to handle harbor authentication
-def dj_auth(params)
- user = params.user
- password = params.password
-
- ret = process.read.lines("php /var/liquidsoap/scripts/liqauth.php '#{user}' '#{password}'")
- ret = list.hd(default="",ret)
-
- if ret == "true" then
- true
- else
- false
- end
-end
+#def dj_auth(params)
+# user = params.user
+# password = params.password
+#
+# ret = process.read.lines("php /var/liquidsoap/scripts/liqauth.php '#{user}' '#{password}'")
+# ret = list.hd(default="",ret)
+#
+# if ret == "true" then
+# true
+# else
+# false
+# end
+#end
# Define harbor's
-djlivessl = input.harbor(transport=cert,"live",port=8020,auth=dj_auth,buffer=2.)
+djlivessl = input.harbor(transport=cert,"live",port=8020,password="hackme",buffer=2.)
djlive = input.harbor("live",port=8010,password="hackme",buffer=2.)
# Append live to the metadata
@@ -38,7 +38,7 @@ djlive = metadata.map(append_live, djlive)
live = fallback(track_sensitive=false, [djlivessl,djlive])
# Simple auto dj
-autodj = playlist(mode="randomize",reload=1,reload_mode="rounds","/var/liquidsoap/autodj/autodj.m3u")
+autodj = playlist(mode="randomize",reload=1,reload_mode="rounds","/tmp/test/music")
# Append replay to the metadata
def append_autodj(m) =
@@ -49,14 +49,14 @@ end
autodj = metadata.map(append_autodj, autodj)
# Define transitions for changing to live
-pdlivejingle = single("/var/liquidsoap/jingles/liveshort.mp3")
+pdlivejingle = single("/tmp/test/liveshort.mp3")
def tolive(old,new)
sequence(merge=false,[once(pdlivejingle),(new:source)])
end
# Fallback to blank for drop outs on the input.harbor
-blankmp3 = single("/var/liquidsoap/jingles/blank.mp3")
+blankmp3 = single("/tmp/test/blank.mp3")
def toauto(old,new)
sequence(merge=false,[once(blankmp3),(new:source)])
@@ -87,18 +87,18 @@ liveout = source.on_frame(liveout, check_live)
output.icecast(
%mp3(bitrate=192),
mount="/live192",
- host="localhost", port=8000, password="hackme",
+ host="icecast", port=8000, password="hackme",
name="host.net", description="Main 192kbps MP3 stream", url="https://www.host.net", genre="Dance Music", liveout)
# Record the live streams
output.file(
%mp3(bitrate=192, id3v2=true),
- {time.string("/var/liquidsoap/shows/%Y-%m-%d-%H_%M_%S_djlive.mp3")},
+ {time.string("/tmp/%Y-%m-%d-%H_%M_%S_djlive.mp3")},
djlivessl,
fallible=true)
output.file(
%mp3(bitrate=192, id3v2=true),
- {time.string("/var/liquidsoap/shows/%Y-%m-%d-%H_%M_%S_djlive.mp3")},
+ {time.string("/tmp/%Y-%m-%d-%H_%M_%S_djlive.mp3")},
djlive,
fallible=true)Refactored script# Set log level
settings.log.level := 4
# Set the ssl certs for the harbor
cert = http.transport.ssl(
certificate="/tmp/test/cert.pem",
key="/tmp/test/key.pem"
)
# Define harbor's
djlivessl = input.harbor(
transport=cert,
"live",
port=8020,
password="hackme",
buffer=2.
)
djlive = input.harbor(
"live",
port=8010,
password="hackme",
buffer=2.
)
# Append live to the metadata
def append_live(m) =
[("artist", 'LIVE: #{m["artist"]}')]
end
djlivessl = metadata.map(append_live, djlivessl)
djlive = metadata.map(append_live, djlive)
# Set the fallback for the live streams
live = fallback([djlivessl, djlive], track_sensitive=false)
# Simple auto dj
autodj = playlist("/tmp/test/music")
# Append replay to the metadata
def append_autodj(m) =
[("artist", 'REPLAY: #{m["artist"]}')]
end
autodj = metadata.map(append_autodj, autodj)
# Define transitions for changing to live
pdlivejingle = single("/tmp/test/liveshort.mp3")
def tolive(_, new)
sequence([(once(pdlivejingle):source), new], merge=false)
end
# Fallback to blank for drop outs on the input.harbor
blankmp3 = single("/tmp/test/blank.mp3")
def toauto(_, new)
sequence([(once(blankmp3):source), new], merge=false)
end
# Set the fallback between live and auto
liveout = mksafe(
fallback(
[live, autodj],
transitions=[tolive, toauto],
transition_length=10.,
track_sensitive=false
)
)
# Skip the Auto DJ to the next track when going live
thread.when(getter.changes(live.is_ready), {live.is_ready() ? autodj.skip() : ()})
# Icecast output MP3 192 kbps stereo
output.icecast(
%mp3,
liveout,
mount="/live192",
host="icecast"
)
# Record the live streams
output.file(
%mp3(bitrate=192, id3v2=true),
{time.string("/tmp/%Y-%m-%d-%H_%M_%S_djlive.mp3")},
djlivessl,
fallible=true
)
output.file(
%mp3(bitrate=192, id3v2=true),
{time.string("/tmp/%Y-%m-%d-%H_%M_%S_djlive.mp3")},
djlive,
fallible=true
) |
|
Hi sorry I should have mentioned. This is Liquidsoap version 2.2.2, installed via OPAM in Ubuntu 22.04.3 LTS. I'm installing... I'll try and do some more testing to work out how to reproduce, but it doesn't happen every time so it's tricky to pin it down. I have seen the issue on two different VPS's from different suppliers with different hardware configurations, so I don't think it's happening on a specific processor type. Thanks, Skillen. |
|
Hi, I'm just having another look at this. I am now able to reproduce it quite consistently. As I said before it doesn't happen every time so you need to connect BUTT to the ssl input.harbor and push some metadata a couple of times. It only seems to happen when I go in to the BUTT settings, select the stream tab and enter something in to the 'update song name manually' field then click on OK. I have now managed to make it fault when doing this after connecting, so it's definately related to setting the metadata and not the initial connection as I first suspected. I don't know if any other method of pushing metadata causes an issue because this is the method which we normally use, we also only use BUTT these days. I have ran Liquidsoap from the shell now so that I can see any output and when the process ends, it closes with the message...
I have tried removing the metadata.map routines from the script to eliminate them and it still happens. I have also tried removing the transitions and it still happens, so I have eliminated these. When I connect on the input.harbor which doesn't use ssl then I can't get it to fault. It's definately related to the input.harbor with ssl and pushing metadata from BUTT. The simplified script which I can now reproduce the issue with is... I have also simplified my liqauthdev.php script so that it simply returns 'true' and doesn't do anything else. I hope this helps, thanks, Skillen. |
|
Is it possible to exclude
By the way, could you post the result of the Conditionsdef one()
if ret == "true" then
true
else
false
end
end
def two()
ret == "true"
end
def three()
bool_of_string(ret, default=false)
end |
|
Hi, the output from the To answer your questions...
Thanks, Skillen. |
|
I reproduced the problem with the butt on ubuntu:22.04.
Log |
|
Thanks for your work. It would be great for us if we could get a fix in the next version. Skillen. |
|
Hi @toots, is this something which you could take a look at as it has been reproduced? Thanks, Skillen. |
|
Thanks for the detailed reproduction steps. I will test them locally. One bit that would help a lot in situations like these is a gdb trace of the segfault. It's pretty easy to grab: |
|
I am not able to reproduce with the latest Couple of related remarks:
|
|
I've refined reproduction script. Here's my trace from 2.2.3.8dcbe42 |
|
Hi, thank you for looking at this. I've installed and ran Next I've removed Let me know if you'd like me to try anything else, thanks, Skillen. |
|
Thanks y'all. Unfortunately, I still cannot reproduce on this debian amd64 machine. I think in this case the next step would be to confirm that it is what I think it is which is a memory corruption happening when one of the SSL binding C elements is garbage collected shortly after being created. Would y'all be able to test with TLS and see if the segfault goes away with it? |
|
I don't know why, but maybe something is broken. settings.log.level := 4
# Set the ssl certs for the harbor
cert = http.transport.tls(
certificate="cert.pem",
key="key.pem"
)
# Main DJ live harbor using ssl and website account authentication
live = input.harbor("live", transport=cert, port=9020)
output.dummy(live, fallible=true)BTW, I can reproduce on the debian:12 + opam. |
|
I was only able to reproduce this on debian. I added a proper domain name to the machine and generated a self-signed cert for it and it started working. Looks like |
|
I'm struggling to get tls working as well. I've installed it with opam, generated some self signed certs and implemented it in the script the same way as @vitoyucepi has. But it doesn't seem to find the package... It's not something which I have used before so I may have missed something, thanks, Skillen. |
|
I use opam to install packages, so The right solution for me is to terminate encrypted connections on nginx. |
|
@vitoyucepi thank you for your help again, you've been fantastic helping me with this issue :) I've got tls kind of working. I can connect to the harbor from BUTT and it appears that pushing the metadata doesn't cause a crash. However after disconnecting from the harbor, I am unable to reconnect again but Liquidsoap is still running. Then if I kill Liquidsoap, a couple of Liquidsoap processes appear with different PID'a which are using 100% of the CPU and are difficult to kill. I think this proves that the tls method doesn't have the metadata issue, although I can't be completely sure because of my inability to reconnect and the other issue which arises. I don't think I want to try and get tls working as an alternative because I don't want to use unsigned certificates, I am using nginx already so I'll look at setting up a proxy pass to the harbor and see if I can get that working. Thanks for the suggestion. |
|
I've been able to reproduce repeated TLS connections here. However, I agree that if you can use |
|
I have reported this to |
|
Great, thanks again both for your help. I'll work on setting up NGINX. Skillen. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have been having an issue with Liquidsoap crashing sometimes when a user connects to the input.harbor using BUTT as a client. I think it happens when the user has set some metadata in BUTT before connecting (select the 'stream' page in the settings, set the 'update song manually' field and click on 'OK'). I also think that setting the metadata after connecting isn't causing an issue. I haven't managed to bottom out specifically what causes the crash, but it could be the metadata.map function so I have included this in the script example. It often takes me a handful of connections to reproduce the issue, so I do not think it happens every time.
I also think this only occurs on the input.harbor which uses ssl. I have not seen the issue on the non-ssl harbor to date, however most of my testing has been on the ssl harbor so it could be possible. I've included both parts in the script as it could be relevant, but for reproduction it's best to focus on the ssl harbor...
Multiple outputs are used to Icecast in the main script, but I do not think this is relevant.
I use supervisor to run Liquidsoap, so when it crashes it is instantly restarted. This is why you can see the LOG START event indicating where Liquidsoap crashed and was restarted. The log extract is as follows...
The last entry in the log before Liquidsoap restarts always seems to be the 'New metadata chunk' entry, which is why I feel this is related to the metadata.
Thank you in advance for your help,
Skillen.
The text was updated successfully, but these errors were encountered: