Support Gameday Audio streams

[tonycpsu]

No description provided.

[mattiasa]

With mlbhls, we always used to get the alternative audio streams as separate tracks in the MPEG-TS stream, but with mlbstreamer I'm only seeing the primary tv audio. Do you happen to know if this is something mlb.tv has changed or is it something streamlink is doing to the HLS stream?

I can get on debugging this more in depth as the radio stream is much more interesting to me than the tv audio, but I figured I'd ask if you already know the answer.

[ChrisVine]

The MLB.tv browser streams are not offering audio overlay at the moment. They have promised it for the regular season so it will be interesting to see whether streamlink handles it.

[mattiasa]

Ok, then let's revisit this after opening day. If the track is included then it should be relatively simple to dig it out.

[mattiasa]

Ok, I've been digging into this and is seems that the standard HLS stream you get with streamlink has video and TV audio embedded in the same segments. However, the HLS stream also defines a number of additional audio tracks which are available as separate streams

#EXTM3U
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",LANGUAGE="en",NAME="English",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-MEDIA:TYPE=CLOSED-CAPTIONS,GROUP-ID="cc",LANGUAGE="en",NAME="English",INSTREAM-ID="CC1"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",NAME="Natural Sound",LANGUAGE="zxx",AUTOSELECT=NO,URI="ballpark_48K/48_complete-trimmed.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",NAME="English Radio",LANGUAGE="en",AUTOSELECT=NO,URI="eng_48K/48_complete-trimmed.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",NAME="Radio Española",LANGUAGE="es",AUTOSELECT=NO,URI="spa_48K/48_complete-trimmed.m3u8"

These can be selected with the parameter --hls-audio-select to streamlink.

However, there's also a bug in ffmpeg which streamlink uses to mux together the different tracks, causing it to fail and we end up with a silent audio track.

I have submitted a patch to ffmpeg at https://patchwork.ffmpeg.org/patch/8285/

With this patch applied to my ffmpeg and --hls-audio-select English Radio as an additional parameter to StreamLink I get an mpeg2ts stream with two audio tracks. It seems streamlink doesn't handle that one audio stream is interleaved with the video and one is external, but that looks like it should be solvable in the streamlink's ffmpeg mux code.

How do you want to handle this in mlbstreamer? I could add a command-line option to select the prefered audio, and if you give it ? as a parameter then it will list all the available audio tracks, for example.

[ChrisVine]

As I mentioned here, I don't get a silent track with 'English Radio' or 'Natural Sound': I seem to get an unreliable version of the TV audio feed.

Your suggested approach looks good. Better perhaps would be to leave the selection of the audio feed to the video player (mplayer and vlc both handled multiple audio feeds with mlbviewer) so that the audio feed can be chosen 'on the fly' without having to restart the stream, but that looks to be beyond the capabilities of streamlink.

[tonycpsu]

Great stuff, folks. @mattiasa: my understanding is that with this patch and some fixes to streamlink, it would be possible to select the preferred audio when streaming / downloading, but that other audio tracks wouldn't be embedded in the file at all. Is this correct? If so that's certainly better than not being able to listen to the alternate tracks at all, but I'm wondering if we could then do something to get the other tracks and mux them in such that they can be selected by the viewer when watching the stream, as was the case when using mlbviewer/mlbhls in previous seasons.

[tonycpsu]

...and I now see that @ChrisVine just asked the same question, so it appears that jetlag is getting in the way of my reading comprehension. :)

[mattiasa]

@tonycpsu That's roughly correct. I posted a longer explanation to the forum post @ChrisVine linked to but it's currently being moderated since I haven't been using the LinuxQuestions forum.

Without any changes to streamlink you will get TV video, TV audio and whichever stream you choose with the hls-audio-select option to streamlink. This is because TV audio and TV video are muxed into the same segments. I'm hoping to patch streamlink and get it to download all hls audio streams if you pass in some special option. I'm looking into that right now. In that case, maybe we don't need any options at all to mlbplay and can just always download all audio streams.

[mattiasa]

With a patched version of streamlink I now get:

Input #0, mpegts, from 'mlb.20180329.nyy@tor.1537.yes.ts':
  Duration: 00:01:20.75, start: 1.466733, bitrate: 1482 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Data: timed_id3 (ID3  / 0x20334449)
    Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 123 kb/s
    Stream #0:2[0x102]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(progressive), 640x360 [SAR 1:1 DAR 16:9], Closed Captions, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Stream #0:3[0x103]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, mono, fltp, 46 kb/s
    Stream #0:4[0x104]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, mono, fltp, 42 kb/s
    Stream #0:5[0x105]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, mono, fltp, 42 kb/s

and if I open that stream with VLC I can select between 4 different audio tracks, TV, Radio, Spanish and Park sounds.

This looks promising.

[mattiasa]

Fyi, I have gotten patches into both streamlink and ffmpeg now and as soon as both of those are released I'll submit patches to make mlbstreamer make use of it. Meanwhile, anyone who wants to build it themselves can install from the master branch of ffmpeg and streamlink, and then add the following to ~/.config/mlbstreamer/config.yaml

streamlink_args: --hls-audio-select '*'

[ChrisVine]

The streamlink patch to take a wildcard does not work for me, whereas if I pass streamlink_args

--hls-audio-select 'English Radio'

then it does work to provide the TV and radio english audio feeds. This is when testing on two games, the current sd/sf game and the current ari/lad game.

This is with streamlink master (bd3ff1f). Does it work for you?

[mkomko]

@ChrisVine I suppose you need the ffmpeg patch as well.

[ChrisVine]

I misspoke. It is mlbstreamer which is at bd3ff1f. streamlink is at 2f994acbe5bac2642c0267638fbdfdb7a48a275a

@mkokmo: No, I have the ffmpeg patch installed. Otherwise English Radio wouldn't work.

[ChrisVine]

@mkomko: are you saying it works for you?

[mkomko]

@ChrisVine No, I haven't tried it yet. I'm waiting until releases that include the patches are available.

[ChrisVine]

@mkomko: the patches are in streamlink master and ffmpeg master. Just be careful when installing ffmpeg master to put it out of the way of your distro's ffmpeg as it is ABI incompatible with ffmpeg-3.4. Install it in /opt/ffmpeg36 or something like that and call your upgraded ffmpeg binary something like ffmpeg36. Also add /opt/ffmpeg36/lib[64] to your ld.so.conf file or bring up ffmpeg36 with LD_LIBRARY_PATH set.

[ChrisVine]

@mkomko: Oh, and also pass streamlink your upgraded ffmpeg binary. So streamlink_args becomes something like:

streamlink_args: --ffmpeg-ffmpeg /usr/bin/ffmpeg36 --hls-audio-select 'English Radio'

which works, or

streamlink_args: --ffmpeg-ffmpeg /usr/bin/ffmpeg36 --hls-audio-select '*'

which doesn't (for me).

[ChrisVine]

As a further datum point, this works to enable a choice of (i) TV audio, (ii) ballpark sounds, or (iii) radio audio:
streamlink_args: --ffmpeg-ffmpeg /usr/bin/ffmpeg36 --hls-audio-select 'English Radio,Natural Sound'

It's just the wildcard which doesn't work.

[mattiasa]

That is weird, as the wildcard "works for me".

@ChrisVine Could you apply #23 which will show the full command line which is sent to both streamlink and ffmpeg?

[ChrisVine]

You don't say whether any additional command is required or where logging takes place (I assume stdin), but with the logging patch applied and starting mlbplay with this command line:

mlbplay -d 2018-04-15 -r 540p sf

and streamlink_args of:

streamlink_args: --ffmpeg-ffmpeg /usr/bin/ffmpeg35 --hls-audio-select '*'

gives this:

[cli][info] Found matching plugin hls for URL https://hlslive-aksc-ewr1.media.mlb.com/ls01/mlb/2018/04/15/Away_VIDEO_eng_San_Francisco_Giants_San__20180415_1523816573529/master_wired60_complete-trimmed.m3u8
error: 'ascii' codec can't encode characters in position 10-11: ordinal not in range(128)

[ChrisVine]

Are you using python-3? I am using python-2.7

[tonywagner]

Works great for me, except I notice some of the alternate audio tracks are not synchronized with the video. Seems to happen both live and archive, and whether I started from the beginning or at an offset. I can use VLC to adjust the track synchronization manually (like the April 15th Milwaukee game, I set it to -13 for the radio track).

[tonycpsu]

I recall some audio sync issues with the radio feeds in past years when using mlbviewer -- I'd often have to set the audio offset in mpv to deal with it. Usually wasn't more than a few hundred ms in most cases, but that's not nothing with a sports broadcast.

[mattiasa]

@ChrisVine You're right, I should have said that you shoudl pass -v to mlbplay. I blame too little tea in the morning.

Yes, I use Python 3, but I did test it with 2.7. However, I tested it with a Cubs game which doesn't have Spanish audio, which is what's causing the problems in Python 2.7.

That happens because of the spanish audio. I normally watch Cubs games which doesn't have a spanish track and doesn't get the unicode error. This is a bug with streamlink's hls debug logging.

[mattiasa]

Yes, there seems to be audio sync issues. I haven't figured out if this is streamlink missing some packets or if it's an error in the stream, because the offset seems to change between innings.

There is timecode metadata field which we might be able to use to synchronize with the video. I haven't had time to look into that yet though.

[ChrisVine]

@mattiasa Yes that seems to be the problem. Cubs and Mariners games play OK with the '*' wildcard, but most streams won't play. This occurs with both your logging patch applied (with or without the -v option), and without it.

I also get the audio sync issues on the radio stream that someone else has mentioned. Some radio audio streams can be up to 5 seconds off, whereas sometimes (not often) synchronization is OK. Obviously you can correct for this with the player so it is not the end of the world.

[ChrisVine]

On the radio audio sync issue, I should mention that it is far further off than mlbviewer used to be; and when it was off with mlbviewer, it was usually only in the first inning. After that, mlb.tv would get it synced up correctly. The off sync with mlbstreamer seems to be a streamlink/ffmpeg issue rather than a mlb.tv one, but this is a hypothesis.

[mattiasa]

I've been debugging this and I once again have to conclude that unicode in python drives me nuts. Carthago delenda est.

What's happening is that streamlink gets a unicode string from requests which contains the byte sequence for the ñ character rather than the unicode character. My guess is that requests does this because the webserver doesn't actually say that the content is utf8. The ugly patch I have come up with for now is:

diff --git a/src/streamlink/stream/hls.py b/src/streamlink/stream/hls.py
index 019d3b5..12a17a9 100644
--- a/src/streamlink/stream/hls.py
+++ b/src/streamlink/stream/hls.py
@@ -451,7 +451,7 @@ class HLSStream(HTTPStream):

             if external_audio and FFMPEGMuxer.is_usable(session_):
                 external_audio_msg = ", ".join([
-                    "(language={0}, name={1})".format(x.language, (x.name or "N/A"))
+                    "(language={0}, name={1})".format(x.language, (x.name or "N/A").encode('utf-8'))
                     for x in external_audio
                 ])
                 logger.debug("Using external audio tracks for stream {0} {1}", name_prefix + stream_name,
diff --git a/src/streamlink/stream/hls_playlist.py b/src/streamlink/stream/hls_playlist.py
index bf91652..7798231 100644
--- a/src/streamlink/stream/hls_playlist.py
+++ b/src/streamlink/stream/hls_playlist.py
@@ -205,7 +205,7 @@ class M3U8Parser(object):
             attr = self.parse_tag(line, self.parse_attributes)
             media = Media(self.uri(attr.get("URI")), attr.get("TYPE"),
                           attr.get("GROUP-ID"), attr.get("LANGUAGE"),
-                          attr.get("NAME"),
+                          attr.get("NAME").encode('latin-1').decode('utf-8'),
                           self.parse_bool(attr.get("DEFAULT")),
                           self.parse_bool(attr.get("AUTOSELECT")),
                           self.parse_bool(attr.get("FORCED")),

I'll see if I can come up with something better which works with both python 2 and python 3.

[ChrisVine]

@mattiasa You could just take the view that in order for spanish audio channels or the wild card to work, you need python-3, and document that accordingly. In other words, python-2 users are told that they are restricted to --hls-audio-select 'English Radio,Natural Sound' or a subset thereof. Since I am not interested in the spanish channels (nor for that matter to 'Natural Sound'/ballpark sound only), that's fine by me.

[tonywagner]

FWIW, audio overlay has returned to the MLB.TV web player -- I just saw it in Chrome. Would be interesting to see how that syncs, compared to using mlbstreamer/streamlink/ffmpeg.

[tonywagner]

Looks like streamlink just made a new release (0.12.0), and the latest ffmpeg (4.0) has the required patch applied too.

[tonycpsu]

I just upgraded to streamlink 0.12.0 and ffmpeg 4.0. With streamlink_args: --hls-audio-select '*', I do get four audio tracks, and the audio sync seems to be fine on the couple games I tested with. Things do get a little weird when live-streaming seekable streams in mpv via --force-seekable, as the audio is silent until I initiate a seek operation, but I only normally care about listening to the radio broadcast on games I've already downloaded.

I'm wondering whether I should make --hls-audio-select '*' the default, and let people override it if they only want certain audio tracks. The extra audio tracks add ~12% to the file size of a 360p stream, but only about 6% of a 720p@60fps stream. Long-term I'm thinking it might be nice to support "profiles" to let people configure sets of options for different viewing scenarios, but for now I'm just glad this works. :)

[mattiasa]

I was thinking that the simple solution at least short-term would be to make --hls-audio-select '*' the default unless --hls-audio-select is included in streamlink_args in the configuration file.

I agree that it would be nice to support something like profiles long-term, which could include resolution, bitrate and similar things as well.

[tonycpsu]

Anyone having issues with lower-bandwidth streams and --hls-audio-select *? I'm trying with the 540p stream and it just hangs forever unless I comment out the streamlink_args option.

[Yankees4life]

Interesting thread, tony. Do you plan to make it a feature in which the user can select the audio feed straight from the menu?

[tonycpsu]

@Yankees4life: perhaps, but it would definitely need to be a separate effort, and I want to make sure the general capability to work with different audio tracks is working first.

[ChrisVine]

@tonycpsu: on 540p streams, yes I normally watch the 540p streams and last week some of them did hang the first time I started a game. Ctrl-C and starting again would then usually work. This week they seem to be OK, so its probably a minor MLB.tv issue.

[ChrisVine]

@tonycpsu: by the way, I suggest that you don't have --hls-audio-select '*' as the default until the problems with python-2.7 unicode errors are sorted out, if they can be. Otherwise only make it the default when running under python-3, and document the issue.

[Yankees4life]

Any success getting this to work on cygwin? I added the streamlink settings to the config file and I'm only getting TV audio

[beergeekdotcom]

Any chance of support for audio only games? Was the main way I used the old system.

[ddcote]

I also primarily used the old system for audio games only. At present I cannot see a way to do it either. Obviously this is not a simply problem I presume.

[mattiasa]

The radio audio is available as a separate HLS stream, but the TV audio is only available together with the video stream, so it is technically possible to get a radio-only feed from mlb.tv, but I haven't looked into if that is possible with Streamlink.

[tonycpsu]

As noted in #38 , this project is being rescoped and renamed very soon, so I'm going to close out this issue now. The good news is that I'm happy to report that audio streams will be fully supported by the new version:

[tonywagner]

Out of curiosity, are those "native" radio feeds, or are they somehow extracted from the alternate audio tracks of the video feeds? If they were the latter, I'd worry they might not be available for non-televised games.

[tonycpsu]

@tonywagner: the former. For games with no video (e.g. a lot of preseason games) audio streams are available.

[tonywagner]

@tonycpsu: Very cool! Looking forward to it.