Multiplex Provider (Configuring ABR with external encoder)

[getroot]

This is a beta version and no API is available yet. In the future, once stability is secured and the API is added, it will be released as a full version. You can test it with the master branch or dev Docker image.

Now you can configure ABR with an external encoder using the Multiplex Provider.

The Multiplex Provider's streams pull from other local streams and organize them into their own tracks. It pulls the tracks after encoding, which can be useful if you want to change codecs or adjust the quality once again. Of course, when using a Multiple Provider, in most cases, the track that bypasses the encoder will be pulled and used. And the Multiplex Provider's stream is sent to the Publisher by unconditionally bypassing the encoder.

Limitations

This has the following limitations:

• Timestamps of each Source Streams must be synchronized, which is the role of the external encoder. Otherwise, ABR switching may not be natural or playback may not be smooth. (MUST)

The <UseIncomingTimestamp> feature added to the RTMP allows you to use the Timestamp of the external encoder.
If you do not use this option or receive streams through another provider, OME will adjust the PTS/DTS of the first frame to 0, so each stream must start at the same time. After beta I will add the UseIncomingTimestamp option to other providers as well.

<RTMP>
	<UseIncomingTimestamp>true</UseIncomingTimestamp>
</RTMP>

• The Keyframe Interval and Sample Rate of each Source Stream SHOULD be the same.
• After all configured Source Stream signals are input, the Multiplex Stream is published.

Configuration

To use this, please enable Multiplex Provider in Server.xml.

<Providers>
	...
	<Multiplex>
		<MuxFilesDir>.</MuxFilesDir>
	</Multiplex>

</Providers>

The multiplex provider uses .mux files to generate streams. MuxFilesDir is the path where the .mux files are located, and can be set to an absolute system path or relative to where Server.xml is located.

Mux file format

mux files can be created or deleted while the system is running. This works dynamically.
The mux file has the format below.

<?xml version="1.0" encoding="UTF-8"?>
<Multiplex>
    
    <OutputStream>
        <Name>stream</Name>
    </OutputStream>

    <SourceStreams>
        <SourceStream>
            <Name>tv1</Name>
            <Url>stream://default/app/tv1</Url>
            <TrackMap>
                <Track>
                    <SourceTrackName>bypass_video</SourceTrackName>
                    <NewTrackName>tv1_video</NewTrackName>
                </Track>
                <Track>
                    <SourceTrackName>bypass_audio</SourceTrackName>
                    <NewTrackName>tv1_audio</NewTrackName>
                </Track>
                <Track>
                    <SourceTrackName>opus</SourceTrackName>
                    <NewTrackName>tv1_opus</NewTrackName>
                </Track>
            </TrackMap>
        </SourceStream>
        <SourceStream>
            <Name>tv2</Name>
            <Url>stream://default/app/tv2</Url>

            <TrackMap>
                <Track>
                    <SourceTrackName>bypass_video</SourceTrackName>
                    <NewTrackName>tv2_video</NewTrackName>
                </Track>
                <Track>
                    <SourceTrackName>bypass_audio</SourceTrackName>
                    <NewTrackName>tv2_audio</NewTrackName>
                </Track>
                <Track>
                    <SourceTrackName>opus</SourceTrackName>
                    <NewTrackName>tv2_opus</NewTrackName>
                </Track>
            </TrackMap>

        </SourceStream>
    </SourceStreams>
    
    <Playlists>
        <Playlist>
            <Name>LLHLS ABR</Name>
            <FileName>abr</FileName> <!-- FileName must be unique in the application -->
            
            <Rendition>
                <Name>1080p</Name>
                <Video>tv1_video</Video>
                <Audio>tv1_audio</Audio>
            </Rendition>
            <Rendition>
                <Name>720p</Name>
                <Video>tv2_video</Video>
                <Audio>tv2_audio</Audio>
            </Rendition>
        </Playlist>
    </Playlists>
    
</Multiplex>

OutputStream
This is information about the stream to be newly created. It must be the same as the file name. <stream name>.mux

SourceStreams
Specifies the internal stream to be muxed. You can also load streams from other VHosts or applications in the format stream://<vhost name>/<app name>/<stream name>. Because multiple streams are muxed into one stream, track names may be duplicated. Therefore, it is necessary to change the Track name for each SourceStream through <TrackMap>. <SourceTrackName> is either <OutputProfile><Encodes><Video><Name> or <OutputProfile><Encodes><Audio><Name>.

Playlist
The same format as <OutputProfile> must be used, and the Playlist must be constructed using the newly mapped Track name in SourceStreams' TrackMap. The Playlist configured here exists only in this stream. The Playlist's FileName must be unique throughout the application.

[trapasso]

Great work!

I did a test with 2 simple input streams and I would like to ask a few simple questions:

1. If the path of the directory where the multiplex configuration files reside is not an absolute path, the path of the executable is added. Wouldn't it be better to add the configuration path?

2. Every second a log of this type appears:
   [00-00 00:00:00.000] I [Multiplex:0] ScheduledChannel Provider | multiplex_stream.cpp:167 | Multiplex Channel : default#app/stream-multiplex: Wait for stream stream-00
   Is it possible to disable it?

3. When a multiplexer starts, as I said above with 2 simple streams, the Multiplex thread has a notable CPU consumption.
   PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
   155723 user 20 0 8936728 53260 30636 S 16,6 0,2 3:58.13 Multiplex
   157400 user 20 0 8936728 53260 30636 S 3,7 0,2 0:57.71 Encopus
   157963 user 20 0 8936728 53260 30636 S 3,7 0,2 0:57.32 Encopus
   155674 user 20 0 8936728 53260 30636 S 3,0 0,2 0:41.24 SPRTMP-t1935
   155673 user 20 0 8936728 53260 30636 S 2,7 0,2 0:41.06 SPRTMP-t1935
   155681 user 20 0 8936728 53260 30636 S 2,7 0,2 0:41.08 OutboundWorker
   155683 user 20 0 8936728 53260 30636 S 2,7 0,2 0:33.40 AW-OVT1
   155680 user 20 0 8936728 53260 30636 S 1,7 0,2 0:22.17 OutboundWorker
   155686 user 20 0 8936728 53260 30636 S 1,7 0,2 0:24.05 AW-RTMPPush1
   155678 user 20 0 8936728 53260 30636 S 1,3 0,2 0:16.42 InboundWorker
   155682 user 20 0 8936728 53260 30636 S 1,3 0,2 0:18.91 AW-OVT0
   155679 user 20 0 8936728 53260 30636 S 1,0 0,2 0:16.42 InboundWorker
   155685 user 20 0 8936728 53260 30636 S 1,0 0,2 0:13.09 AW-RTMPPush0
   157962 user 20 0 8936728 53260 30636 S 1,0 0,2 0:10.87 Decaac
   ...
   It's normal?

Many thanks in advance.

[getroot]

Thank you for your various comments!

1. As you said, like the others, I plan to change this to start from the configuration folder when it is a relative path. The scheduled channel was also developed to start from the binary folder, and I plan to change that as well.

2. I think this log is needed for me, but if you don't want this log to appear, you can delete the .mux file or create a mux stream. Otherwise, I'll patch it so you can hide it with the Logger.xml file. Now, due to my mistake, the Tag is ScheduledChannel.

3. Hmm... that may be the case because it has not been optimized in many ways. I will take a look at this before releasing it as a full version.

[wieshka]

How would this work with OriginMapStore?

I am currently in production, at scale using Wowza Streaming Engine, and multiplexing is one of main reasons why I am on WSE as they have very mature implementation for it.

I see this as a must have feature for mostly larger deployments, hence, OriginMapStore functionality needs to be work nicely together with this. Consider architecture, where there is TCP LoadBalancer -> Pool of Origin servers <- OriginMapStore -> Pool of Edge servers.

[getroot]

What problems have you experienced when using Mux Provider and OriginMapStore together?

Mux Provider and OriginMapStore should work well. In fact, OriginMapStore does not care about how the stream was created. The Mux Provider ultimately just gathers multiple streams and outputs one general stream. This means that it will not affect Edge's ability to pull streams from Origin via OriginMapStore.

[maruf-cs]

Suppose I want to multiplex ('mux') 'stream720' and 'stream480' into 'stream'. However, upon inspection, I noticed that 'stream' is not present in Redis, while 'stream720' and 'stream480' are. Is this the intended behavior? If 'stream' is not present, how will the edge know from which origin to pull from? I'm testing it on version 0.16.5.

[getroot]

@maruf-cs
If 'stream' doesn't appear on the redis server, it might be a bug. Could you please post this as an issue?

[maruf-cs]

I've created an issue here #1577 (comment)

[getroot]

This feature was officially released in 0.16.5.
https://airensoft.gitbook.io/ovenmediaengine/live-source/multiplex-channel