Live Streaming: Stop Response Task when user cancels the connection

[Connum]

Hi there,

first of all, I'm coming from a PHP and JavaScript background, so this might be very simple, but I'm stuck at this point. I'm trying to write a module for live streaming of MPEG Transport Streams via FFMPEG. I finally got this working today in an asynchronous way, so that I can have multiple simultaneous connections. I oriented myself by the code of the StaticFilesModule, and made some adaptions because I don't know the content length and I don't need to support seeking via byte range headers.

It works well, I can view the stream via VLC for example, or download it via a browser. However, as I can see in the console log, the request never ends. I can see in the Task Manager that ffmpeg keeps running in the background and keeps using resources. I would like to stop the request from being processed / the response from being continously generated and the ffmpeg task from running as soon as the connection is being closed, that is the stream is stopped in a player or the download is aborted in a browser.

As far as I can see, this is not implemented for static files either, so a very large download would keep on being processed even after the connection was stopped. So I'm not sure whether this is possible with EmbedIO without changes to the code, but it would definitely be something to consider for (large) static files as well to save resources.

Anyway, here's my current code:

namespace TSserver
{
    using Unosquare.Swan;
    using Unosquare.Swan.Formatters;
    using System;
    using System.Collections.Generic;
    using System.IO;
    using System.Threading;
    using System.Threading.Tasks;
#if NET46
    using System.Net;
#else
    using Unosquare.Net;
    using Unosquare.Labs.EmbedIO;
    using Unosquare.Labs.EmbedIO.Constants;
    using System.Diagnostics;
#endif

    /// <summary>
    /// TSserver Module
    /// </summary>
    public class TSserverModule : WebModuleBase
    {
        /// <summary>
        /// The chunk size for sending files
        /// </summary>
        private const int ChunkSize = 4096;

        /// <summary>
        /// Initializes a new instance of the <see cref="TSserverModule"/> class.
        /// </summary>
        /// <param name="basePath">The base path.</param>
        /// <param name="jsonPath">The json path.</param>
        public TSserverModule()
        {
            AddHandler(ModuleMap.AnyPath, HttpVerbs.Head, (context, ct) => HandleGet(context, ct, false));
            AddHandler(ModuleMap.AnyPath, HttpVerbs.Get, (context, ct) => HandleGet(context, ct));
        }
        
        /// <summary>
        /// Gets the Module's name
        /// </summary>
        public override string Name => nameof(TSserverModule).Humanize();

        private async Task<bool> HandleGet(HttpListenerContext context, CancellationToken ct, bool sendBuffer = true)
        {
            Stream buffer = null;

            try {
                var ffmpeg = new Process
                {
                    StartInfo = new ProcessStartInfo
                    {
                        FileName = "ffmpeg.exe",
                        Arguments = "-re -loop 1 -i \"./default.png\" -i \"./jeopardy.mp3\" -c:v libx264 -tune stillimage -r 25 -vcodec mpeg2video -profile:v 4 -bf 2 -b:v 4000k -maxrate:v 5000k -acodec mp2 -ac 2 -ab 128k -ar 48000 -f mpegts -mpegts_original_network_id 1 -mpegts_transport_stream_id 1 -mpegts_service_id 1 -mpegts_pmt_start_pid 4096 -streamid 0:289 -streamid 1:337 -metadata service_provider=\"MYCALL\" -metadata service_name=\"My Station ID\" -y pipe:1",
                        UseShellExecute = false,
                        RedirectStandardOutput = true,
                        CreateNoWindow = true
                    }
                };

                ffmpeg.Start();

                buffer = ffmpeg.StandardOutput.BaseStream as FileStream;
            
                SetHeaders(context.Response);
            
                if (sendBuffer == false)
                {
                    //context.Response.ContentLength64 = buffer?.Length ?? fileSize;
                    return true;
                }
                
                // If buffer is null something is really wrong
                if (buffer == null)
                {
                    return false;
                }
            
                await WriteToOutputStream(context.Response, buffer, 0, ct);
            }
            catch (HttpListenerException)
            {
                // Connection error, nothing else to do
            }
            finally
            {
                buffer?.Dispose();
            }

            return true;
        }

        private void SetHeaders(HttpListenerResponse response)
        {
            response.ContentType = "video/mp2t";
            response.SendChunked = true;
            //response.AddHeader(Headers.AcceptRanges, "bytes");
            response.AddHeader(Headers.CacheControl, "no-cache");
            response.AddHeader("Pragma", "no-cache");
            response.AddHeader("Expires", "0");
            response.OutputStream.Flush();
        }

        private static async Task WriteToOutputStream(
            HttpListenerResponse response,
            Stream buffer,
            int lowerByteIndex,
            CancellationToken ct)
        {
            var streamBuffer = new byte[ChunkSize];
            var sendData = 0;
            var readBufferSize = ChunkSize;
            
            // I used CopyToAsync instead of the ReadAsync and WriteAsync for simplicity:

            await buffer.CopyToAsync(response.OutputStream, 4096, ct);
            response.OutputStream.Flush();

            //while (true)
            //{
            //    //if (sendData + ChunkSize > response.ContentLength64) readBufferSize = (int)(response.ContentLength64 - sendData);

            //    //buffer.Seek(lowerByteIndex + sendData, SeekOrigin.Begin);
            //    var read = await buffer.ReadAsync(streamBuffer, 0, readBufferSize, ct);

            //    if (read == 0) break;

            //    sendData += read;
            //    await response.OutputStream.WriteAsync(streamBuffer, 0, readBufferSize, ct);

            //    response.OutputStream.Flush();
            //}
        }

    }
}

And this is the console output after I have stopped the download:

16:27:02.712 INF >> [WebServer] Web server prefix 'http://localhost:9696/' added.
16:27:02.719 INF >> [WebServer] Finished Loading Web Server.
16:27:02.744 INF >> [WebServer] Started HTTP Listener
16:27:02.944 DBG >> [WebServer] Start of Request 10d04e99 - Source ::1:64902 - GET: /
16:27:02.949 DBG >> [WebServer] TStunnel Module::TStunnelModule.<.ctor>b__1_1

As you can see, the request never stops being processed, as opposed to serving a static file or in this case making the stream a finite length by adding -t 1? to the ffmpeg arguments, which results in:

16:28:50.491 DBG >> [WebServer] Start of Request b9cd23a5 - Source ::1:65008 - GET: /
16:28:50.496 DBG >> [WebServer] TStunnel Module::TStunnelModule.<.ctor>b__1_1
16:28:52.108 TRC >> [WebServer] Result: True
16:28:52.114 DBG >> [WebServer] End of Request b9cd23a5

How do I approach this? Do I need to handle the CancellationToken in a way?

Thanks in advance!

[geoperez]

Hi,

Checking your code I can't see where you stop ffmpeg, or ffmpeg closes itself?

[Connum]

Hi,

well, that's what I'm trying to achieve... As it's a live stream, it's not supposed to be stopped in any other circumstance except that the client closes the connection. How can I catch that condition and then stop the ffmpeg process?

[geoperez]

Yeah, but why are you starting a ffmpeg each time a client connects? This is the expected behavior?

I would recommend leave ffmpeg outside the HTTP connection and just use the StaticFileModule to serve a m3u8 file for HLS (https://en.wikipedia.org/wiki/HTTP_Live_Streaming)

[Connum]

I might later add some kind of caching, so that only one ffmpeg instance per channel will be used to deliver to the clients, if there are several clients requesting the same channel. But in general, yes, it's intended that ffmpeg is only operating when a client is requesting a channel.

There will be several channels (about 10, most in 720p) that will be re-streamed, so I don't want to have several ffmpeg instances running 24/7 all days of the year reading and writing data all the time, because that would be overkill for the internet bandwidth and my HDDs. :)

There must surely be a way to detect whether the response.OutputStream is still "alive"?

[geoperez]

Any idea @mariodivece ?

[mariodivece]

There are only 2 correct approaches here:

1. Create a RAM drive from which you can serve the video fragments and serve them using embedio -- use the static files module for this. I have done this and works very well. This requires an instance of ffmpeg per channel but it does not stress the hard drives as all data is temporarily stored in RAM.
2. Perform custom encoding using the FFMPEG APIs directly using a wrapper (requires expertise in video encoding). https://github.com/Ruslan-B/FFmpeg.AutoGen

[Connum]

Thanks for your reply! Do you have an example for the RAM drive solution? Is it created with C# or is it kind of a virtual drive in Windows that I then use to read and write?

The problem still is the constant download of 10 Streams... However, I have thought about the following solution to that: On each request to the m3u8/TS files, a timestamp associated to the ffmpeg process is updated. In the background, every few seconds it is checked whether the timestamp is older than x seconds, and if so, the process is being stopped. On the next request to that channel, the process is started again.

This should work, but it still feels like much overhead for all of this... I wonder, how PHP handles this? A PHP script stops as soon as a client closes the browser window, unless you explicitly set ignore_user_abort for the script to keep running in the background until its finished.

Shouldn't it be possible to have a timer constantly check if the client is still there? I'd just need a push in the right direction what I could check to see if the response is still being sent to the client. response.OutputStream seems to just keep existing even though the client has disconnected.

[mariodivece]

For Ram Disk, how about this: https://stackoverflow.com/a/14728377/1357191
There are hundreds of RAM disk solitions out there.

On the streaming side: why do you care so much about whether or not a user is connected? Don't kill the ffmpeg processes creating the m3u8 and chunk files. Just let them run continuously and serve the m3u8 and ts chunks with embedio. Killing the ffmpeg process and restarting it when a user connects won't really work (will fail half the time).

[Connum]

Hi,
it's because that would be pretty heavy on my internet bandwidth. That server is supposed to run on a local PC that runs 24/7, so my bandwidth would have to constantly consume several HD streams at once. That's why I want to close the ffmpeg process when it's not currently needed.

[geoperez]

Hi @Connum , can I close this issue?

[Connum]

Hi @geoperez,

well, I implemented the whole functionality with PHP in the end, where the ffmpeg process gets terminated as soon as the client aborts the connection. I would still be interested in solving this with embedio for perfocmance reasons though, but I didn't couldn't work it out with my poor C# skills. It musst be possible in some way!

[mariodivece]

Closing this one.

[MajidSafari]

@Connum Hi , are you solve this problem ?

[Connum]

No, unfortunately not. I ended up not using embedio at all.

[MajidSafari]

so ,

can help me for live stream , use other code !?

[geoperez]

@rdeago any idea how to pass the interruption of the network connection to the WebModule?

[Genteure]

Maybe CancellationToken?
Create a new CancellationTokenSource per connection, call Cancel when connection is closed.
CancellationTokenSource.CreateLinkedTokenSource can be used to "merge" the global CancellationToken and the per connection token, so it's not a breaking change.

[rdeago]

EmbedIO always sets the IgnoreWriteExceptions property to true on a HTTP listener, which could explain why writing on the response stream never fails, as it should do upon client disconnection. @geoperez is there any particular reason for this?

[geoperez]

No particular reason. We can add this as part of the WebServer options.

[rdeago]

Please be sure that client disconnection just cause a DBG-level log. No reason to log a stack trace, as it may happen at any moment and is not caused by anything in server-side code.

The default value in v2 should probably be true for compatibility, but in v3 I'd make it false because it looks more predictable.