Memory leak

[pubpy2015]

Hi,

With 1 video: 3840x2160, 15 fps, duration 15 minute.
First time open this video: the memory consumption from 218 to 228 MB. After video finished, memory still 228 MB
Second time: memory increase from 228-> 239 MB,...
....

[SuRGeoNix]

Hi, thanks for submitting this. I knew that issue and I tried once in the past to fix it with no success. I'm afraid is from FFmpeg linked libraries and it's not easy to identify it but I will give another try in the next update.
The good thing, as far as I remember, is that is not 11MB each time you open it... GC does some job i guess. In any case, I should fix this.

[pubpy2015]

Hi,

Today i am trying to debug but no luck!
Maybe you need to rewrite MediaDecoder.cs because now it's very complicate.

[pubpy2015]

Add a method Close and call before Open() to release memory of previous file:
public void Close()
{
Pause();
video?.Dispose();
video = null;
audio?.Dispose();
audio = null;
if (swsCtx != null)
{
sws_freeContext(swsCtx);
swsCtx = null;
}
if (vCodecCtx != null)
{
avcodec_close(vCodecCtx);
vCodecCtx = null;
}
if (aCodecCtx != null)
{
avcodec_close(aCodecCtx);
aCodecCtx = null;
}
}

[SuRGeoNix]

I would bet that is the codecs (probably the video codec), but it's not that easy to free an ffmpeg resource. It will cause stack overflows and other mem leaks and it will crash the whole application.

And yes a code clean up is required. But, I left like that after a created DecoderContext until I will finish it and tested :)

I will close this for now as it is not critical one. And we re-open in the future if required!

[pubpy2015]

Hi,

Seem the cause of memory leak here:

hasMoreFrames = true;

[pubpy2015]

After longtime testing, i confirm that: by remove "hasMoreFrames = true" the memory does not increase by time.

[SuRGeoNix]

That was a very nice catch!

Just found some time and tested. I can confirm that it was the one was causing the mem leak. Not sure yet what I messed up there, probably misused with AVERROR(EAGAIN) combination.

[SuRGeoNix]

Just also confirmed that by removing it will cause issues on many videos. This is how decoding is suppose to work. Just need to find (probably the pkt) what I'm missing to dispose. I thought it is required only for draining but I was wrong.

[pubpy2015]

Hi,

Please try this:

public int GetNextFrame(out AVMediaType mType)
{
int ret = 0;
mType = AVMEDIA_TYPE_UNKNOWN;

            if (!drainMode)
            {
                av_packet_unref(pkt);
                ret = av_read_frame(fmtCtx, pkt);
                // 0 if OK, 
                // < 0 on error or end of file
                if (ret == AVERROR_EOF)
                {
                    av_packet_unref(pkt);
                    if (type != AVMEDIA_TYPE_VIDEO) return AVERROR_EOF;

                    ret = avcodec_send_packet(dec.vCodecCtx, null);
                    if (ret != 0 && ret != AVERROR(EAGAIN)) return ret;

                    drainMode = true;
                    return GetNextFrame(out mType);
                }
                else if (ret != 0)
                {
                    av_packet_unref(pkt);
                    return ret;
                }

                if (!activeStreamIds.Contains(pkt->stream_index))
                {
                    av_packet_unref(pkt);
                    return GetNextFrame(out mType);
                }
                if (fmtCtx->streams[pkt->stream_index]->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) 
                {
                    av_packet_unref(pkt);
                    mType = AVMEDIA_TYPE_SUBTITLE; 
                    return ret; 
                }

                ret = avcodec_send_packet(fmtCtx->streams[pkt->stream_index]->codec, pkt);
                // 0 on success, otherwise negative error code: 
                // AVERROR(EAGAIN): input is not accepted in the current state - user must read output with avcodec_receive_frame() 
                // (once all output is read, the packet should be resent, and the call will not fail with EAGAIN). 
                // AVERROR_EOF: the decoder has been flushed, and no new packets can be sent to it (also returned if more than 1 flush packet is sent) 
                // AVERROR(EINVAL): codec not opened, it is an encoder, or requires flush 
                // AVERROR(ENOMEM): failed to add packet to internal queue, or similar other errors: legitimate decoding errors

                //// Allow INVALID DATA?
                //if (ret == AVERROR_INVALIDDATA) 
                //{
                //    av_packet_unref(pkt);
                //    errors++; 
                //    Log("Stream " + pkt->stream_index.ToString() + " - Error[" + ret.ToString("D4") + "], Msg: " + ErrorCodeToMsg(ret)); 
                //    return 0; 
                //}

                if (ret != 0 && ret != AVERROR(EAGAIN))
                {
                    av_packet_unref(pkt);
                    return ret;
                } 
            }

            drainMode = false;
            av_frame_unref(frame);
            frame = av_frame_alloc();
            ret = avcodec_receive_frame(fmtCtx->streams[pkt->stream_index]->codec, frame);
            // 0: success, 
            // a frame was returned AVERROR(EAGAIN): output is not available in this state - user must try to send new input 
            // AVERROR_EOF: the decoder has been fully flushed, and there will be no more output frames 
            // AVERROR(EINVAL): codec not opened, or it is an encoder other negative values: legitimate decoding errors
            if (ret == 0)
            {
                if (type == AVMEDIA_TYPE_VIDEO && pkt->stream_index == dec.vStream->index)
                {
                    mType = AVMEDIA_TYPE_VIDEO;

                    // Couldn't find a way to do it directly [libavutil/hwcontext_d3d11va.c -> d3d11va_frames_init()] - Without it, first frame (after flush buffers) will be YUV Green screen (still happens few times) - maybe will retry with CreateQuery/GetData event
                    if (!hwFramesInit || dec.player.vFrames.Count == 0)
                    {
                        // In case GPU fails to alocate FFmpeg decoding texture
                        if (dec.hwAccelSuccess && frame->hw_frames_ctx == null) dec.hwAccelSuccess = false;
                        if (dec.hwAccelSuccess) Thread.Sleep(40);
                        hwFramesInit = true;
                    }
                }
                else if (pkt->stream_index == dec.aStream->index)
                    mType = AVMEDIA_TYPE_AUDIO;

                return ret;
            }
            else if (ret == AVERROR(EAGAIN))
            {
                //return GetNextFrame(out mType);
            }

            return ret;
        }

[SuRGeoNix]

To actually test this issue the easiest way to identify that it does not work is to test it with an AV1 codec. If you cant find one easy, try this -> https://r7---sn-vuxbavcx-5uid.googlevideo.com/videoplayback?expire=1606917827&ei=Y0rHX-TzMp3W1wKvsbaQAw&ip=2a02%3A587%3Ac4a0%3A3f00%3A3863%3Afdd2%3Abc5%3Aa0a4&id=o-AC0jlEH78FnMfMJz6UdDOceJpLNUfkDBDKxeEg6Mcriy&itag=399&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C271%2C278%2C313%2C394%2C395%2C396%2C397%2C398%2C399%2C400%2C401&source=youtube&requiressl=yes&mh=Vy&mm=31%2C29&mn=sn-vuxbavcx-5uid%2Csn-4g5e6ne6&ms=au%2Crdu&mv=m&mvi=7&pl=34&initcwndbps=662500&vprv=1&mime=video%2Fmp4&ns=jMNPN2Suc_66t-OngC4ZoDEF&gir=yes&clen=77672503&dur=664.958&lmt=1604939197293100&mt=1606895803&fvip=6&keepalive=yes&c=WEB&txp=5531432&n=mAtUvJl1A6QrqHDpE&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cns%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIhAJToASpD7Xn6bNI1LmtkEspCcG65noWd9t9W6DRgQhkbAiAv-0HFtjwe38iVgFd5TRvlTrVdsp9e2hoZiWiUvHhFmw%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRgIhAOp6jHsg_h5d5qLqRIXLeVgrpp2t1tkU3bBVQuzJ5NlGAiEA3ZQm4dTxdRSVCrejbzIGb-tyIECP8nbrhTgToevYKkA%3D&ratebypass=yes

If expired just extract again with youtube-dl
youtube-dl -f 399 -g https://www.youtube.com/watch?v=TM1hP6yAtII

I'm preparing the 2.4 now (adding better web streaming support and history/recent with sessions for fast resume on torrent streaming). I will give another try to fix it before that but it's not a priority, I guess. Thanks, for trying tho.

[SuRGeoNix]

OK. Finally, I think i fixed that the right way.

Just replace the line at the end of the function

if (ret == AVERROR(EAGAIN)) return GetNextFrame(out mType);

with

if (ret == AVERROR(EAGAIN))
{
	fixed (AVFrame** ptr = &frame) av_frame_free(ptr);
	return GetNextFrame(out mType);
}