Some freezes with static files

[rudnypc]

I'm have a problem with static files, sometimes the screen freezes for while

I believe that probleme is: "force_key_frames"

[m1k1o]

Did you try it with and without video-keyframes enabled? Is it giving different results for you?

[rudnypc]

Yes, I don't have the same problem but ffmpeg generates TS files with different duration

[rudnypc]

I removed -force_key_frames and -segment_times, the video works better

[m1k1o]

But that would evenutally disable seeking functionality. Since length of the segments is not known anymore. And we cannot determine, if segment have been transcoded or if it overlaps with existing.

[rudnypc]

but do you think ffprobe is generating incorrectly? do you have the same problem?

is skip_frame nokey the problem?

[rudnypc]

What do you think about fixed segment time?
Something like
ffmpeg -i <VIDEO> -sn -vf scale=-2:1080 -preset ultrafast -c:v libx264 -b:v 14400k -c:a aac -ac 2 -b:a 320k -force_key_frames 'expr:gte(t,n_forced*4)' -hls_time 4 -hls_playlist_type vod -max_muxing_queue_size 2048 -hls_segment_filename t/720p_%03d.ts t/720p.m3u

[m1k1o]

but do you think ffprobe is generating incorrectly?

When not using keyframes, then they are not generated by ffprobe. Just by an algorithm, so that all chunk times are known. I did not see problems with video I tested with, but there might be different encoding causing different problems. Maybe adding no-scenecut could force keyframes to by only the selected.

What do you think about fixed segment time?

I don't see how it should be different to just telling ffmpeg where to split the video explicitly.

[rudnypc]

ffprobe generated this:

ffprobe -v error -skip_frame nokey -show_entries frame=pkt_pts_time -show_entries format=duration -show_entries stream=duration,width,height -select_streams v -of json MYMOVIE.mkv
{
    "frames": [
        {
            "pkt_pts_time": "0.000000"
        },
        {
            "pkt_pts_time": "10.005000"
        },
        {
            "pkt_pts_time": "20.005000"
        },
        {
            "pkt_pts_time": "28.285000"
        },
        {
            "pkt_pts_time": "33.525000"
        },
        {
            "pkt_pts_time": "37.165000"
        },
        {
            "pkt_pts_time": "47.165000"
        },
        {
            "pkt_pts_time": "57.165000"
        },
        {
            "pkt_pts_time": "67.165000"
        },
        {
            "pkt_pts_time": "76.605000"
        },
        {
            "pkt_pts_time": "78.165000"
        },
        {
            "pkt_pts_time": "79.525000"
        },
        {
            "pkt_pts_time": "80.925000"
        },
        {
            "pkt_pts_time": "88.365000"
        },
        {
            "pkt_pts_time": "89.405000"
        },
        {
            "pkt_pts_time": "99.405000"
        },
        {
            "pkt_pts_time": "102.045000"
        },
        {
            "pkt_pts_time": "103.805000"
        },
        {
            "pkt_pts_time": "106.125000"
        },
        {
            "pkt_pts_time": "107.925000"
        },
        {
            "pkt_pts_time": "109.205000"
        },
        {
            "pkt_pts_time": "110.365000"
        },
        {
            "pkt_pts_time": "115.725000"
        },
        {
            "pkt_pts_time": "121.085000"
        },
        {
            "pkt_pts_time": "123.965000"
        },
        {
            "pkt_pts_time": "131.125000"
        },
        {
            "pkt_pts_time": "132.765000"
        },
        {
            "pkt_pts_time": "134.005000"
        },
        {
            "pkt_pts_time": "137.885000"
        },
        {
            "pkt_pts_time": "140.605000"
        },
        {
            "pkt_pts_time": "144.205000"
        },
        {
            "pkt_pts_time": "148.765000"
        },
        {
            "pkt_pts_time": "150.765000"
        },
        {
            "pkt_pts_time": "156.245000"
        }
        .......................

When I started to play, I found the command:

 2434 root      0:02 ffmpeg -loglevel warning -ss 52.464056 -i media/MYMOVIE.mkv -to 59.459263 -copyts -force_key_frames 55.961659,59.459263 -sn -vf scale=-2:540 -c:v libx264 -preset faster -profile:v high -level:v 4.0 -b:v 1800k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 55.961659,59.459263 -segment_start_number 15 -segment_list_type flat -segment_list pipe:1 transcode/vod-540p-517768070/540p-%05d.ts
 2476 root      0:00 bash
 2483 root      0:00 ps -a
 2484 root      0:00 cat
bash-5.1# ps -a|cat -
PID   USER     TIME  COMMAND
    1 root      0:02 ./go-transcode serve
 2476 root      0:00 bash
 2527 root      0:02 ffmpeg -loglevel warning -ss 69.952074 -i media/MYMOVIE.mkv -to 76.947282 -copyts -force_key_frames 73.449678,76.947282 -sn -vf scale=-2:540 -c:v libx264 -preset faster -profile:v high -level:v 4.0 -b:v 1800k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 73.449678,76.947282 -segment_start_number 20 -segment_list_type flat -segment_list pipe:1 transcode/vod-540p-517768070/540p-%05d.ts
 2569 root      0:00 ps -a
 2570 root      0:00 bash
bash-5.1# ps -a|cat -
PID   USER     TIME  COMMAND
    1 root      0:02 ./go-transcode serve
 2476 root      0:00 bash
 2739 root      0:05 ffmpeg -loglevel warning -ss 108.425715 -i media/MYMOVIE.mkv -to 115.420922 -copyts -force_key_frames 111.923319,115.420922 -sn -vf scale=-2:540 -c:v libx264 -preset faster -profile:v high -level:v 4.0 -b:v 1800k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 111.923319,115.420922 -segment_start_number 31 -segment_list_type flat -segment_list pipe:1 transcode/vod-540p-517768070/540p-%05d.ts

I'm missing something? why the the force_key_frames and segment_times is a little different from ffprobe?

when a I'm looking inside from playlist, it's generated like a fixed time 3.498:

#EXTM3U
#EXT-X-VERSION:4
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:4.75
#EXTINF:3.498, no desc
540p-00001.ts
#EXTINF:3.498, no desc
540p-00002.ts
#EXTINF:3.498, no desc
540p-00003.ts
#EXTINF:3.498, no desc
540p-00004.ts
#EXTINF:3.498, no desc
540p-00005.ts
#EXTINF:3.498, no desc
540p-00006.ts
#EXTINF:3.498, no desc
540p-00007.ts
#EXTINF:3.498, no desc
540p-00008.ts
#EXTINF:3.498, no desc
540p-00009.ts
#EXTINF:3.498, no desc

am i looking and calculating wrong?

I found this document:
https://www.nginx.com/wp-content/uploads/2018/12/NGINX-Conf-2018-slides_Choi-streaming.pdf

the playlist result looks like ffprobe

[m1k1o]

By default, usage of keyframes is disabled: video-keyframes: false.

That means, video is not split by keyframes, but in fixed time 3.498. Only if you enable it in config, that ffprobe will be executed, keyframe times will be generated (and cached if wanted) and used in splitting videos.

go-transcode/hlsvod/manager.go

Lines 122 to 123 in 14e1d91

	// if media has video, use keyframes as reference for segments if allowed so
	if m.metadata.Video != nil && m.metadata.Video.PktPtsTime == nil && m.config.VideoKeyframes {

[rudnypc]

I try to enable the video-keyframes: false tag, but I believe it didn't work.
I try to put inside and outside of vod tag:

  videokeyframes: true
  video-keyframes: true
  VideoKeyframes: true

Only ffprobe running is:
ffprobe -v error -show_format -show_streams -of json MYMOVIE.mkv

[m1k1o]

Thanks for reporting, I missed that one to pass down from config file to the transcoding backend. Could you please try again?

[rudnypc]

Yes, the problem with the tag was fixed.
I found this command:
ffprobe -v error -skip_frame nokey -show_entries frame=pkt_pts_time -show_entries format=duration -show_entries stream=duration,width,height -select_streams v -of json media/MYMOVIE.mp4

The output is something like this:

 {
            "pkt_pts_time": "67.359000",
            "side_data_list": [
                {

                }
            ]
        },
        {
            "pkt_pts_time": "71.780000",
            "side_data_list": [
                {

                }
            ]
        },
        {
            "pkt_pts_time": "76.702000",
            "side_data_list": [
                {

                }
            ]
        },
        {
            "pkt_pts_time": "81.707000",
            "side_data_list": [
                {

                }
            ]
        },
        {
            "pkt_pts_time": "85.002000",
            "side_data_list": [
                {

                }
            ]
        },
        {
            "pkt_pts_time": "89.798000",
            "side_data_list": [
                {

                }
            ]
        }

When I play, I found this command too:
ffmpeg -loglevel warning -ss 79.204500 -i media/MYVIDEO.mkv -to 85.002000 -copyts -force_key_frames 81.707000,85.002000 -sn -vf scale=-2:720 -c:v libx264 -preset faster -profile:v high -level:v 4.0 -b:v 2800k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 81.707000,85.002000 -segment_start_number 27 -segment_list_type flat -segment_list pipe:1 transcode/vod-720p-4273827325/720p-%05d.ts

But I have a question, Should the -SS match with the first force_key_frames?

[rudnypc]

When I open the generated HLS file, I found:
#EXTINF:4.630, no desc
720p-00009.ts

When I download the segment, the time is 2 second:

[m1k1o]

Segments are 3.50s long, but with offset 1.25s. That means, the shortest segment length can be 2.25s, longest can be 4.75s.

go-transcode/hlsvod/manager.go

Lines 60 to 61 in b3af116

	segmentLength: 3.50,
	segmentOffset: 1.25,

An algorithm takes all key frames and tries to use most of them. That means, segment length should average 3.50s, but if there is key frames 1.25s earlier or later in the stream, it would be taken. If there are multiple key frames in this timespan, only one is taken.

go-transcode/hlsvod/utils.go

Lines 11 to 58 in b3af116

	func convertToSegments(rawTimeList []float64, duration time.Duration, segmentLength float64, segmentOffset float64) []float64 {
	durationSec := duration.Seconds()
	minSegmentLength := segmentLength - segmentOffset
	maxSegmentLength := segmentLength + segmentOffset
	timeList := append(rawTimeList, durationSec)
	segmentStartTimes := []float64{0}
	lastTime := float64(0)
	for _, time := range timeList {
	// skip it regardless
	if time-lastTime < minSegmentLength {
	continue
	}
	// use it as-is
	if time-lastTime < maxSegmentLength {
	lastTime = time
	segmentStartTimes = append(segmentStartTimes, lastTime)
	continue
	}
	// count segments between current and last time
	numOfSegmentsNeeded := math.Ceil((time - lastTime) / segmentLength)
	durationOfEach := (time - lastTime) / numOfSegmentsNeeded
	for i := 1; i < int(numOfSegmentsNeeded); i++ {
	lastTime += durationOfEach
	segmentStartTimes = append(segmentStartTimes, lastTime)
	}
	// use time directly instead of setting in the loop so we won't lose accuracy due to float point precision limit
	lastTime = time
	segmentStartTimes = append(segmentStartTimes, lastTime)
	}
	// would be equal to duration unless the skip branch is executed for the last segment, which is fixed below
	if len(segmentStartTimes) > 1 {
	// remove last segment start time
	segmentStartTimes = segmentStartTimes[:len(segmentStartTimes)-1]
	lastSegmentLength := durationSec - lastTime
	if lastSegmentLength > maxSegmentLength {
	segmentStartTimes = append(segmentStartTimes, durationSec-lastSegmentLength/2)
	}
	}
	return append(segmentStartTimes, durationSec)
	}

Minimum buffer length is 3 segments, and maximum is 5. That means, maximum 5 segments are transcoded at the same time and minimum 3 of them are always available. That means, initially there are 5 segments transcoded, and after 2 segments, another 5 segments are transcoded, and offset is controlled by -ss. And so on, throughout the stream. When playing head moves because of seeking, next segments are being transcoded until we reach the end of the stream.

go-transcode/hlsvod/manager.go

Lines 62 to 63 in b3af116

	segmentBufferMin: 3,
	segmentBufferMax: 5,

[rudnypc]

Ok, I changed some values to work better:
1º I'm using the ffprobe
2º I changed the segment value to:

                segmentLength:    29.50,
                segmentOffset:    29.50,
                segmentBufferMin: 1,
                segmentBufferMax: 50,

3º I changed the playlist generator to ignore the first result (0.00000)

         // playlist segments
        for i := 2; i < len(m.breakpoints); i++ {
                playlist = append(playlist,
                        fmt.Sprintf("#EXTINF:%.3f, no desc", m.breakpoints[i]-m.breakpoints[i-1]),
                        m.getSegmentName(i-1),
                )
        }

Now it's working much better, thanks

[m1k1o]

I changed the playlist generator to ignore the first result (0.00000)

I applied this fix in recent commit.

Thinking about passing other values as configs.

[rudnypc]

Fixed, thank you \o/

[rudnypc]

sorry, but your fix to segment is making the ffmpeg to start after sometime, the skip is only in the segment generator.
ffmpeg -loglevel warning -ss 0.208333 -i MYVIDEO.......

[m1k1o]

Hmm, so that mean we miss first seconds of the stream? It should be removed only from playlist? I'll look into it today again. Thanks for letting me know.

[rudnypc]

yes, the zero number should be removed only from playlist segment.

[m1k1o]

Fixed.

[rudnypc]

The start value is correct, but the ts is wrong, the segment should start with 00001. The calculation time of the 720p-00001.ts is from 0 and the next keyframe.

#EXTM3U
#EXT-X-VERSION:4
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:4.75
#EXTINF:2.502, no desc
720p-00002.ts
#EXTINF:2.586, no desc
720p-00003.ts
#EXTINF:3.003, no desc
720p-00004.ts
#EXTINF:2.502, no desc

[rudnypc]

because this I added "-1" in the segment name:
m.getSegmentName(i-1),

I don't know if this will create a problem without keyframe.

[m1k1o]

I didn't notice that, sorry, my mistake!

[rudnypc]

The profile-0001 segment is a m3u8

[rudnypc]

[m1k1o]

It looks like it returns manifest only if the transcoding did not start yet. When i request second segment, then even first one works.

UPDATE: Even after re-requesting first manifest it returns it correctly.

[m1k1o]

Usecase of requesting segments before manifests was not considered, that is why it fails. The initialization request was expected to be manifest. That means, the first request to any segment ot playlist returns playlist. But this is unexpected behaviour.