Set bitrate limits for transcoding (fixes #638) #1135

Nutomic · 2018-09-28T07:39:10Z

I haven't tested this yet, because I already spent way too much time researching all the different recommendations for video bitrates. I haven't even tested this yet...

Turns out this topic is extremely complicated, probably enough for someone to write a computer science master thesis on optimizing it.

TODO:

write tests
verify quality of output videos
write a script to transcode old, high bitrate videos with these bitrate values
make optimize-old-videos.ts optimize owned videos only
make optimize-old-videos.ts regenerate the associated torrents (and other metadata in the db?)
write documentation for optimize-old-videos.ts

Nutomic · 2018-09-28T07:44:14Z

I also found out about Adaptive bitrate streaming. Does Peertube implement this already?

rigelk · 2018-09-28T08:07:41Z

We use our own implementation of adaptative bitrate streaming. Since we don't use HTTP directly but webtorrent, that's more practical than adapting existing ABS implementations.

Nutomic · 2018-09-28T19:17:11Z

Hmm, for some reason transcoding is not working on my local dev instance. ffmpeg version is n4.0.2 (on Manjaro Linux), is that too new?

Also, I noticed that there aren't any tests for transcoding or other helpers yet. And I don't understand Typescript well enough to write that from scratch.

rigelk · 2018-09-28T21:51:14Z

@Nutomic hopefully we have tests https://github.com/Chocobozzz/PeerTube/blob/develop/server/tests/api/videos/video-transcoder.ts (and we spent some decent time on them ^^)

Your version of ffmpeg should be fine. Newer versions of ffmpeg have never introduced problems, but rather improved performance. I myself run the same version if I recall.

rigelk · 2018-09-28T23:21:12Z

server/helpers/ffmpeg-utils.ts

+
+    // Force a keyframe every 2 seconds, and on scene cuts.
+    // https://superuser.com/a/1098329
+    command.outputOptions('-force_key_frames "expr:gte(t,n_forced*2)"')


This option is probably the one causing the error. Try without it first.

Actually running your code without that part passes the test. Since it's out of scope of the current issue at stake, I suggest you remove it so that we can merge the rest. We can always deal with that option in another PR once you have made it work.

Yeah I noticed that too. No idea why it is failing, as the same parameter works fine on the command line.

If I might suggest using this diff: https://copycat.drycat.fr/?fc042d85910bda21#oPHiEhl9PSjfLLH3z2H6H31amvd12GGpA7kvX8pr6y8=

It probably clashes with some other option in use in PeerTube ; but to properly debug it from within our library to ffmpeg (fluent-ffmpeg), we have yet to attach a logger.

Nutomic · 2018-09-29T00:44:22Z

Wrote a test (didn't find that file before), added a script to optimize old videos, and moved the keyframe stuff into a new issue.

Transcoding is still not working if I upload a video into the dev instance, but it works fine from the script. Transcoding is enabled in Peertube settings. Maybe it will help if I reset the database etc.

package.json

rigelk · 2018-09-29T01:24:26Z

Okay - that looks overall good - great work! I will do a proper review tomorrow (it's quite late here)

If in the meantime you still have the error after upload, that needs to be investigated, because that means our test suite is not covering an important case.

Nutomic · 2018-09-29T01:29:39Z

Thanks :)

I'm not getting any upload error, Peertube just shows "waiting for transcoding", but doesn't start FFmpeg. It might just be a problem with my setup.

Nutomic · 2018-09-29T02:11:41Z

I just realized that the optimize videos script still has some problems:

redundant videos from other servers from other servers should be ignored
torrents need to be regenerated after running the script

That means simply going over the files won't be enough, and we have to get the info from the database somehow.

rigelk · 2018-09-29T02:47:32Z

Both very good points. The prune script makes the difference between "owned" and other videos that come from the redundancy, so at least you have an example of that. Torrents are generated in

PeerTube/server/models/video/video.ts

Line 1355 in 8c2b975

async createTorrentAndSetInfoHash (videoFile: VideoFileModel) {

Nutomic · 2018-09-29T05:38:10Z

Thanks! It seems like I can get the VideoModel with VideoModel.loadByUUIDWithFile(uuid), and then call createTorrentAndSetInfoHash() on it. But how do I get the VideoFileModel for that? It would be really good if there were some comments in the video.ts file (which has 1500 lines!).

scripts/optimize-old-videos.ts

rigelk · 2018-09-29T09:35:07Z

Alright. From within a failing transcoding job's logger instance, I get the following stderr (stdout is empty):

Stream mapping:
  Stream #0:0 -> #0:0 (vp8 (native) -> h264 (libx264))  
  Stream #0:1 -> #0:1 (vorbis (native) -> aac (native))
[aac @ 0x562aaaf0dec0] [Eval @ 0x7ffc31f4a640] Undefined constant or missing '(' in 'undefined'
[aac @ 0x562aaaf0dec0] Unable to parse option value \"undefined\"
[aac @ 0x562aaaf0dec0] Error setting option maxrate to value undefined.
Error initializing output stream 0:1 -- Error while opening encoder for output stream #0:1 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!"

The input file is a .webm from the tests. Most likely this is due to you getting metadat.format.bit_rate, which might return undefined for no apparent reason… 😞

Nutomic · 2018-09-29T18:38:38Z

Okay that one should be fixed. Not sure if tests are working now because they are timing out for me, but that might just be because of my slow laptop.

I still need help figuring out how to recreate the torrent (and update the file size?). Also, are the checks in optimize-old-videos.ts enough to make sure it doesn't touch any files that are currently being uploaded or transcoded?

rigelk · 2018-09-29T18:39:54Z

@Nutomic a VideoModel holds an array of the files related to it (the different versions of a same video, each with a different resolution/fps pair). It's accessible as VideoModel.VideoFiles: VideoFileModel[]. I guess we don't document it because it follows the syntax defined by sequelize (which has a rich documentation), but that's still difficult for me ;)

Nutomic · 2018-09-29T23:59:25Z

Thanks, I think I figured it out for the most part. This is all pretty complicated because I never touched Nodejs before. So I have to look up every little thing that I wouldn't even have to think about in Java...

There are still some problems:

It looks like I really broke the transcoding somehow, that's why the tests are failing. But I have no clue what could be the problem, and I can't see any errors reported.
optimize-old-videos.ts:48 throws an error, for which I couldn't find any solution online: Error: Model not initialized: "ApplicationModel" needs to be added to a Sequelize instance before "findOne" can be called.
Repeating from my last comment in case you missed it: are the checks in optimize-old-videos.ts enough to make sure it doesn't touch any files that are currently being uploaded or transcoded?

Nutomic · 2018-09-30T00:29:53Z

shared/models/videos/video-resolution.enum.ts

+      // quality according to Google Live Encoder: 3000 - 6000 Kbps
+      // Quality according to YouTube Video Info: 3277 Kbps
+      return 3300 * 1000
+    }


I noticed that Peertube supports 60 fps video, so I set a higher bitrate for that. Unfortunately I can't use the VIDEO_TRANSCODING_FPS constant from server/initializers because that throws an error when I import it here (probably because this is in the shared/ code).

rigelk · 2018-09-30T09:37:38Z

@Nutomic I leave aside the questions I can't answer as of now, but don't worry I didn't forget about them. I'll let @Chocobozzz answer them next week.

server/tests/api/videos/video-transcoder.ts

server/helpers/ffmpeg-utils.ts

shared/models/videos/video-resolution.enum.ts

rigelk

the file …is 3.5MBytes 😅

Nutomic · 2018-09-30T21:14:59Z

Okay I can probably just convert one of the existing files.

Nutomic · 2018-09-30T21:23:16Z

So unless I just put a 240p or 360p video, the file will always be pretty large. Or should I convert a video on the fly inside the test?

rigelk · 2018-09-30T21:31:46Z

@Nutomic you could generate one on the fly with ffmpeg for the test, yes.

Nutomic · 2018-10-08T06:06:44Z

Just rebased on develop. Tests are still failing, but that seems completely unrelated to my changes...

Chocobozzz · 2018-10-08T14:25:47Z

Thanks @Nutomic

Chocobozzz · 2018-10-08T15:47:12Z

server/helpers/ffmpeg-utils.ts

+    // https://slhck.info/video/2017/03/01/rate-control.html
+    // https://trac.ffmpeg.org/wiki/Limiting%20the%20output%20bitrate
+    const targetBitrate = getTargetBitrate(options.resolution, fps, VIDEO_TRANSCODING_FPS)
+    command.outputOptions([`-b:v ${ targetBitrate }`, `-maxrate ${ targetBitrate }`, `-bufsize ${ targetBitrate * 2 }`])


One small question: what happens if the input file has an average bitrate < targetBitrate?

@Nutomic won't the -b:v ${ targetBitrate } increase the bitrate of the video if it is lower than the targetBitrate? I feel like we should have just kept maxrate and bufsize to benefit from ffmpeg's automatic tuning.

Damn you're right. I just checked and it works fine if you remove the -b:v parameter (tested with low bitrate, high bitrate and very high bitrate like in the unit test). I was having some bug during development, where FFmpeg generated videos with a too high bitrate for no reason, but that is gone now.

Can one of you just make a commit to remove that parameter?

Sure :) no worries, it's my fault for not double checking that

Nutomic · 2018-10-08T16:52:27Z

Those are maximum values, so if possible, it transcodes with a lower bitrate. See my tests here.

Thanks for all the help figuring this stuff out, @rigelk @Chocobozzz :)

rigelk · 2018-10-08T19:21:29Z

Should be fixed in e1d7b98

Chocobozzz · 2018-10-09T06:34:23Z

Perfect thanks @Nutomic & @rigelk

rigelk self-assigned this Sep 28, 2018

rigelk added the Status: Requires Tests Either requires manual test, or writing tests, or both label Sep 28, 2018

rigelk reviewed Sep 28, 2018

View reviewed changes

Nutomic mentioned this pull request Sep 29, 2018

Set Keyframe interval for FFmpeg #1147

Closed

Nutomic force-pushed the video-bitrate branch from 3ecb6d2 to e025ca9 Compare September 29, 2018 00:59

rigelk reviewed Sep 29, 2018

View reviewed changes

package.json Show resolved Hide resolved

Nutomic commented Sep 29, 2018

View reviewed changes

scripts/optimize-old-videos.ts Outdated Show resolved Hide resolved

Nutomic commented Sep 30, 2018

View reviewed changes

rigelk reviewed Sep 30, 2018

View reviewed changes

server/tests/api/videos/video-transcoder.ts Outdated Show resolved Hide resolved

rigelk reviewed Sep 30, 2018

View reviewed changes

server/helpers/ffmpeg-utils.ts Outdated Show resolved Hide resolved

rigelk reviewed Sep 30, 2018

View reviewed changes

shared/models/videos/video-resolution.enum.ts Outdated Show resolved Hide resolved

rigelk requested changes Sep 30, 2018

View reviewed changes

Nutomic added 21 commits October 7, 2018 23:38

fix tests?

b400b0a

update torrent and file size, some more fixes/improvements

805b8ca

use higher bitrate for high fps video, adjust bitrates

f5e64e6

add test video

56fb33e

don't throw error if resolution is undefined

3c6fee6

generate test fixture on the fly

d2109ff

use random noise video for bitrate test, add promise

a311dab

shorten test video to avoid timeout

3e6773b

use existing function to optimize video

221f562

various fixes

62bbdfb

increase test timeout

16dc4d2

limit test fixture size, add link

335dbaf

test fixes

40f96d1

add await

762cb54

more test fixes, add -b:v parameter

a11e5e1

replace ffmpeg wiki link

9334721

fix ffmpeg params

208624b

fix unit test

68a29af

add test fixture to .gitgnore

ec21c9a

add video transcoding fps model

8c74e41

add missing file

0583b4b

Nutomic force-pushed the video-bitrate branch from 4a30647 to 0583b4b Compare October 8, 2018 04:44

Chocobozzz merged commit edb4ffc into Chocobozzz:develop Oct 8, 2018

Chocobozzz reviewed Oct 8, 2018

View reviewed changes

Nutomic mentioned this pull request Oct 9, 2018

Scale bitrate linearly with FPS #1233

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set bitrate limits for transcoding (fixes #638) #1135

Set bitrate limits for transcoding (fixes #638) #1135

Nutomic commented Sep 28, 2018 •

edited

Loading

Nutomic commented Sep 28, 2018

rigelk commented Sep 28, 2018 •

edited

Loading

Nutomic commented Sep 28, 2018 •

edited

Loading

rigelk commented Sep 28, 2018 •

edited

Loading

rigelk Sep 28, 2018

rigelk Sep 28, 2018

Nutomic Sep 28, 2018

rigelk Sep 29, 2018

rigelk Sep 29, 2018

Nutomic commented Sep 29, 2018

rigelk commented Sep 29, 2018

Nutomic commented Sep 29, 2018

Nutomic commented Sep 29, 2018

rigelk commented Sep 29, 2018

Nutomic commented Sep 29, 2018

rigelk commented Sep 29, 2018

Nutomic commented Sep 29, 2018

rigelk commented Sep 29, 2018

Nutomic commented Sep 29, 2018

Nutomic Sep 30, 2018

rigelk commented Sep 30, 2018

rigelk left a comment

Nutomic commented Sep 30, 2018

Nutomic commented Sep 30, 2018

rigelk commented Sep 30, 2018

Nutomic commented Oct 8, 2018

Chocobozzz commented Oct 8, 2018

Chocobozzz Oct 8, 2018

rigelk Oct 8, 2018

Nutomic Oct 8, 2018

rigelk Oct 8, 2018

Nutomic commented Oct 8, 2018

rigelk commented Oct 8, 2018

Chocobozzz commented Oct 9, 2018

Set bitrate limits for transcoding (fixes #638) #1135

Set bitrate limits for transcoding (fixes #638) #1135

Conversation

Nutomic commented Sep 28, 2018 • edited Loading

Nutomic commented Sep 28, 2018

rigelk commented Sep 28, 2018 • edited Loading

Nutomic commented Sep 28, 2018 • edited Loading

rigelk commented Sep 28, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nutomic commented Sep 29, 2018

rigelk commented Sep 29, 2018

Nutomic commented Sep 29, 2018

Nutomic commented Sep 29, 2018

rigelk commented Sep 29, 2018

Nutomic commented Sep 29, 2018

rigelk commented Sep 29, 2018

Nutomic commented Sep 29, 2018

rigelk commented Sep 29, 2018

Nutomic commented Sep 29, 2018

Choose a reason for hiding this comment

rigelk commented Sep 30, 2018

rigelk left a comment

Choose a reason for hiding this comment

Nutomic commented Sep 30, 2018

Nutomic commented Sep 30, 2018

rigelk commented Sep 30, 2018

Nutomic commented Oct 8, 2018

Chocobozzz commented Oct 8, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nutomic commented Oct 8, 2018

rigelk commented Oct 8, 2018

Chocobozzz commented Oct 9, 2018

Nutomic commented Sep 28, 2018 •

edited

Loading

rigelk commented Sep 28, 2018 •

edited

Loading

Nutomic commented Sep 28, 2018 •

edited

Loading

rigelk commented Sep 28, 2018 •

edited

Loading