Rewrite live media transcoder

[MatMaul]

Following a point of @nvllsvm on matrix I am opening an issue for the rewrite of the transcoder engine.

Some random thoughts:

1. should probably be an HTTP micro service, because it is easy to integrate with anything. Also this will allow us to put the transcoding engine on a different machine than the server, can be quite useful.
2. Language probably Python or Go as stated by @nvllsvm. I would prefer Go mainly because I like to have a type system, but Python is easier for beginners :)
3. scope: live accelerated transcoding with subtitles support. I think we should not make a fully generic transcoding tool, it is too complicated to handle all the options and there is already several alternatives for that (Handbrake).
4. output: streamable video available through HTTP. It should be easily doable to serve that in Jellyfin through a very simple proxy.
5. input: this is a bit more tricky :) We can either consider that the transcoder server has the same medias mounted as the server but it is a bit restricting. The best is probably to pass that through HTTP too.

Let the fun begins :)

[JustAMan]

Nice summary. I prefer Python because I like Python :D
As for inputs - we may want to implement it in a Unix philosophy - "pipe in -> pipe out", e.g. by using websockets. That way the service would be completely stateless and won't have to have any access to media files.

[MatMaul]

Not really, since HLS uses fragmented video segments the one to one pipe model doesn't really map. I think serving the result as HTTP is simple enough and map quite well the use case, we are aiming at live transcoding after all :)

[JustAMan]

How come it doesn't map? Jellyfin server pipes media in, jellyfin client pipes transcoded result out... All fits IMHO.

[MatMaul]

It doesn't really output a stream in the HLS use case, it's a set of independent named files (3s each in jellyfin) with a playlist that references them. I think you need some kind of directory structure to expose that. You don't need to list files however so HTTP seems ok.
That's for the output, for the input at least a seekable source and probably a fs-like interface for stuffs like bluray or dvd backups.

[MatMaul]

Here is a working prototype:
https://github.com/MatMaul/pytranslive
https://github.com/MatMaul/translive-server

[JustAMan]

It doesn't really output a stream in the HLS use case, it's a set of independent named files (3s each in jellyfin) with a playlist that references them. I think you need some kind of directory structure to expose that. You don't need to list files however so HTTP seems ok.
That's for the output, for the input at least a seekable source and probably a fs-like interface for stuffs like bluray or dvd backups.

Well, actually in case of HLS it's a set of independent URLs, not necessary files... so that's still fine to "pipe them out" of the transcoding server.
As for seekable source... I doubt it is really that much needed, ffmpeg can encode stuff fed from stdin which is certainly not seekable. Seems like a waste of bandwidth to transfer stuff to skip, though, I agree.

[MatMaul]

Pipe is the wrong terminology then, pipe is one bitstream in one bitstream out, no name, no url, nothing. Several named streams is clearly not a pipe.
In the end we say the same stuff, and it works perfectly fine with HTTP.

[jaredallard]

If you guys decide to go with Go, it'd be awesome because then I could contribute to something 😁

Also -- I'm not sure that python can be considered easier for beginners, Go is a pretty easy language to pick up

[JustAMan]

@EraYaN this looks like a proper feature for your "chunk server". Thoughts?

[EraYaN]

Seems like it, though I don't have the time to take a look, but if anyone else wants to take a look: https://github.com/EraYaN/FFmpegSegmentManager (it's a rough test)
Although we should probably stick with the same deps and technologies as the current server, so we don't add extra build deps of runtime deps.

[ntindle]

Found this issue. Was there anything wrong with @MatMaul ’s implementation?

[EraYaN]

Well one problem was that one option was in Go ;). That aside, it will need a lot of extra work and will share a lot of code with the server, so we might be better off doing it in C#. This will also keep the footprint low on the build and while running it. And the main goal is that it should produce reproducible chunks, if at all possible. This would make caching these trivial. And lots of things have changed in the server and lot more things will change. If we ever do something like this it will definitely be a version 11.

[sevenrats]

moved
https://features.jellyfin.org/posts/2355/rewrite-the-livetv-transcoder