Is there StreamingMediaEncoder in torchcodec?

[darkAlert]

Hello!

I often use the StreamingMediaEncoder class from torio.io to encode RTMP streams. But the Torchcodec documentation only has audio encoding examples. Before I upgrade PyTorch to 2.9, could you tell me if torchcodec supports output stream encoding like torio.io?

[NicolasHug]

Hi @darkAlert ,

StreamingMediaEncoder does a bunch of things, can you help me understand what specific features you were relying on?

For example, do you need video encoding,or a multi-stream encoder than encodes both audio and video stream at the same time?

We plan to release a video encoder soon (video stream only). A multi-stream encoder that can encode both audio and video can be in scope too, depending on how popular this feature request is.

[darkAlert]

We plan to release a video encoder soon (video stream only).

@NicolasHug That's exactly it

I work in computer vision and often use this functionality to generate output streams on the fly with various visualizations, such as bounding boxes. In my opinion, StreamingMediaEncoder does this faster and more efficiently than others! Especially when encoding a stream from tensors on the GPU (with hw_accel). Thank you for your work!

I'd also really like to be able to encode RTSP in addition to RTMP.

Regarding audio encoding, I once needed to add a subtitle track to a video stream (I wanted to use it as a data stream for server-side rendering), but this feature wasn't supported at the time. I considered encoding the data into an audio stream (for example, in pcm_s16le format). It encoded well, but decoding the data was a problem...