Is there any way to replace video stream in a vrs file?

[chenjoya]

Hi, thank you for the amazing work. We want to mosaic the people's face in the video stream of a vrs, but we can only find the way to extract image from a vrs file rather than write to it.

Could you provide some guidelines to achieve it? Many thanks.

[georges-berenger]

Have you tried class RecordFilterCopier in https://github.com/facebookresearch/vrs/blob/main/vrs/utils/FilterCopyHelpers.h ? It should allow you to do what you need.

[georges-berenger]

Put your custom image filter code in a class derived from the RecordFilterCopier class, that you pass to vrs::utils::filterCopy via a custom MakeStreamFilterFunction that injects the right filter for each stream (or calls makeCopier for the streams that shouldn't be modified).

[chenjoya]

Thank you so much. I have solved my problems. In fact I am working with Meta's Ego4D team and we need this to blur face in VRS file. I provide my implementation here: https://github.com/chenjoya/vrsfacem/blob/master/vrs/vrs/utils/cli/MakeFaceMosaicCopier.cpp
Do you have some guidances?

[georges-berenger]

good to hear you're unblocked.
The code looks good.
Suggestion:
shouldCopyVerbatim() is a lot more complicated than need be:
makeFaceMosaicCopier could use RecordFileReader::mightContainImages and RecordFileReader::mightContainAudio to either return a FaceMosaic if mightContainImages is true, otherwise a makeZeroFilterCopier if mightContainAudio is true, otherwise, a Copier. Then you can delete most of the code in shouldCopyVerbatim and filterAudio entirely.

[chenjoya]

Nice. VRS is really talented. I learned a lot.

Thank you so much!

[georges-berenger]

You're welcome!