Allow passing a file-like object to torchvision.io.video.read_video

[voegtlel]

🚀 The feature

As the title states, I'd like to be able to pass a file-like object to read_video.

Motivation, pitch

As far as I can see, there should be no issue allowing this, as pyav supports that. The current if not os.path.exists(filename): is the only thing preventing this, as I see it. It could be specially useful e.g. for webdataset, which currently extracts the videos to /tmp, but could also pass it directly from memory, which would be much more efficient.

I wonder if there is any reason to not expose this interface?

Alternatives

No response

Additional context

No response

[NicolasHug]

Hi @voegtlel , thank you for the feature request. Can you detail exactly what you mean by a "file-like object"? What Python type would that be exactly? Thank you!

[voegtlel]

Hi @NicolasHug,

sure, but I can only refer to the documentation of pyav

av.open(file, mode='r', **kwargs)
Main entrypoint to opening files/streams.

Parameters:
file (str) – The file to open, which can be either a string or a file-like object.

Although I must admit, that this is misleading, because typing only states str as type, but the documented string says, that file-like objects are allowed. I believe they refer to io, which is also what I'd understand:

The io module provides Python’s main facilities for dealing with various types of I/O. There are three main types of I/O: text I/O, binary I/O and raw I/O. These are generic categories, and various backing stores can be used for each of them. A concrete object belonging to any of these categories is called a file object. Other common terms are stream and file-like object.

[NicolasHug]

Do you think this is already handled by some of the private APIs like _read_video_from_memory ?

For context, the video decoders of torchvision aren't really maintained lately and we hope to provide better support for video decoding soon. It is possible for us to address some minor improvements as long as they're super easy, but big changes may not be in scope for torchvision. TBH, it's still not clear to me what exactly this feature request is about (i.e. what types need to be supported), so it's hard for me to give you a definite answer just yet

[voegtlel]

I could imagine that this API could solve it as well. Effectively, this request is about allowing streams in the old api, which is not possible right now because of this line: https://github.com/pytorch/vision/blob/main/torchvision/io/video.py#L270 which expects a filename, but the pyav api could also handle a BytesIO object. I.e. if that line would be like:

if not isinstance(filename, io.IOBase) and not os.path.exists(filename):

[NicolasHug]

Thanks for the details.

OK, I guess we can move the if not os.path.exists(filename): within the if get_video_backend() != "pyav": block and just let pyav do its own input checking.

[voegtlel]

@NicolasHug Thank you so much for the quick implementation! :) One week OOO and it's already in there, great! 👏