added stream specification option to read_video

[CCareaga]

Added functionality described in issue #1220

[codecov-io]

Codecov Report

Merging #1222 into master will decrease coverage by 0.01%.
The diff coverage is 87.5%.

@@            Coverage Diff             @@
##           master    #1222      +/-   ##
==========================================
- Coverage   65.64%   65.63%   -0.02%     
==========================================
  Files          74       74              
  Lines        5784     5788       +4     
  Branches      884      886       +2     
==========================================
+ Hits         3797     3799       +2     
- Misses       1723     1725       +2     
  Partials      264      264

Impacted Files	Coverage Δ
torchvision/io/video.py	72.93% <87.5%> (+1.61%)	⬆️
torchvision/transforms/transforms.py	80.94% <0%> (-0.59%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8635be9...f3354a0. Read the comment docs.

[bjuncek]

Looks good to me - neat fix

[fmassa]

Thanks for the PR!

I think this is a good feature to have, but I'm not yet sure about how it should be handled.

I wonder if we can have multiple video / audio streams in a video? In which case, the proposed solution wouldn't be enough.

[CCareaga]

@fmassa Interesting, do you have a scenario in mind where a video might have multiple streams of a certain modality? Also, would the av package be able to handle that scenario?
I think that in the mean time, the stream selection option is useful (I originally proposed the feature for a project I'm working on).

[fmassa]

@CCareaga some video formats supports an arbitrary number of streams per type, see e.g., https://video.stackexchange.com/questions/20933/can-video-have-multiple-streams-like-it-has-2-audio-streams

PyAV handles this case (at least in its API), because we can query streams by it index.

One possible scenario I'd imagine would be having stereo video (for example for augmented reality with headsets).

Still, the current API of torchvision does not support that, as we return a single audio and video elements. We would need anyway to do a backwards-incompatible change to be able to support this case.

In this case, I'd say it would be ok for now to assume that we can specify it in the way you proposed.

[fmassa]

This looks good to me, thanks a lot!

I have only one more comment, which relates to the default empty type and shapes.

[CCareaga]

@fmassa Oh okay, that totally seems like a valuable feature. With the current read video function it would probably require a decent overhaul.