Skip to content

Conversation

ahmadsharif1
Copy link
Contributor

The benchmark shows that we are slower than non-batch for batch decoding. We can make improvements to the decoder to make this more performant.

python benchmarks/decoders/benchmark_decoders.py --bm_video_paths=/home/ahmads/personal/lerobot/benchmarks/video/outputs/video_benchmark/videos/libx264_yuv420p_None_None/lerobot_pusht_image.mp4,/home/ahmads/personal/lerobot/benchmarks/video/outputs/video_benchmark/videos/libx265_yuv420p_None_None/lerobot_pusht_image.mp4,/home/ahmads/jupyter/853.mp4 --decoders=torchvision,torchcodec,torchcodec_batch
[ video=/home/ahmads/personal/lerobot/benchmarks/video/outputs/video_benchmark/videos/libx264_yuv420p_None_None/lerobot_pusht_image.mp4 h264 96x96, 16.1s 10.0fps ]
                                             |  10 seek()+next()  |  1 next()  |  10 next()  |  100 next()
1 threads: -----------------------------------------------------------------------------------------------
      TorchCodecDecoderNonCompiled           |        30.6        |    14.9    |     16.6    |     37.0   
      TVNewAPIDecoderWithBackendVideoReader  |        62.7        |     4.0    |      5.0    |     15.1   
      TorchCodecDecoderNonCompiledBatch      |        36.7        |    16.9    |     19.8    |     34.5   

Times are in milliseconds (ms).

[ video=/home/ahmads/personal/lerobot/benchmarks/video/outputs/video_benchmark/videos/libx265_yuv420p_None_None/lerobot_pusht_image.mp4 hevc 96x96, 16.1s 10.0fps ]
                                             |  10 seek()+next()  |  1 next()  |  10 next()  |  100 next()
1 threads: -----------------------------------------------------------------------------------------------
      TorchCodecDecoderNonCompiled           |        31.2        |    20.4    |     22.2    |     42.9   
      TVNewAPIDecoderWithBackendVideoReader  |        68.9        |     3.6    |      4.6    |     15.9   
      TorchCodecDecoderNonCompiledBatch      |        33.6        |    27.8    |     28.9    |     45.1   

Times are in milliseconds (ms).

[---------- video=/home/ahmads/jupyter/853.mp4 h264 3840x2160, 4.988317s 59.94005593469701fps -----------]
                                             |  10 seek()+next()  |  1 next()  |  10 next()  |  100 next()
1 threads: -----------------------------------------------------------------------------------------------
      TorchCodecDecoderNonCompiled           |       1917.2       |   243.9    |    388.7    |    1665.2  
      TVNewAPIDecoderWithBackendVideoReader  |      28424.0       |   268.6    |    905.9    |   11896.4  
      TorchCodecDecoderNonCompiledBatch      |       2035.0       |   314.8    |    523.0    |    3046.3  

Times are in milliseconds (ms).

[ video=/home/ahmads/personal/lerobot/benchmarks/video/outputs/video_benchmark/videos/libx264_yuv420p_None_None/lerobot_pusht_image.mp4 h264 3840x2160, 4.988317s 59.94005593469701fps ]
                                    |  create()+next()
1 threads: -------------------------------------------
      TorchCodecDecoderNonCompiled  |        14.9     

Times are in milliseconds (ms).

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Sep 4, 2024
@ahmadsharif1 ahmadsharif1 merged commit 2b4f809 into meta-pytorch:main Sep 6, 2024
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants