Video Foundation Models & Data for Multimodal Understanding
-
Updated
Jun 5, 2024 - Python
Video Foundation Models & Data for Multimodal Understanding
[ECCV 2020] PyTorch code for XML on TVRetrieval dataset - TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval
[arXiv 23] Pytorch code for "Overcoming Weak Visual-Textual Alignment for Video Moment Retrieval"
[EMNLP 2022] Pytorch code for "Modal-specific Pseudo Query Generation for Video Corpus Moment Retrieval"
MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Models (CVPR 2023)
[NeurIPS 2021] Moment-DETR code and QVHighlights dataset
[IJCAI 2023] Text-Video Retrieval with Disentangled Conceptualization and Set-to-Set Alignment
[ICCV 2023] DiffusionRet: Generative Text-Video Retrieval with Diffusion Model
[CVPR 2023 Highlight] Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning
[NeurIPS 2022 Spotlight] Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations
Authors official PyTorch implementation of the "ViSiL: Fine-grained Spatio-Temporal Video Similarity Learning" [ICCV 2019]
My source code for HCM AI 2023
A curated list of video-text datasets in a variety of languages. These datasets can be used for video captioning (video description) or video retrieval.
[TIP 2022] Official code of paper “Video Question Answering with Prior Knowledge and Object-sensitive Learning”
Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Pre-training Dataset and Benchmarks
The code for the paper "GMMFormer: Gaussian-Mixture-Model Based Transformer for Efficient Partially Relevant Video Retrieval" (AAAI'24)
A large Cross-Modal Video Retrieval Dataset with Reading Comprehension
A PyTorch implementation of EmpiricalMVM
A PyTorch implementation of VIOLET
Authors official PyTorch implementation of the "Self-Supervised Video Similarity Learning" [CVPRW 2023]
Add a description, image, and links to the video-retrieval topic page so that developers can more easily learn about it.
To associate your repository with the video-retrieval topic, visit your repo's landing page and select "manage topics."