Skip to content
/ Multimodal-Video-Captioning Public

Master Thesis on Multimodal Video Captioning, done at Huawei's Research Center in Amsterdam.

3 stars 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

El-Zag/Multimodal-Video-Captioning

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
models/32frm/vatex/best-checkpoint
models/32frm/vatex/best-checkpoint
 
 
prepro
prepro
 
 
scripts
scripts
 
 
src
src
 
 
README.md
README.md
 
 
attention_mask.npy
attention_mask.npy
 
 
aud_mean.npy
aud_mean.npy
 
 
aud_std.npy
aud_std.npy
 
 
launch_container.sh
launch_container.sh
 
 
linelists.py
linelists.py
 
 
redo_prepro.py
redo_prepro.py
 
 
reproduce_results.ipynb
reproduce_results.ipynb
 
 
setup.sh
setup.sh
 
 

Repository files navigation

Multimodal Video Captioning

Code for my Master Thesis on Multimodal Video Captioning. The SwinBERT model was used as baseline, and I integrated audio features extracted with VGGish to the architecture, resulting in an up to +1.6 gain in captioning Metrics.

About

Master Thesis on Multimodal Video Captioning, done at Huawei's Research Center in Amsterdam.

Topics

audio nlp natural-language-processing computer-vision deep-learning transformer video-captioning multimodal multimodal-deep-learning

Resources

Readme
Activity

Stars

3 stars

Watchers

1 watching

Forks

0 forks
Report repository

Languages