Pretraining VideoMAE on HMDB51

[chenliz1]

Hi Zhan,

Thank you for your excellent work! We are surprised by VideoMAE's data efficiency (paper section 4.3), especially on small datasets like HMDB51. We are trying to use your code to reproduce your experiment on HMDB51. However, we cannot achieve the same finetune accuracy as the table2 reported (61.1 %):

Our experiments show that the model is converged after 2400 pretraining epochs. We are using eight Tesla V100 32GB GPUs. Also, we changed the batch size, learning rate and temporal stride as the appendix described.

I wonder if you can also share your complete experiment configurations for UCF101 and HMDB51? Some settings like warmup epochs number may also be critical for reproducing.

[jianyulei1]

Hi chen,

Can you share your settings for reference？My training doesn 't work, I ' d like to refer to your settings for UCF101 and HMDB51.

[chenliz1]

Hi Jianyu,

Here's my seetings for HMDB51 on 8 GPUs 1 Server:

OMP_NUM_THREADS=1 python -m torch.distributed.launch --nproc_per_node=8
--master_port 12320
run_mae_pretraining.py
--data_path ${DATA_PATH}
--mask_type tube
--mask_ratio 0.9
--model pretrain_videomae_base_patch16_224
--decoder_depth 4
--batch_size 24
--num_frames 16
--sampling_rate 2
--opt adamw
--opt_betas 0.9 0.95
--warmup_epochs 40
--save_ckpt_freq 100
--epochs 4800
--log_dir ${OUTPUT_DIR}
--output_dir ${OUTPUT_DIR}
--lr 3e-4

The setting for UCF101 is similar, only need to modify the epochs number to 3200 and sampling_rate to 4.

[jianyulei1]

Thank you for your reply, can you share the fine-tuning settings for HMDB51 and UCF101 with me? Thank you very much.

…

------------------ 原始邮件 ------------------ 发件人: "MCG-NJU/VideoMAE" ***@***.***>; 发送时间: 2022年7月27日(星期三) 凌晨5:34 ***@***.***>; ***@***.******@***.***>; 主题: Re: [MCG-NJU/VideoMAE] Pretraining VideoMAE on HMDB51 (Issue #38) Hi Jianyu, Here's my seetings for HMDB51 on 8 GPUs 1 Server: OMP_NUM_THREADS=1 python -m torch.distributed.launch --nproc_per_node=8 --master_port 12320 run_mae_pretraining.py --data_path ${DATA_PATH} --mask_type tube --mask_ratio 0.9 --model pretrain_videomae_base_patch16_224 --decoder_depth 4 --batch_size 24 --num_frames 16 --sampling_rate 2 --opt adamw --opt_betas 0.9 0.95 --warmup_epochs 40 --save_ckpt_freq 100 --epochs 4800 --log_dir ${OUTPUT_DIR} --output_dir ${OUTPUT_DIR} --lr 3e-4 The setting for UCF101 is similar, only need to modify the epochs number to 3200 and sampling_rate to 4. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[chenliz1]

For UCF101:

OMP_NUM_THREADS=1 python -m torch.distributed.launch --nproc_per_node=8
--master_port 12320
run_class_finetuning.py
--model vit_base_patch16_224
--data_set UCF101
--nb_classes 101
--data_path ${DATA_PATH}
--finetune ${MODEL_PATH}
--log_dir ${OUTPUT_DIR}
--output_dir ${OUTPUT_DIR}
--batch_size 8
--update_freq 2
--num_sample 1
--input_size 224
--short_side_size 224
--save_ckpt_freq 10
--num_frames 16
--sampling_rate 4
--opt adamw
--lr 1e-3
--opt_betas 0.9 0.999
--weight_decay 0.05
--epochs 100
--dist_eval
--test_num_segment 5
--test_num_crop 3
--enable_deepspeed
# --eval

[jianyulei1]

Thank you for your reply, if I want to set batch size to 8, LR need to be changed?

[chenliz1]

Thank you for your reply, if I want to set batch size to 8, LR need to be changed?
For the finetuning, I set --batch_size 8 and --update_freq 2, so theoretically I am training the model with 128 batches on 8 GPUs. But if you want to use a different total batch size, you may need to tune the LR for better results.

[youcaiSUN]

Hi, @chenliz1, could you share the time it takes to pre-train on HMDB51 and UCF101 in your setting? Besides, what is your reproduced result on UCF101? Thanks!

[Morning-YU]

Hi, @chenliz1 , can you tell me how to process UCF101 dataset？What the data format looks like？Looking forward to your reply！

[yztongzhan]

Hi @chenliz1 @jianyulei1 @youcaiSUN @Morning-YU ! The pre-trained models and scripts on UCF101 are available! Scripts on HMDB51 are coming soon!