Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding resnext101 64x4d model #5935

Merged
merged 15 commits into from
May 9, 2022

Conversation

YosuaMichael
Copy link
Contributor

@YosuaMichael YosuaMichael commented May 3, 2022

Resolve #3485

Training and Validation script

We train the model using the following script:

python -u ~/script/run_with_submitit.py \
    --timeout 3000 --ngpus 8 --nodes 1 --batch-size=128 \
    --partition train --model resnext101_64x4d \
    --data-path="/datasets01_ontap/imagenet_full_size/061417" \
    --lr=0.5 --lr-scheduler=cosineannealinglr --lr-warmup-epochs=5 --lr-warmup-method=linear \
    --auto-augment=ta_wide --epochs=600 --random-erase=0.1 --weight-decay=0.00002 \
    --norm-weight-decay=0.0 --label-smoothing=0.1 --mixup-alpha=0.2 --cutmix-alpha=1.0 \
    --train-crop-size=176 --model-ema --val-resize-size=232 --ra-sampler --ra-reps=4

From the training, we found the best result is on epoch 455 with the EMA model.
We validate with the 1 gpu and batch_size==1 using the following script (with final output):

python -u ~/script/run_with_submitit.py \
    --timeout 3000 --nodes 1 --ngpus 1 --batch-size=1 \
    --partition train --model "resnext101_64x4d" \
    --data-path="/datasets01_ontap/imagenet_full_size/061417" \
    --weights="ResNeXt101_64X4D_Weights.IMAGENET1K_V1" \
    --test-only
# Acc@1 83.246 Acc@5 96.454

Quantized model

We do quantize the model by using the following script:

python train_quantization.py \
    --device='cpu' --post-training-quantize --backend='fbgemm' \
    --model=resnext101_64x4d --weights="ResNeXt101_64X4D_Weights.IMAGENET1K_V1" \
    --train-crop-size 176 --val-resize-size 232 --data-path /datasets01_ontap/imagenet_full_size/061417/

And from the result we validate it again using 1 gpu and batch_size=1 with the following script:

python train_quantization.py --device='cpu' --eval-batch-size=1 --test-only --backend='fbgemm' \
    --model='resnext101_64x4d'  --data-path="/datasets01_ontap/imagenet_full_size/061417" \
    --weights="ResNeXt101_64X4D_QuantizedWeights.IMAGENET1K_FBGEMM_V1"
# Acc@1 82.898 Acc@5 96.326

@YosuaMichael YosuaMichael self-assigned this May 3, 2022
@YosuaMichael YosuaMichael changed the title Models/resnext101 64x4d Adding resnext101 64x4d model May 3, 2022
@YosuaMichael YosuaMichael marked this pull request as ready for review May 4, 2022 13:05
@YosuaMichael YosuaMichael requested a review from datumbox May 4, 2022 13:15
Copy link
Contributor

@datumbox datumbox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@YosuaMichael LGTM, thanks! Just one minor nit, see below.

docs/source/models.rst Show resolved Hide resolved
@YosuaMichael
Copy link
Contributor Author

The failing test seems to not related to this PR, see #5971

@YosuaMichael YosuaMichael merged commit 4c02f10 into pytorch:main May 9, 2022
@datumbox datumbox mentioned this pull request May 9, 2022
24 tasks
facebook-github-bot pushed a commit that referenced this pull request May 11, 2022
Summary:
* Add resnext101_64x4d model definition

* Add test for resnext101_64x4d

* Add resnext101_64x4d weight

* Update checkpoint to use EMA weigth

* Add quantization model signature for resnext101_64x4d

* Fix class name and update accuracy using 1 gpu and batch_size=1

* Apply ufmt

* Update the quantized weight and accuracy that we still keep the training log

* Add quantized expect file

* Update docs and fix acc1

* Add recipe for quantized to PR

* Update models.rst

Reviewed By: YosuaMichael

Differential Revision: D36281598

fbshipit-source-id: 300bd36343b8ad8b185a246b794e078bdf67f5c8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add the pretrained resnext101_64x4d model
3 participants