[ViT] Graduate ViT from prototype

[yiwen-song]

Discussed offline with @datumbox, now that all the concerns have been addressed, we can finally graduate ViT models from prototype.

Testing

1. Unittest

EXPECTTEST_ACCEPT=1 pytest test/test_models.py -k ${model_name}

2. Test the legacy weights api

PYTHONPATH=$PYTHONPATH:`pwd` python -u ~/workspace/scripts/run_with_submitit.py --timeout 3000 --ngpus 1 --nodes 1 --partition train --model vit_b_16 --batch-size 1 --test-only --pretrained

Job ID: 14506

3. Test the new weights api in prototype

Run a test job

PYTHONPATH=$PYTHONPATH:`pwd` python -u ~/workspace/scripts/run_with_submitit.py --timeout 3000 --ngpus 1 --nodes 1 --partition train --model vit_b_16 --batch-size 1 --test-only --weights ViT_B_16_Weights.ImageNet1K_V1

Job ID: 14504

cc @datumbox

[facebook-github-bot]

💊 CI failures summary and remediations

As of commit 8d6b0d3 (more details on the Dr. CI page):

---

💚 💚 Looks good so far! There are no failures yet. 💚 💚

---

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

[datumbox]

@sallysyw Looks good. I've added only 2 minor comments, please let me know your thoughts.

Also please add on the PR with the correct tags, so that we can produce easier the release notes. You can read here how tags work.

[datumbox]

@sallysyw There are a few more things to be done to complete the migration:

• Expose the models to TorchHub. See Expose raft_large and raft_small to torchhub #5174
• Add pre-trained model documentation. See this example

The above were omitted earlier as the model was in prototype but now that it graduates, they need to be added.

[yiwen-song]

@sallysyw There are a few more things to be done to complete the migration:

• Expose the models to TorchHub. See Expose raft_large and raft_small to torchhub #5174
• Add pre-trained model documentation. See this example

The above were omitted earlier as the model was in prototype but now that it graduates, they need to be added.

Sure thing! Let me do it right now.

[yiwen-song]

For hubconf.py, linter insists that the import should happen in alphabetical order - which means vit must be put under the segmentation section, any ideas how to avoid that? @datumbox @NicolasHug

[datumbox]

@sallysyw Everything in that file is in alphabetic order, so adding it below VGG makes sense. If anything I think removed the comments make sense because they are incorrect.

Concerning the unused import, try doing:
# noqa: F401

The rest of the changes look good to me. Ping me when you fix the linters to review and merge. :)

[datumbox]

LGTM, thanks!

[github-actions]

Hey @sallysyw!

You merged this PR, but no labels were added. The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py