Visual Attention Network (VAN)

[FrancescoSaverioZuppichini]

What does this PR do?

This PR adds Visual Attention Network.

Currently, the model can be used as follows

import requests
from io import BytesIO
res = requests.get('https://github.com/huggingface/transformers/blob/master/tests/fixtures/tests_samples/COCO/000000039769.png?raw=true')
image = Image.open(BytesIO(res.content))

feature_extractor = AutoFeatureExtractor.from_pretrained("zuppif/van-base")
model = VanForImageClassification.from_pretrained("zuppif/van-base").eval()

inputs = feature_extractor(image, return_tensors="pt")
outputs = model(**inputs)
print(model.config.id2label[torch.argmax(outputs.logits).item()])
# tabby, tabby cat

TODO

• modeling
• weights
• doc
• tests

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[sgugger]

Thanks for adding this model!

[NielsRogge]

Re-using this for all models? ;)

You're sure it expects the same transforms at evaluation time?

[FrancescoSaverioZuppichini]

it was trained on imagenet

[NielsRogge]

To be updated

[NielsRogge]

Thanks for adding!

Did you checks the tests pass?

[FrancescoSaverioZuppichini]

It is done in the CI, isn't it?

[FrancescoSaverioZuppichini]

Thanks for the reviews. I've resolved all the comments that can be resolved and asked the authors if we can create an organization for them in the hub.