Add an initial Segformer implementation

[jimexist]

this is a copy from the huggingface pytorch model.

this is inference only so i omitted the parts about dropout, etc.

for image classification with num_labels

the model outputs a tensor with shape [batch_size, num_labels] where the last dim is logits

for image segmentation with num_labels, height and width

the model outputs a tensor with shape [batch_size, num_labels, height // 4, width // 4] to represent each 4-pixel patch's logits

Classification example

this shall give you burger label:

classification logits [3.275261e-5, 0.0008562019, 0.0008868563, 0.9977506, 0.0002465068, 0.0002241473, 2.846596e-6]
label: hamburger

Segmentation example

input image (url is https://huggingface.co/datasets/hf-internal-testing/fixtures_ade20k/resolve/main/ADE_val_00000001.jpg):

output segmentation image:

[LaurentMazare]

This looks like a neat addition, thanks for working on this.
Would you mind adding to the PR a couple sample input and output so as to illustrate what to expect from the model?

[jimexist]

This looks like a neat addition, thanks for working on this. Would you mind adding to the PR a couple sample input and output so as to illustrate what to expect from the model?

@LaurentMazare i added some in the PR description

[LaurentMazare]

This looks like a neat addition, thanks for working on this. Would you mind adding to the PR a couple sample input and output so as to illustrate what to expect from the model?

@LaurentMazare i added some in the PR description

Apologies if I misread your update but I was more expecting some example images as well as the segmentation masks generated by the model.

[jimexist]

This looks like a neat addition, thanks for working on this. Would you mind adding to the PR a couple sample input and output so as to illustrate what to expect from the model?

@LaurentMazare i added some in the PR description

Apologies if I misread your update but I was more expecting some example images as well as the segmentation masks generated by the model.

sample image added with readme updated.

[jimexist]

segmentation image added

[LaurentMazare]

Actually I meant more adding some details to the PR rather than to the repo, it's a bit tricky to add more image file in it as it's already on the large side and we would prefer limiting how much more we add. Could you remove the image files and replace them by some wget commands in the readme?

[jimexist]

Actually I meant more adding some details to the PR rather than to the repo, it's a bit tricky to add more image file in it as it's already on the large side and we would prefer limiting how much more we add. Could you remove the image files and replace them by some wget commands in the readme?

ah i see. i have included the image into the pull request itself and updated the readme file accordingly, also removed the image files from git history.

[LaurentMazare]

Thanks!

[LaurentMazare]

Also I would encourage you to advertise this new model on reddit/X/... or via some form of blog post. Would be great to attract some attention on it as it seems like a potentially pretty useful model and it can be nicely illustrated with your sample pics.