Adding TF Implementation of BEiT

[MadElf1337]

Feature request

Addition of TF implementation of BEiT

Motivation

I have always seen that there is a discrepancy in the availability of models for PyTorch and the models available in TensorFlow, and want to have models for usage in both backends.

Your contribution

I will add the implementation of BEiT in TF :)

cc - @gante

[LysandreJik]

cc @NielsRogge @amyeroberts

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[amyeroberts]

Hi @MadElf1337 do you have any updates? Are you still planning on contributing this model?

[MadElf1337]

Yep I’m still working on the model, had to keep it aside for a bit due to my uni exam schedule, but will start again the day my exams are over

Regarding the updates, I am done with the architecture, have to write the functions for specific purposes(like segmentation) and the tests

[amyeroberts]

Great - glad to hear you're still interested :)

As @NielsRogge pointed out, data2vec vision is an extension of BEiT. This means the porting should be a lot simpler! In our pytorch BEiT implementation, you can see this from the #Copied from statements. Ideally the TF implementation would reflect this and be the same as our pytorch implementation, however TF data2vec vision is already implemented. So, we need to move the data2vec code to beit, and then add the necessary #Copied from statement in data2vec. Does this make sense?

Could you open a draft PR for the model please so that the code is visible?

Good luck with the last of your exams!

[MadElf1337]

Yes I’ll open a draft PR to show the code that’s been done till date

And thanks!