[DEST] add DEST model

[zhby99]

Motivation

We are from NVIDIA and we have developed a simplified and inference-efficient transformer for dense prediction tasks. The method is based on SegFormer with hardware-friendly design choices, resulting in better accuracy and over 2x reduction in inference speed as compared to the baseline. We believe this model would be of particular interests to those who want to deploy an efficient vision transformer for production, and it is easily adaptable to other tasks. Therefore, we would like to contribute our method to mmsegmentation in order to benefit a larger audience.

The paper was accepted to Transformer for Vision workshop at CVPR 2022, here below are some resource links:
Paper https://arxiv.org/pdf/2204.13791.pdf (Table 3 shows the semseg results)
Code https://github.com/NVIDIA/DL4AGX/tree/master/DEST
A webinar on its application https://www.nvidia.com/en-us/on-demand/session/other2022-drivetraining/

Modification

Add backbone(smit.py) and head(dest_head.py) of DEST

BC-breaking (Optional)

N/A

Use cases (Optional)

N/A

[CLAassistant]

All committers have signed the CLA.

[zhby99]

@MeowZheng Please help to review the PR. Thanks!
@lean76

[zhby99]

@MeowZheng Please help to review the PR. Thanks! @lean76

[zhby99]

The checkpoints and logs for smit-b0~5 are under https://drive.google.com/drive/folders/1MOnW41XKPRYseOtxh_wgsbgpt7xwmeLb?usp=sharing for reference

[MeowZheng]

@MeowZheng Please help to review the PR. Thanks! @lean76

Thanks for your contribution, please sign CLA first.

[zhby99]

@MeowZheng Please help to review the PR. Thanks! @lean76

Thanks for your contribution, please sign CLA first.

Thanks for your reply, I have signed CLA.

[MeowZheng]

Hi @zhby99
To merge DEST quickly into MMSegmentaion repo, we suggest submitting it in community projects.
I'm sorry that I didn't mention this submission in the email, as we just started trying to use projects for working with the community.

Please modify this pr following the example_project, and if you have any questions, please do not hesitate to contact me.

[zhby99]

@MeowZheng Hi Miao, Thanks for your suggestion, I have moved all related codes into projects directory, looking forward to your comments!

[MeowZheng]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

boyinz seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

There might be 2 users in this pr boyinz and zhby99, and some commits are from boyinz who might be some default name of f your local git before you config it. CLA check is pending until you modify the commit history to remove boyinz.

Moreover, there are some ci problems, but we have fixed it in #2520

Therefore, I suggest you might close this pr and create a new one based on the latest master.

Sorry to make your modification many times.

[zhby99]

@MeowZheng Thanks for your comments, I have rebased this pr on latest master and reset the commits with single user id.

[MeowZheng]

please follow the example project to add Prerequisites and Checklist.

This project needs to provide the checkpoints, so would you like to share the model checkpoints？

[zhby99]

please follow the example project to add Prerequisites and Checklist.

This project needs to provide the checkpoints, so would you like to share the model checkpoints？

Please find the checkpoints and logs under https://drive.google.com/drive/folders/1MOnW41XKPRYseOtxh_wgsbgpt7xwmeLb?usp=sharing .
The prerequisites of this project is same with mmseg(i.e any environment that can build mmseg could train and test this project)
For checklist, I wonder if it is just something for reviewers and me to check? should we also add them to the README files?

[MeowZheng]

please follow the example project to add Prerequisites and Checklist.
This project needs to provide the checkpoints, so would you like to share the model checkpoints？

Please find the checkpoints and logs under https://drive.google.com/drive/folders/1MOnW41XKPRYseOtxh_wgsbgpt7xwmeLb?usp=sharing . The prerequisites of this project is same with mmseg(i.e any environment that can build mmseg could train and test this project) For checklist, I wonder if it is just something for reviewers and me to check? should we also add them to the README files?

As it may not be compatible between the different versions of mmseg and mmcv, we need to fix the version of them.
The checklist is for other users in the community, and they can understand the status of this algorithm.
Projects sometimes merge the model that is not official reimplement, just copy the model and model ckpt, whose performance can not be guaranteed, and the checklist works for providing the status of the model.

There is an example https://github.com/open-mmlab/mmsegmentation/tree/dev-1.x/projects/HieraSeg

BTW, I think we may talk more efficiently on WeChat if it is convenient for you. Would you like to email me your wechat id?

Thanks again for your contribution, and the documentation for community contribution processes needs to be improved

[zhby99]

please follow the example project to add Prerequisites and Checklist.
This project needs to provide the checkpoints, so would you like to share the model checkpoints？

Please find the checkpoints and logs under https://drive.google.com/drive/folders/1MOnW41XKPRYseOtxh_wgsbgpt7xwmeLb?usp=sharing . The prerequisites of this project is same with mmseg(i.e any environment that can build mmseg could train and test this project) For checklist, I wonder if it is just something for reviewers and me to check? should we also add them to the README files?

As it may not be compatible between the different versions of mmseg and mmcv, we need to fix the version of them. The checklist is for other users in the community, and they can understand the status of this algorithm. Projects sometimes merge the model that is not official reimplement, just copy the model and model ckpt, whose performance can not be guaranteed, and the checklist works for providing the status of the model.

There is an example https://github.com/open-mmlab/mmsegmentation/tree/dev-1.x/projects/HieraSeg

BTW, I think we may talk more efficiently on WeChat if it is convenient for you. Would you like to email me your wechat id?

Thanks again for your contribution, and the documentation for community contribution processes needs to be improved

Thanks for your suggestions. I have sent you an email with my wechat id.

[zhby99]

@MeowZheng Hi Miao, I have updated this PR with checklists and prerequisites

[openmmlab-bot]

zhby99，您好！您在MMSeg项目中给我们提的PR非常重要，感谢您付出私人时间帮助改进开源项目，相信很多开发者会从你的PR中受益。
我们非常期待与您继续合作，OpenMMLab专门成立了贡献者组织MMSIG，为贡献者们提供开源证书、荣誉体系和专享好礼，可通过添加微信：openmmlabwx 联系我们（请备注mmsig+GitHub id），由衷希望您能加入！
Dear zhby99,
First of all, we want to express our gratitude for your significant PR in the MMSeg project. Your contribution is highly appreciated, and we are grateful for your efforts in helping improve this open-source project during your personal time. We believe that many developers will benefit from your PR.
We are looking forward to continuing our collaboration with you. OpenMMLab has established a special contributors' organization called MMSIG, which provides contributors with open-source certificates, a recognition system, and exclusive rewards. You can contact us by adding our WeChat（if you have WeChat): openmmlab_wx, or join in our discord： https://discord.gg/qH9fysxPDW. We sincerely hope you will join us!
Best regards！