Identify other SOTA models for segmentation with ONNX supported TorchScript operators

[mxochicale]

🚀 Feature

When looking for other State-Of-The-Art (SOTA) models for our pipeline, let's check that such model is with ONNX supported TorchScript operators as I had an issue with "aten::max_unpool2d" in SegNet.py

Potential list of models (Please edit this issue and add more or add them as a new comment):

• LeViT-UNet: https://sh-tsang.medium.com/review-levit-unet-make-faster-encoders-with-transformer-for-medical-image-segmentation-a5ce4ad22581
• Weakly supervised convolutional LSTM: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara-holoscan/resources/holoscan_endoscopy_sample_data google-citations
• Feedback U-Net with convolutional LSTM: https://link.springer.com/article/10.1007/s00371-021-02221-3 google-citations
• "Long Short-Term Memory (LSTM) model can record the relationship between elements in a spatial distance. This memory function can be realized by training LSTM model. But one disadvantage of LSTM model is that the order of memorizing information can only be from front to back. In order to better classify the types of nystagmus, this paper uses BiLSTM to solve this problem. BiLSTM is composed of two LSTMs with opposite directions" https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2023.1160904/full
• IMPROVED FEATURE PYRAMID NETWORK: "This method enables the
  feature pyramid network to more comprehensively extract
  pupil features and improve the network's adaptability to
  different eyes" > VISION TRANSFORMER > LSTM : > Zhang, Heming, and Changyuan Wang. "Integrated neural network-based pupil tracking technology for wearable gaze tracking devices in flight training." IEEE Access (2024). https://ieeexplore.ieee.org/abstract/document/10520295/
• This is a collection of our NAS and Vision Transformer work (EfficientViT, tinyViT) https://github.com/microsoft/Cream
• tinyViT https://github.com/wkcn/TinyViT?tab=readme-ov-file https://github.com/microsoft/Cream/tree/main/TinyViT
• MobileSAM: https://github.com/ChaoningZhang/MobileSAM
  • supports ONNX export
• FasterViT: Fast Vision Transformers with Hierarchical Attention:
  • https://arxiv.org/abs/2306.06189
  • https://github.com/NVlabs/FasterViT?tab=readme-ov-file
  • Supports ONNX Conversion
• [4.2.3. Hybrid ViT in both Encoder-Decoder]: MaxViT-UNet: Multi-Axis Attention for Medical Image Segmentation
  • https://github.com/PRLAB21/MaxViT-UNet?tab=readme-ov-file
• Fast Segment Anything
  • https://arxiv.org/pdf/2306.12156
  • https://github.com/CASIA-IVA-Lab/FastSAM
• Vision Transformer Segmentation with huggyface library: https://github.com/elcaiseri/Vision-Transformer-Segmentation
• More ViT in medical imaging are reviewd here https://arxiv.org/pdf/2312.00634
• EfficientViT-SAM (16-May-2024): https://arxiv.org/pdf/2402.05008
• Gaze Estimation using Transformer: https://github.com/yihuacheng/GazeTR/blob/main/model.py

[mxochicale]

Discussed with Zakaria that Vision Transformer from " Zhang, Heming, and Changyuan Wang. "Integrated neural network-based pupil tracking technology for wearable gaze tracking devices in flight training." IEEE Access (2024)" https://ieeexplore.ieee.org/abstract/document/10520295/ could be a good architecture to compare against our UNET baseline!