Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing foundation models #7771

Open
mikelitu opened this issue May 14, 2024 · 0 comments
Open

Implementing foundation models #7771

mikelitu opened this issue May 14, 2024 · 0 comments

Comments

@mikelitu
Copy link

mikelitu commented May 14, 2024

Is your feature request related to a problem? Please describe.
I have been using MONAI for my project the last few months, the main problem is I need to use publicly available foundation models such as RAFT or MiDaS to generate intermidiate inputs for my models. This makes the implementation of such networks a bit tricky as you have to stick to certain input requirements (size, normalization, etc.) and implement those outside the MONAI bundles.

Describe the solution you'd like
Implementing a set of foundation models for MONAI. For my own application, I need depth estimation and optical flow estimation. However, I think it will benefitial to add models for segmentation and object detection also. I think there are two possible solutions:

  1. Create native foundation models in MONAI for these tasks.
  2. "Porting" some of the general foundation models available in Pytorch such as RAFT or MiDaS and implement them inside the MONAI framework.

Additional context
I'd like to use this post as a discussion about the implementation of such foundation models. I am more than happy to participate on the process, as it would massively benefit my own work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant