Replies: 1 comment
-
Hi there, This seems possible to me. The new dataloader shouldn't be necessary, we could define dataset transforms which take a 3D volume and extracts 2D slices from it. Something along the lines of the example here where the Adding temporal components to the DDPM and the autoencoder should be fairly doable, too. Do you have any interest in having a go at implementing this? Mark |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
First of all thanks for that great library, bringing generative models to the monai framework and also for the nice tutorials!
I was wondering if it would be possible to add the implementation described by Blattmann et al. in their paper "Align your Latents:
High-Resolution Video Synthesis with Latent Diffusion Models" (https://arxiv.org/abs/2304.08818) (see also https://research.nvidia.com/labs/toronto-ai/VideoLDM/), where they, as far as I understand, trained for the sake of efficiency a pre-trained 2D Autoencoder that was fine-tuned on a temporal dimension (for video-generation, which would be the z-dimension in medical images) by adding 3D layers to the decoder.
For GenerativeModels-integration, I was thinking of
Maybe, this approach could also help to generate synthetic medical 3D datasets in a diagnostic resolution.
Looking forward to hear your opinion on this topic.
Best, Lorenz
Beta Was this translation helpful? Give feedback.
All reactions