maisi unconditioned inference return noise

[LFetty]

Hallo,

I tried longer to make the diffusion unet (rectified flow version) work but without success. the VAE performes correct and encoding and reconstruction look fine. I take all the configs from the repo so configuration as well as weights are equivalently loaded as in https://github.com/Project-MONAI/tutorials/blob/main/generation/maisi/maisi_inference_tutorial.ipynb.

Using the diffusion unet with the existing code of diffusion_model_infer.py with

from scripts.diff_model_infer import run_inference
data = run_inference(
  args,
  device,
  autoencoder,
  diffusion_unet,
  1.4,#scale_factor,
  None,#top_region_index_tensor,
  None, #bottom_region_index_tensor,
  torch.tensor([[1.5,1.5,4.0]]).to('cuda'),#spacing_tensor,
  torch.tensor([1]).to('cuda'),#modality_tensor,
  (96,96,96),#output_size,
  4,
  logger,
)

I saw that there was an issue with the architecture which was different during training and the implementation but I'm not sure if that is related.
Project-MONAI/MONAI#7991.
Some observations I made is when I would like to just reconstruction an image and basically make one step, the recon get's extremly noise.

For any suggestion what could be the potential issue, I'm thankful.

Best,
Lukas

[Can-Zhao]

scale_factor is loaded from checkpoint. output_size can not be [96,96,96]. It needs to be divisible by 128

[LFetty]

Thanks for the quick answer. I indeed tested more and came also to the conclusion that higher output resolution helps a lot! Sorry I did overread that information in the paper. However, using the smallest possible output size of 128x128x128 have still very bad results. I guess the reason is that there are only very few training cases with that resolution level.

In the paper I did not find info about what the training resolution of the diffusion unet was. As the resolution would be very difference for all volumes was the latent representation cropped to a specific size?