[MAISI] The Role of FOV at inference time using finetuned Diffusion Net

[alejandrodl]

I'm sharing here an interesting issue my team and I just had, along with its solution, which was kindly provided by the MAISI team.

We tried finetuning the MAISI’s Diffusion Net model on a dataset of 221 medical volumes that did not belong to the original model's training data. We ran the training with the tutorial notebook provided and after that, when we ran inference, the generated images seemed to collapse into noise:

In the end, the model happened to be finetuned correctly, only that the FOV (Field Of View) at inference was different from the one at training time, hence the mismatch. When adjusted correctly, we got appropriate generation results:

This magnitude FOV = dimensions * spacing at inference has to fall within the range of that of training samples, otherwise the generations would be incorrectly computed. In order to adjust inference parameters accordingly, one must:

1. Pick a random training image
2. Round dimensions dim to numbers divisible by 128 (e.g. 239x221x137 -> 256x256x128). This is new_dim parameter.
3. Get spacing (a.k.a voxel size) data from the medical volume (e.g. 0.66 x 0.86 x 0.92)
4. Compute new_spacing = spacing * (dim/new_dim)
5. Inside config_maisi_diff_model.json file, change dim variable to new_dim and spacing to new_spacing .

If one runs inference with these two new parameters, generations should be correct.

P.S. Thanks a lot to the MAISI team, keep rocking! :)

[Can-Zhao]

Thank you for the sharing!!!