Scale and Position of Objects Differ between OBJ vs DICOM/NRRD

[eldrickm]

First of all, a deep thank you for working on this project. It works wonderfully and has been extremely helpful. I am planning on donating on your behalf to the ICRC.

I am working on aligning an .obj file of an object I am exporting out of 3D Slicer with it's DICOM / NRRD representation. I am exporting each of these files directly from the same data set in 3D Slicer.

However, I am observing that there are differences in the scale, rotation, and position of the origin across both imports. For example, I have exported a cuboid object from 3D slicer (Figure 1) - the peach-colored box corresponds to the OBJ file to be exported from a segmentation (Figure 2). Notice that they are basically superimposed on each other in 3D slicer.

When I import the OBJ file, all is as expected. (Figure 3 - I've re-oriented it so it faces up)

But when I import the NRRD (DICOM is also affected) as a rendered volume (Fig. 4) I observe the following:

1. There is a scale mismatch (see the dramatic size difference)
2. The position of both objects differ. The OBJ file has the correct origin relative to the object - near one of the sides, while the rendered volume's origin seems to be placed somewhere in the center of the geometry (see the difference in the coordinate axes of both objects. I am showing the coordinate axes at the pivot of both game objects). I have offset the two geometries so that it can be more easily seen.
3. Rotation seems to be unaffected - I can manually adjust only the position and scale and have the two objects align.
4. The scale and position values to correctly align the two object can be different between datasets. I've tested it with two datasets trying to see if the scale / position offsets would be constant.

Could you help me debug or find the issue? Thank you kindly in advance!

Here is a ZIP of the two files: https://drive.google.com/file/d/1z6vPmmyuQD5B45JExmGrz2bEbyVzrXdP/view?usp=sharing

[mlavik1]

Hi @eldrickm !
Thanks for the kind words, I'm very happy to hear that :)
When I wrote the importers I simply assumed that you would wouldn't need the relative sizes between multiple imported datasets to be right (originally you could only import one 😅 ), so I simply stored the imported dataset inside of a (1,1,1) sized box, and then scale the axes to make the the dimension right for datasets with a non-uniform dimension.
So the real size is ignored, which I realise is a big limitation.

The importers seem set the scale correctly (I think?):

• SimpleITKImageFileImporter.cs (for NRRD files)
• SimpleITKImageSequenceImporter.cs (for DICOM files)

However, the VolumeObjectFactory.CreateObject function then scales down the dataset to make sure all imported datasets have more or less the same size.

if(dataset.scaleX != 0.0f && dataset.scaleY != 0.0f && dataset.scaleZ != 0.0f)
{
    float maxScale = Mathf.Max(dataset.scaleX, dataset.scaleY, dataset.scaleZ);
    volObj.transform.localScale = new Vector3(dataset.scaleX / maxScale, dataset.scaleY / maxScale, dataset.scaleZ / maxScale);
}

(see

UnityVolumeRendering/Assets/Scripts/VolumeObject/VolumeObjectFactory.cs

Line 45 in 70ed1d1

if(dataset.scaleX != 0.0f && dataset.scaleY != 0.0f && dataset.scaleZ != 0.0f)

)

I suppose you could try to remove that / maxScale ?
There might still be some other issues too.. And we probably need to handle different scale units? (mm, etc.)

I can also take a look at it when I have time (this weekend or early next week I think! Getting some visitors first :) )

[eldrickm]

Thank you for the prompt reply Matias!

Yes - it looks like by removing the normalization, the true size is actually preserved (after scaling down by appropriate units!). For example, for our dataset, the true size is given by the result of the following:

UnityVolumeRendering/Assets/Scripts/Importing/ImageSequenceImporter/SimpleITK/SimpleITKImageSequenceImporter.cs

Line 121 in 70ed1d1

volumeDataset.scaleX = (float)(spacing[0] * size[0]);

Then I just have to divide by 1000 to get appropriate scaling (mm vs m issue).

I am now investigating how to correct for positioning - it seems that one can use the metadata in the NRRD (the space_origin field) or the DICOM (the image_position field aka (0020, 0032) tag) to derive this but I'm not exactly sure how to use that data. I would appreciate any hints when you have the time! Thank you again in advance 🙌

[mlavik1]

That's great to hear! :)

So for that the relevant file to look in would be SimpleITKImageSequenceImporter.cs.
It doesn't seem like SimpleITK has a property for image position though.. But I think you can read DICOM attributes by tag, using the GetMetaData function.
There is something example code here: https://simpleitk.readthedocs.io/en/master/link_DicomImagePrintTags_docs.html
(if you print the tags like that, you can see how they need to be formatted. I think it's "0020|0032").

Please let me know if you get that to work, or if you have any further questions or issues :)

[eldrickm]

Yes I was able to read the tags using the OpenDICOM importer - in fact it already takes it into account!

UnityVolumeRendering/Assets/Scripts/Importing/ImageSequenceImporter/OpenDICOM/DICOMImporter.cs

Line 188 in 70ed1d1

Tag posTag = new Tag("(0020,0032)");

My question is how to use that metadata in positioning the volume render game object. Do you know the relationship between the coordinates that are given in that metadata field and say, the transform.localPosition of the volume render game object?

[mlavik1]

Oh right, you're using the OpenDICOM importer :)
Hmm.. I guess you could use the position of the first slice then?

We sort the slices here:

UnityVolumeRendering/Assets/Scripts/Importing/ImageSequenceImporter/OpenDICOM/DICOMImporter.cs

Line 122 in 70ed1d1

files.Sort((DICOMSliceFile a, DICOMSliceFile b) => { return a.location.CompareTo(b.location); });

After that, you could fetch the location of the first slice, and maybe use that as position? (that is, files[0].location)

[mlavik1]

Hmm I noticed that the location is a float though (the X value, I suppose).
You should probably read the X,Y,Z value as well then. The X value is read here:

UnityVolumeRendering/Assets/Scripts/Importing/ImageSequenceImporter/OpenDICOM/DICOMImporter.cs

Line 198 in 70ed1d1

slice.location = (float)Convert.ToDouble(elemLoc.Value[0]);

So simply change that to read elemLoc.Value[0], elemLoc.Value[1], elemLoc.Value[2] to a Vector3.

By the way, if you're on Windows or Linux I would recommend using the newer SimpleITK importer for DICOM btw :) https://github.com/mlavik1/UnityVolumeRendering/blob/master/Documentation/SimpleITK.md
Though, if your project is more cross platform, then I guess OpenDICOM is the only choice.

[eldrickm]

Hi Matias,

Sorry for the delay in response!

Yes, that was my thought as well, however I am not sure which game object in particular to assign the value to.
Each slice also has its own location, so I was not sure which slice's location to use either. Do you have any insight on the relationship between the slice's location numbers and the positioning of the game object as a whole?

[mlavik1]

Hi again!
Sorry for the delay. Busy week 😅

Hmm.. I'm not too sure about that, to be honest.
I guess the code for reading the slice locations would need to be changed a bit if you need the relative positions between the objects to be correct.

According to this the slice locations need to be interpreted according to another DICOM tag: Image Orientation (0020,0037)?

[eldrickm]

Thank you for the link - I'll take a closer look and will see if I can get a solution within the next few weeks!

[mlavik1]

Great, thanks! :)