Problem of missing classes

[YichiZhang98]

Hi, thanks for your nice work! However, I have several questions about adapting this framework to new datasets concerning the problem of missing classes. For example, if I want to segment 4 organs. However, for some images, there only exists 3 organs, (the other organ may be excised and not exist in this image).

1. When we use images with missing classes as support image, it will raise an error due to the missing class.
2. When we inference images with missing classes, the output of MedLAM for this class seems to have no meaning since this target does not exist.

Do you have any suggestions?

Best regards.

[LWHYC]

Hi, thanks for your nice work! However, I have several questions about adapting this framework to new datasets concerning the problem of missing classes. For example, if I want to segment 4 organs. However, for some images, there only exists 3 organs, (the other organ may be excised and not exist in this image).

1. When we use images with missing classes as support image, it will raise an error due to the missing class.
2. When we inference images with missing classes, the output of MedLAM for this class seems to have no meaning since this target does not exist.

Do you have any suggestions?

Best regards.

Hi,
thanks for your interest in MedLSAM!

1. Unfortunately, it's not possible to use support images without all the interested classes in our current MedLAM framework.
2. Your observation is correct. In MedLAM, localization involves two steps: Unified Anatomical Mapping (UAM) and Multi Scale Similarity (MSS). UAM moves initial random points by calculating relative positions, so we can determine if the movement goes beyond the image itself. We plan to add this check in future updates to ascertain if a class is missing.

Best.

[Z-Hd]

Does this mean that each support image must include all the labels? This is very difficult to achieve for the segmentation of spinal vertebrae. Is there a way to make the images in the support set cover all the labels, without the need for each image to contain all the labels?

Best.