Extensions. Registered subclasses fail isinstance() checks with their parent classes

[h-mayorquin]

When creating NWB extension classes with inheritance, Python's standard isinstance() and issubclass() checks fail to recognize the inheritance relationship defined in the schema. In ndx-anatomical-localization, we define a base Space class and a subclass AllenCCFv3Space that inherits from Space in the schema specification. However, after loading the extension, isinstance(allen_space, Space) returns False even when allen_space is an AllenCCFv3Space instance. This causes constructor calls to fail with errors like "incorrect type for 'space' (got 'AllenCCFv3Space', expected 'Space')" when we try to pass a subclass instance to a method that accepts the parent class type. We believe this is related to how docval performs type checking, but it fundamentally breaks normal Python inheritance patterns and polymorphism for extension classes.

As a workaround, we are currently explicitly listing every possible subclass by name in the type specification: {"name": "space", "type": (Space, "AllenCCFv3Space"), ...} (example). While this works it would be good to fix this bug. We plan to add multiple canonical space classes for different official brain atlases (Allen Mouse CCF, Allen Human, Waxholm, MNI, etc.), and requiring every method that accepts a Space to explicitly list all current and future subclasses is error-prone and creates a maintenance burden. Each time we add a new atlas space, we would need to update the type specifications across the entire codebase.

To reproduce this issue, checkout the branch fix_schema_issue in ndx-anatomical-localization where the workaround has been removed from AnatomicalCoordinatesImage.__init__ and run:

from ndx_anatomical_localization import Space, AllenCCFv3Space, AnatomicalCoordinatesImage
from pynwb.image import Image
import numpy as np

allen_space = AllenCCFv3Space()

# This should return True but returns False
print(isinstance(allen_space, Space))  # False

# This fails with: "incorrect type for 'space' (got 'AllenCCFv3Space', expected 'Space')"
image = AnatomicalCoordinatesImage(
    name="test",
    space=allen_space,  # Should be accepted since AllenCCFv3Space inherits from Space
    method="test",
    image=Image(name="img", data=np.random.rand(10, 10)),
    x=np.random.rand(10, 10),
    y=np.random.rand(10, 10),
    z=np.random.rand(10, 10),
)

The error raised is:

packages/hdmf/utils.py", line 660, in _check_args
    raise ExceptionType(msg)
TypeError: AnatomicalCoordinatesImage.__init__: incorrect type for 'space' (got 'AllenCCFv3Space', expected 'Space')

[oruebel]

Looking at the code, I think this is a problem with how the classes are being constructed in the extension, rather than an issue of the get_class itself:

1. AllenCCFv3Space inherits from TempAllenCCFv3Space here

@register_class("AllenCCFv3Space", "ndx-anatomical-localization")
class AllenCCFv3Space(TempAllenCCFv3Space)

2. TempAllenCCFv3Space in turn is autogenerate via get_class here

TempAllenCCFv3Space = get_class("AllenCCFv3Space", "ndx-anatomical-localization")

3. But Space is then redefined later as an explicitly generated class here

@register_class("Space", "ndx-anatomical-localization")
class Space(TempSpace):

I.e., I think what you are creating here are two separate inheritance paths, so I think your class inheritance actually look like this right now:

                   TempSpace
                        |
                        |
          +-------------+-------------+
          |                           |
          |                           |
          v                           v
TempAllenCCFv3Space                 Space
          |
          |
          v
    AllenCCFv3Space

As such, you would actually need to check for TempSpace as your common base class.

I am not sure whether the @register_class("Space", "ndx-anatomical-localization") overwrites the autogenerated TempSpace class in the registry such that get_class would see it, but I think it should.If it does, then what should work is if you move this part here

TempAllenCCFv3Space = get_class("AllenCCFv3Space", "ndx-anatomical-localization")
TempAnatomicalCoordinatesTable = get_class("AnatomicalCoordinatesTable", "ndx-anatomical-localization")
Localization = get_class("Localization", "ndx-anatomical-localization")
TempAnatomicalCoordinatesImage = get_class("AnatomicalCoordinatesImage", "ndx-anatomical-localization")

to appear after you define your own Space class. I think this could potentially fix the issue, because then get_class would at least have a chance to see your custom Space class in the registry, but in the way the code is currently, TempAllenCCFv3Space = get_class("AllenCCFv3Space", "ndx-anatomical-localization") has no way of knowing about your custom Space class because that class is not being created until later in the code.

[h-mayorquin]

Thanks @oruebel, it was a simple as this.