Support slicer type annotations in static analysis (Sphinx Autodoc and MyPy).

[allemangD]

With #6847 it is now possible to import MRML and other extension types directly from the slicer namespace, however static analysis tools and documentation generation still do not understand these.

Sphinx Autodoc in particular does not support any of mrml, slicer.logic, slicer.parameterNodeWrapper, vtkTeem, vtkAddon, or vtkITK as these all depend on content not available outside the Slicer context. (See RTFD mrml and slicer.logic are empty.)

MyPy fails on these same modules for the same reasons.

---

The only solutions I am aware of are either:

• Somehow invoke sphinx and mypy from within the Slicer context. Note that subprocess is not sufficient as the extension modules (MRML etc) will not be loaded.
  • I'm not certain this is even possible.
• Somehow generate type stub .pyi files for slicer, mrml, vtkTeem, vtkAddon, and vtkITK.
  • This is probably more effort, would require modifying the VTK python wrapping utilities.
  • This has the additional benefit of improving IDE support for inline documentation and code completion.

---

Related issues:

• Generate .pyi files for VTK classes #6688
• Generate .pyi files for VTK based classes #6689

[lassoan]

Somehow invoke sphinx and mypy from within the Slicer context.

This should be no problem, you can run sphinx from the Slicer application's Python console. You can also run code in the Slicer's application Python environment from the command line.

However, I agree that we should generate .pyi stubs (for many reasons). We can use the same method that is used in VTK - see here. We just need to run the script during the build and include the pyi files in the package.

[jcfr]

the extension modules (MRML etc) will not be loaded.

In prior experiments, we were able to successfully generate documentation from module like vtkITK, mrml, ... See https://www.slicer.org/wiki/Documentation/Labs/DocumentationImprovments#ReadTheDocs_.2F_GitHub_3

[jcfr]

Somehow generate type stub

I am now able to generate *.pyi files for all VTK modules (vtkITK, vtkAddon, MRML, Logic, ...) built in Slicer.

The remaining part is to consolidate these into a the appropriate set .pyi files matching the current package structure.

For example, I now have the file bin/MRMLCorePython.pyi along side bin/MRMLCorePython.so. That said, since all the classes from mrml are imported into the slicer package, I am not sure how to ensure the reference to the slicer.vtkMRMLNode type is found when generating the documentation. See #6876 (comment)

[jcfr]

For reference:

Partial content of bin/MRMLCorePython.pyi

from typing import overload, Any, Callable, TypeVar, Union

Callback = Union[Callable[..., None], None]
Buffer = TypeVar('Buffer')
Pointer = TypeVar('Pointer')
Template = TypeVar('Template')

import vtkmodules.vtkCommonCore

class SequenceFileType(int): ...

INVALID_SEQUENCE_FILE:'SequenceFileType'
METAIMAGE_SEQUENCE_FILE:'SequenceFileType'
NRRD_SEQUENCE_FILE:'SequenceFileType'

class vtkArchive(vtkmodules.vtkCommonCore.vtkObject):
    def GetNumberOfGenerationsFromBase(self, type:str) -> int: ...
    @staticmethod
    def GetNumberOfGenerationsFromBaseType(type:str) -> int: ...
    def IsA(self, type:str) -> int: ...
    @staticmethod
    def IsTypeOf(type:str) -> int: ...
    @staticmethod
    def ListArchive(archiveFileNameFileName:str, files:[str, ...]) -> bool: ...
    def NewInstance(self) -> 'vtkArchive': ...
    @staticmethod
    def SafeDownCast(o:'vtkObjectBase') -> 'vtkArchive': ...
    @staticmethod
    def UnZip(zipFileName:str, destinationDirectory:str) -> bool: ...
    @staticmethod
    def Zip(zipFileName:str, directoryToZip:str) -> bool: ...

class vtkCacheManager(vtkmodules.vtkCommonCore.vtkObject):
    CacheClearEvent:int
    CacheDeleteEvent:int
    CacheDirtyEvent:int
    CacheLimitExceededEvent:int
    CachedFile:int
    InsufficientFreeBufferEvent:int
    NoCachedFile:int
    OldCachedFile:int
    def AddCachePathToFilename(self, filename:str) -> str: ...
    def CacheSizeCheck(self) -> None: ...
    def CachedFileExists(self, filename:str) -> int: ...
    def ClearCache(self) -> int: ...
    def ClearCacheCheck(self) -> int: ...
    def ComputeCacheSize(self, dirname:str, size:int) -> float: ...
    def DeleteFromCache(self, target:str) -> None: ...
    def DeleteFromCachedFileList(self, target:str) -> None: ...
    def EncodeURI(self, uri:str) -> str: ...
    def FindCachedFile(self, target:str, dirname:str) -> str: ...
    def FreeCacheBufferCheck(self) -> None: ...
    def GetCachedFiles(self) -> (str, ...): ...
    def GetCurrentCacheSize(self) -> float: ...
    def GetEnableForceRedownload(self) -> int: ...
    def GetFileFromURIMap(self, uri:str) -> str: ...
    def GetFilenameFromURI(self, uri:str) -> str: ...
    def GetFreeCacheSpaceRemaining(self) -> float: ...
    def GetInsufficientFreeBufferNotificationFlag(self) -> int: ...
    def GetNumberOfGenerationsFromBase(self, type:str) -> int: ...
    @staticmethod
    def GetNumberOfGenerationsFromBaseType(type:str) -> int: ...
    def GetRemoteCacheDirectory(self) -> str: ...
    def GetRemoteCacheFreeBufferSize(self) -> int: ...
    def GetRemoteCacheLimit(self) -> int: ...
    def IsA(self, type:str) -> int: ...
    def IsLocalReference(self, uri:str) -> int: ...
    def IsRemoteReference(self, uri:str) -> int: ...
    @staticmethod
    def IsTypeOf(type:str) -> int: ...
    def LocalFileExists(self, uri:str) -> int: ...
    def MapFileToURI(self, uri:str, fname:str) -> None: ...
    def MarkNode(self, __a:str) -> None: ...
    def MarkNodesBeforeDeletingDataFromCache(self, __a:str) -> None: ...
    def NewInstance(self) -> 'vtkCacheManager': ...
    @staticmethod
    def SafeDownCast(o:'vtkObjectBase') -> 'vtkCacheManager': ...
    def SetCurrentCacheSize(self, _arg:float) -> None: ...
    def SetEnableForceRedownload(self, _arg:int) -> None: ...
    def SetInsufficientFreeBufferNotificationFlag(self, _arg:int) -> None: ...
    def SetMRMLScene(self, scene:'vtkMRMLScene') -> None: ...
    def SetRemoteCacheDirectory(self, dir:str) -> None: ...
    def SetRemoteCacheFreeBufferSize(self, _arg:int) -> None: ...
    def SetRemoteCacheLimit(self, _arg:int) -> None: ...
    def UpdateCacheInformation(self) -> None: ...

class vtkCodedEntry(vtkmodules.vtkCommonCore.vtkObject):
    def Copy(self, aEntry:'vtkCodedEntry') -> None: ...
    def GetAsPrintableString(self) -> str: ...
    def GetAsString(self) -> str: ...
    def GetCodeMeaning(self) -> str: ...
    def GetCodeValue(self) -> str: ...
    def GetCodingSchemeDesignator(self) -> str: ...
    def GetNumberOfGenerationsFromBase(self, type:str) -> int: ...
    @staticmethod
    def GetNumberOfGenerationsFromBaseType(type:str) -> int: ...
    def GetValueSchemeMeaning(self) -> (str, ...): ...
    def Initialize(self) -> None: ...
    def IsA(self, type:str) -> int: ...
    @staticmethod
    def IsTypeOf(type:str) -> int: ...
    def NewInstance(self) -> 'vtkCodedEntry': ...
    @staticmethod
    def SafeDownCast(o:'vtkObjectBase') -> 'vtkCodedEntry': ...
    def SetCodeMeaning(self, _arg:str) -> None: ...
    def SetCodeValue(self, _arg:str) -> None: ...
    def SetCodingSchemeDesignator(self, _arg:str) -> None: ...
    def SetFromString(self, content:str) -> bool: ...
    @overload
    def SetValueSchemeMeaning(self, value:str, scheme:str, meaning:str) -> None: ...
    @overload
    def SetValueSchemeMeaning(self, valueSchemeMeaning:(str, ...)) -> bool: ...

class vtkDataFileFormatHelper(vtkmodules.vtkCommonCore.vtkObject):
    def GetClassNameFromFormatString(self, fileformat:str) -> str: ...
    @staticmethod
    def GetFileExtensionFromFormatString(fileformat:str) -> str: ...
    def GetITKSupportedExtensionClassNameByIndex(self, idx:int) -> str: ...
    def GetITKSupportedExtensionGenericNameByIndex(self, idx:int) -> str: ...
    def GetITKSupportedReadFileFormats(self) -> 'vtkStringArray': ...
    def GetITKSupportedWriteFileExtensions(self) -> 'vtkStringArray': ...
    def GetITKSupportedWriteFileFormats(self) -> 'vtkStringArray': ...
    def GetNumberOfGenerationsFromBase(self, type:str) -> int: ...
    @staticmethod
    def GetNumberOfGenerationsFromBaseType(type:str) -> int: ...
    def IsA(self, type:str) -> int: ...
    @staticmethod
    def IsTypeOf(type:str) -> int: ...
    def NewInstance(self) -> 'vtkDataFileFormatHelper': ...
    @staticmethod
    def SafeDownCast(o:'vtkObjectBase') -> 'vtkDataFileFormatHelper': ...

[...]


class vtkMRMLNode(vtkmodules.vtkCommonCore.vtkObject):
    HierarchyModifiedEvent:int
    IDChangedEvent:int
    ReferenceAddedEvent:int
    ReferenceModifiedEvent:int
    ReferenceRemovedEvent:int
    ReferencedNodeModifiedEvent:int
    def AddAndObserveNodeReferenceID(self, referenceRole:str, referencedNodeID:str, events:'vtkIntArray'=...) -> 'vtkMRMLNode': ...
    def AddNodeReferenceID(self, referenceRole:str, referencedNodeID:str) -> 'vtkMRMLNode': ...
    def AddNodeReferenceRole(self, referenceRole:str, mrmlAttributeName:str=..., events:'vtkIntArray'=...) -> None: ...
    def AddToSceneOff(self) -> None: ...
    def AddToSceneOn(self) -> None: ...
    def Copy(self, node:'vtkMRMLNode') -> None: ...
    def CopyContent(self, node:'vtkMRMLNode', deepCopy:bool=True) -> None: ...
    def CopyReferences(self, node:'vtkMRMLNode') -> None: ...
    def CopyWithScene(self, node:'vtkMRMLNode') -> None: ...
    def CreateNodeInstance(self) -> 'vtkMRMLNode': ...
    def DisableModifiedEventOff(self) -> None: ...
    def DisableModifiedEventOn(self) -> None: ...
    def EndModify(self, previousDisableModifiedEventState:int) -> int: ...
    def GetAddToScene(self) -> int: ...
    def GetAttribute(self, name:str) -> str: ...
    @overload
    def GetAttributeNames(self) -> (str, ...): ...
    @overload
    def GetAttributeNames(self, attributeNames:'vtkStringArray') -> None: ...
    def GetContentModifiedEvents(self) -> 'vtkIntArray': ...
    def GetCustomModifiedEventPending(self, eventId:int) -> int: ...
    def GetDescription(self) -> str: ...
    def GetDisableModifiedEvent(self) -> int: ...
    def GetHideFromEditors(self) -> int: ...
    def GetID(self) -> str: ...
    def GetInMRMLCallbackFlag(self) -> int: ...
    def GetModifiedEventPending(self) -> int: ...
    def GetName(self) -> str: ...
    def GetNodeReference(self, referenceRole:str) -> 'vtkMRMLNode': ...
    def GetNodeReferenceID(self, referenceRole:str) -> str: ...
    def GetNodeReferenceRoles(self, roles:[str, ...]) -> None: ...
    def GetNodeTagName(self) -> str: ...
    def GetNthNodeReference(self, referenceRole:str, n:int) -> 'vtkMRMLNode': ...
    def GetNthNodeReferenceID(self, referenceRole:str, n:int) -> str: ...
    def GetNthNodeReferenceRole(self, n:int) -> str: ...
    def GetNumberOfGenerationsFromBase(self, type:str) -> int: ...
    @staticmethod
    def GetNumberOfGenerationsFromBaseType(type:str) -> int: ...
    def GetNumberOfNodeReferenceRoles(self) -> int: ...
    def GetNumberOfNodeReferences(self, referenceRole:str) -> int: ...
    def GetSaveWithScene(self) -> int: ...
    def GetScene(self) -> 'vtkMRMLScene': ...
    def GetSelectable(self) -> int: ...
    def GetSelected(self) -> int: ...
    def GetSingletonTag(self) -> str: ...
    def GetTypeDisplayName(self) -> str: ...
    def GetUndoEnabled(self) -> bool: ...
    def HasCopyContent(self) -> bool: ...
    def HasNodeReferenceID(self, referenceRole:str, referencedNodeID:str) -> bool: ...
    def HideFromEditorsOff(self) -> None: ...
    def HideFromEditorsOn(self) -> None: ...
    def InvokeCustomModifiedEvent(self, eventId:int, callData:Pointer=...) -> None: ...
    def InvokePendingModifiedEvent(self) -> int: ...
    def IsA(self, type:str) -> int: ...
    def IsSingleton(self) -> bool: ...
    @staticmethod
    def IsTypeOf(type:str) -> int: ...
    def Modified(self) -> None: ...
    def NewInstance(self) -> 'vtkMRMLNode': ...
    def OnNodeAddedToScene(self) -> None: ...
    def ProcessChildNode(self, __a:'vtkMRMLNode') -> None: ...
    def ProcessMRMLEvents(self, caller:'vtkObject', event:int, callData:Pointer) -> None: ...
    def RemoveAttribute(self, name:str) -> None: ...
    def RemoveNodeReferenceIDs(self, referenceRole:str) -> None: ...
    def RemoveNthNodeReferenceID(self, referenceRole:str, n:int) -> None: ...
    def Reset(self, defaultNode:'vtkMRMLNode') -> None: ...
    @staticmethod
    def SafeDownCast(o:'vtkObjectBase') -> 'vtkMRMLNode': ...
    def SaveWithSceneOff(self) -> None: ...
    def SaveWithSceneOn(self) -> None: ...
    def SelectableOff(self) -> None: ...
    def SelectableOn(self) -> None: ...
    def SelectedOff(self) -> None: ...
    def SelectedOn(self) -> None: ...
    def SetAddToScene(self, _arg:int) -> None: ...
    def SetAddToSceneNoModify(self, value:int) -> None: ...
    def SetAndObserveNodeReferenceID(self, referenceRole:str, referencedNodeID:str, events:'vtkIntArray'=...) -> 'vtkMRMLNode': ...
    def SetAndObserveNthNodeReferenceID(self, referenceRole:str, n:int, referencedNodeID:str, events:'vtkIntArray'=...) -> 'vtkMRMLNode': ...
    def SetAttribute(self, name:str, value:str) -> None: ...
    def SetDescription(self, _arg:str) -> None: ...
    def SetDisableModifiedEvent(self, onOff:int) -> None: ...
    def SetHideFromEditors(self, _arg:int) -> None: ...
    def SetInMRMLCallbackFlag(self, flag:int) -> None: ...
    def SetName(self, _arg:str) -> None: ...
    def SetNodeReferenceID(self, referenceRole:str, referencedNodeID:str) -> 'vtkMRMLNode': ...
    def SetNthNodeReferenceID(self, referenceRole:str, n:int, referencedNodeID:str) -> 'vtkMRMLNode': ...
    def SetSaveWithScene(self, _arg:int) -> None: ...
    def SetScene(self, scene:'vtkMRMLScene') -> None: ...
    def SetSceneReferences(self) -> None: ...
    def SetSelectable(self, _arg:int) -> None: ...
    def SetSelected(self, _arg:int) -> None: ...
    def SetSingletonOff(self) -> None: ...
    def SetSingletonOn(self) -> None: ...
    def SetSingletonTag(self, _arg:str) -> None: ...
    def SetUndoEnabled(self, _arg:bool) -> None: ...
    def StartModify(self) -> int: ...
    def URLDecodeString(self, inString:str) -> str: ...
    def URLEncodeString(self, inString:str) -> str: ...
    def UndoEnabledOff(self) -> None: ...
    def UndoEnabledOn(self) -> None: ...
    def UpdateReferenceID(self, oldID:str, newID:str) -> None: ...
    def UpdateReferences(self) -> None: ...
    def UpdateScene(self, __a:'vtkMRMLScene') -> None: ...
    def XMLAttributeDecodeString(self, inString:str) -> str: ...
    def XMLAttributeEncodeString(self, inString:str) -> str: ...

[...]

[lassoan]

Great progress! It'll be awesome to have these stubs.

It seems that the API documentation docstrings are missing from the pyi file. Is there an option to include them? They would be pretty important for the usability of the stubs.