Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions Documentation/docs/migration_guides/itk_6_migration_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -394,6 +394,74 @@ for thread in threads:
issues with callbacks, you can disable GIL release by setting `-DITK_PYTHON_RELEASE_GIL=OFF`
when building ITK.

Python lazy loading is always on
--------------------------------

The Python wrapping layer's lazy-loading mechanism has been rewritten on top
of the standard-library [PEP 562](https://peps.python.org/pep-0562/)
module-level `__getattr__` / `__dir__` protocol. Lazy first-touch resolution
is now the only mode of operation: there is no eager-import alternative.

### Removed

- `itkConfig.LazyLoading` — previously a boolean attribute on `itkConfig`
that, when set to `False`, forced every wrapped submodule to be imported
at `import itk` time.
- `ITK_PYTHON_LAZYLOADING` — the environment variable that seeded the value
of `itkConfig.LazyLoading` at startup.
- The custom `itk.support.lazy.LazyITKModule` `types.ModuleType` subclass
(and its `ITKLazyLoadLock`, `_lazy_itk_module_reconstructor`, and
`not_loaded` sentinel) that the previous implementation relied on.

### What you need to do

If your code or test suite contains either of the following, remove
them:

- `itkConfig.LazyLoading = False` — assignment alone is silently ignored
(module attributes are mutable, but no ITK code reads the value
anymore). On a fresh `import itkConfig` with no prior assignment, a
*read* of `itkConfig.LazyLoading` raises `AttributeError` because the
attribute is no longer defined. Delete both the writes and any reads:

```python
import itkConfig

# Remove any assignments — they are silently no-ops:
itkConfig.LazyLoading = False

# Remove any reads — without a prior assignment they raise
# AttributeError, since itkConfig no longer defines this attribute:
if itkConfig.LazyLoading:
...
```

- `ITK_PYTHON_LAZYLOADING` — the environment variable is no longer
consulted. Any export is silently ignored; remove it from launch
scripts and CI configurations:

```bash
ITK_PYTHON_LAZYLOADING=0 python my_script.py # drop the env var
```

The observable contract from a caller's point of view is otherwise
unchanged: `import itk` returns the package, attribute access on `itk`
or any `itk.<Submodule>` resolves on first touch, `dir(itk)` lists the
full set of lazily-discoverable attributes, and pickle / cloudpickle
round-trips through `sys.modules` continue to work.

### Why this changed

The previous `LazyITKModule.__getattribute__` intercepted *every* attribute
read on the `itk` package just to check a sentinel; the PEP 562 hook only
fires on a miss, so steady-state attribute access now hits the standard
module fast path. The legacy `ITKLazyLoadLock` also combined
`threading.RLock` and `multiprocessing.RLock`, and constructing the
multiprocessing half pinned the multiprocessing start method at
`import itk` time — breaking downstream callers that wanted to call
`multiprocessing.set_start_method(...)` after import. The replacement uses
only `threading.RLock`.

Modern CMake Interface Libraries
---------------------------------

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,6 @@
# limitations under the License.
#
# ==========================================================================
import itkConfig

itkConfig.LazyLoading = False
import itk
import numpy as np

Expand Down
4 changes: 2 additions & 2 deletions Wrapping/Generators/Python/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
## - itk/
## - __init__.py # cmake copied file
## - support/
## - itk(Extras|Template|Base|LazyLoading|...).py # static python files cmake copied
## - (extras|template_class|base|_lazy_submodule|...).py # static python files cmake copied
## - Configuration/
## -- ITK${MODULE_ITEM}Config.py # igenerator.py output config database index files for .so
## -- ITK${MODULE_ITEM}_snake_case.py # igenerator.py output config database index files for .so
Expand Down Expand Up @@ -185,7 +185,7 @@ if(NOT EXTERNAL_WRAP_ITK_PROJECT)
support/types
support/extras
support/xarray
support/lazy
support/_lazy_submodule
support/helpers
support/init_helpers
support/build_options
Expand Down
5 changes: 0 additions & 5 deletions Wrapping/Generators/Python/Tests/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -110,11 +110,6 @@ itk_python_add_test(
COMMAND
${CMAKE_CURRENT_SOURCE_DIR}/lazy.py
)
itk_python_add_test(
NAME PythonNoLazyModule
COMMAND
${CMAKE_CURRENT_SOURCE_DIR}/nolazy.py
)
itk_python_add_test(
NAME PythonNoDefaultFactories
COMMAND
Expand Down
37 changes: 32 additions & 5 deletions Wrapping/Generators/Python/Tests/lazy.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,45 @@
# limitations under the License.
#
# ==========================================================================
import itkConfig

# Override environmental variable default to force LazyLoading
itkConfig.LazyLoading = True
import itk

# Test PEP 366 compliance of LazyITKModule
# Test PEP 366 compliance of itk and per-submodule namespaces
assert itk.__package__ == "itk"
from itk import ITKCommon

assert ITKCommon.__package__ == "itk"

# PEP 562 __dir__: lazy attributes are visible to tab-completion and
# tooling without forcing a SWIG load. Only dunder attributes (which
# the lazy __getattr__ explicitly skips) have been touched up to this
# point, so the underlying SWIG modules have not been loaded yet --
# "Image" must show up in dir() but stay absent from the module's
# __dict__ until first access.
assert "Image" in dir(itk)
assert "Image" not in vars(itk)
assert "Image" in dir(ITKCommon)
assert "Image" not in vars(ITKCommon)

# Factory hook: under the default configuration, accessing a class
# whose module declares a needed factory (ITKIOImageBase -> ImageIO)
# must trigger the corresponding factory registration as a side effect
# of the lazy load. nodefaultfactories.py covers the disabled path.
import itkConfig

if itkConfig.DefaultFactoryLoading:
factories_before = len(itk.ObjectFactoryBase.GetRegisteredFactories())
_ = itk.ImageFileReader
factories_after = len(itk.ObjectFactoryBase.GetRegisteredFactories())
assert factories_after > factories_before

# stdlib pickle round-trip on a per-submodule namespace: the lazy
# submodule is registered in sys.modules and its instance __reduce_ex__
# resolves through importlib.import_module, so plain pickle (not just
# cloudpickle) must round-trip to the same instance.
import pickle

assert pickle.loads(pickle.dumps(itk.ITKCommon)) is itk.ITKCommon

# Test pickling used bash Dask
_has_cloudpickle: bool = False
try:
Expand Down
18 changes: 5 additions & 13 deletions Wrapping/Generators/Python/Tests/multiprocess_lazy_loading.py
Original file line number Diff line number Diff line change
@@ -1,35 +1,27 @@
#!/usr/bin/env python
# LazyLoading must be threadsafe
# Lazy module loading must be threadsafe
#
# The loading of modules in python *must* occur as a
# single atomic transaction in multiprocessing environments (i.e.
# the module should only be loaded by one thread).
#
# The LazyLoading of ITK did not treat the loading of
# Historically ITK's lazy loader did not treat the loading of
# modules as an atomic transaction, and multiple threads
# would attempt to load the cascading dependencies out of
# order.
#
# The `getattr` override that allows LazyLoading to work
# in the case where the module is *not* loaded now blocks
# while the first thread completes the delayed module loading.
# The PEP 562 ``__getattr__`` hook that triggers a delayed
# load now blocks while the first thread completes the load.
# After the first thread completes module load as an atomic
# transaction, the other threads fall through (skip loading)
# and return the value requested.
#
# Need to use a recursive lock for thread ownership so that the
# first thread can can acquire a RLock as often as needed while
# recursively processing dependent modules lazy loads. Other threads need
# recursively processing dependent module loads. Other threads need
# to wait until this first thread releases the RLock.


# NOTE: This test requires itkConfig.LazyLoading=True
# Explicitly set to override potential environmental
# variable settings.
import itkConfig

itkConfig.LazyLoading = True

from multiprocessing.pool import ThreadPool
from multiprocessing import cpu_count

Expand Down
46 changes: 0 additions & 46 deletions Wrapping/Generators/Python/Tests/nolazy.py

This file was deleted.

Loading
Loading