Skip to content

shared location for different jit builds #2397

@wyli

Description

@wyli

Describe the bug
building different versions of the _extensions currently uses the same module name,
This can cause issues of incompatible compiled modules.

log:
https://github.com/Project-MONAI/MONAI/runs/2870435032?check_suite_focus=true#step:6:9618

======================================================================
ERROR: test_cuda_3_1_batches_3_dimensions_1_channels_2_classes_1_mixtures (tests.test_gmm.GMMTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/site-packages/parameterized/parameterized.py", line 533, in standalone_func
    return func(*(a + p.args), **p.kwargs)
  File "/__w/MONAI/MONAI/tests/test_gmm.py", line 323, in test_cuda
    gmm = GaussianMixtureModel(features_tensor.size(1), mixture_count, class_count)
  File "/__w/MONAI/MONAI/monai/networks/layers/gmm.py", line 42, in __init__
    "gmm", {"CHANNEL_COUNT": channel_count, "MIXTURE_COUNT": mixture_count, "MIXTURE_SIZE": mixture_size}
  File "/__w/MONAI/MONAI/monai/_extensions/loader.py", line 86, in load_module
    verbose=verbose_build,
  File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 952, in load
    keep_intermediates=keep_intermediates)
  File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1168, in _jit_compile
    return _import_module_from_library(name, build_directory, is_python_module)
  File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1493, in _import_module_from_library
    return imp.load_module(module_name, file, path, description)
  File "/opt/conda/lib/python3.6/imp.py", line 243, in load_module
    return load_dynamic(name, filename, file)
  File "/opt/conda/lib/python3.6/imp.py", line 343, in load_dynamic
    return _load(spec)
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 658, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 571, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 922, in create_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: /github/home/.cache/torch_extensions/gmm_1_2_1/gmm_1_2_1.so: undefined symbol: _ZN2at5emptyEN3c108ArrayRefIlEERKNS0_13TensorOptionsENS0_8optionalINS0_12MemoryFormatEEE

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions