Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build with Module_GenericLabelInterpolator:BOOL=ON while avoiding implicit download #1157

Open
TheChymera opened this issue Aug 13, 2019 · 30 comments

Comments

@TheChymera
Copy link

commented Aug 13, 2019

I would like to build ITK with the module. Setting it in the build system attempts to implicitly download it during install. I would like to avoid this, so as to best ensure that only files explicitly known to be needed by the package manager will be downloaded.

�[32;01m * �[39;49;00mPackage:    sci-libs/itk-5.0.1
�[32;01m * �[39;49;00mRepository: chymeric
�[32;01m * �[39;49;00mMaintainer: maintainer-wanted@gentoo.org sci-biology@gentoo.org
�[32;01m * �[39;49;00mUSE:        abi_x86_64 amd64 elibc_glibc kernel_linux python_single_target_python3_6 python_targets_python2_7 python_targets_python3_6 userland_GNU
�[32;01m * �[39;49;00mFEATURES:   network-sandbox preserve-libs sandbox userpriv usersandbox
>>> Unpacking source...
>>> Unpacking InsightToolkit-5.0.1.tar.gz to /var/tmp/portage/sci-libs/itk-5.0.1/work
>>> Source unpacked in /var/tmp/portage/sci-libs/itk-5.0.1/work
>>> Preparing source in /var/tmp/portage/sci-libs/itk-5.0.1/work/InsightToolkit-5.0.1 ...
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/sci-libs/itk-5.0.1/work/InsightToolkit-5.0.1 ...
 �[33;01m*�[0m cmake-utils_src_prepare has not been run, please open a bug on https://bugs.gentoo.org/
>>> Working in BUILD_DIR: "/var/tmp/portage/sci-libs/itk-5.0.1/work/itk-5.0.1_build"
cmake -C /var/tmp/portage/sci-libs/itk-5.0.1/work/itk-5.0.1_build/gentoo_common_config.cmake -G Unix Makefiles -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=ON -DITK_USE_SYSTEM_DCMTK=ON -DGDCM_USE_SYSTEM_OPENJPEG=ON -DITK_USE_SYSTEM_DOUBLECONVERSION=ON -DITK_USE_SYSTEM_CASTXML=ON -DITK_USE_SYSTEM_HDF5=ON -DITK_USE_SYSTEM_JPEG=ON -DITK_USE_SYSTEM_PNG=ON -DITK_USE_SYSTEM_SWIG=ON -DITK_USE_SYSTEM_TIFF=ON -DITK_USE_SYSTEM_ZLIB=ON -DITK_USE_KWSTYLE=OFF -DITK_BUILD_DEFAULT_MODULES=ON -DITK_COMPUTER_MEMORY_SIZE=1 -DWRAP_ITK_JAVA=OFF -DWRAP_ITK_TCL=OFF -Ddouble-conversion_INCLUDE_DIRS=/usr/include/double-conversion -DExternalData_OBJECT_STORES=/var/tmp/portage/sci-libs/itk-5.0.1/work/InsightToolkit-5.0.1/.ExternalData -DModule_GenericLabelInterpolator:BOOL=ON -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DITK_USE_REVIEW=OFF -DITK_INSTALL_LIBRARY_DIR=lib64 -DITK_WRAP_PYTHON=OFF -DCMAKE_BUILD_TYPE=Gentoo -DCMAKE_TOOLCHAIN_FILE=/var/tmp/portage/sci-libs/itk-5.0.1/work/itk-5.0.1_build/gentoo_toolchain.cmake  /var/tmp/portage/sci-libs/itk-5.0.1/work/InsightToolkit-5.0.1
loading initial cache file /var/tmp/portage/sci-libs/itk-5.0.1/work/itk-5.0.1_build/gentoo_common_config.cmake
-- The CXX compiler identification is GNU 9.1.0
-- The C compiler identification is GNU 9.1.0
-- Check for working CXX compiler: /usr/bin/x86_64-pc-linux-gnu-g++
-- Check for working CXX compiler: /usr/bin/x86_64-pc-linux-gnu-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/bin/x86_64-pc-linux-gnu-gcc
-- Check for working C compiler: /usr/bin/x86_64-pc-linux-gnu-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found Git: /usr/bin/git (found version "2.22.0") 
-- Found PythonInterp: /var/tmp/portage/sci-libs/itk-5.0.1/temp/python3.6/bin/python (found version "3.6.9") 
-- Performing Test C_HAS_WARNING-Wno-uninitialized
-- Performing Test C_HAS_WARNING-Wno-uninitialized - Success
-- Performing Test C_HAS_WARNING-Wno-unused-parameter
-- Performing Test C_HAS_WARNING-Wno-unused-parameter - Success
-- Performing Test C_HAS_WARNING-Wall
-- Performing Test C_HAS_WARNING-Wall - Success
-- Performing Test C_HAS_WARNING-Wno-long-double
-- Performing Test C_HAS_WARNING-Wno-long-double - Failed
-- Performing Test C_HAS_WARNING-Wcast-align
-- Performing Test C_HAS_WARNING-Wcast-align - Success
-- Performing Test C_HAS_WARNING-Wdisabled-optimization
-- Performing Test C_HAS_WARNING-Wdisabled-optimization - Success
-- Performing Test C_HAS_WARNING-Wextra
-- Performing Test C_HAS_WARNING-Wextra - Success
-- Performing Test C_HAS_WARNING-Wformat_2
-- Performing Test C_HAS_WARNING-Wformat_2 - Success
-- Performing Test C_HAS_WARNING-Winvalid-pch
-- Performing Test C_HAS_WARNING-Winvalid-pch - Success
-- Performing Test C_HAS_WARNING-Wno-format-nonliteral
-- Performing Test C_HAS_WARNING-Wno-format-nonliteral - Success
-- Performing Test C_HAS_WARNING-Wpointer-arith
-- Performing Test C_HAS_WARNING-Wpointer-arith - Success
-- Performing Test C_HAS_WARNING-Wshadow
-- Performing Test C_HAS_WARNING-Wshadow - Success
-- Performing Test C_HAS_WARNING-Wunused
-- Performing Test C_HAS_WARNING-Wunused - Success
-- Performing Test C_HAS_WARNING-Wwrite-strings
-- Performing Test C_HAS_WARNING-Wwrite-strings - Success
-- Performing Test C_HAS_WARNING-funit-at-a-time
-- Performing Test C_HAS_WARNING-funit-at-a-time - Success
-- Performing Test C_HAS_WARNING-Wno-strict-overflow
-- Performing Test C_HAS_WARNING-Wno-strict-overflow - Success
-- Performing Test CXX_HAS_WARNING-Wall
-- Performing Test CXX_HAS_WARNING-Wall - Success
-- Performing Test CXX_HAS_WARNING-Wno-long-double
-- Performing Test CXX_HAS_WARNING-Wno-long-double - Failed
-- Performing Test CXX_HAS_WARNING-Wcast-align
-- Performing Test CXX_HAS_WARNING-Wcast-align - Success
-- Performing Test CXX_HAS_WARNING-Wdisabled-optimization
-- Performing Test CXX_HAS_WARNING-Wdisabled-optimization - Success
-- Performing Test CXX_HAS_WARNING-Wextra
-- Performing Test CXX_HAS_WARNING-Wextra - Success
-- Performing Test CXX_HAS_WARNING-Wformat_2
-- Performing Test CXX_HAS_WARNING-Wformat_2 - Success
-- Performing Test CXX_HAS_WARNING-Winvalid-pch
-- Performing Test CXX_HAS_WARNING-Winvalid-pch - Success
-- Performing Test CXX_HAS_WARNING-Wno-format-nonliteral
-- Performing Test CXX_HAS_WARNING-Wno-format-nonliteral - Success
-- Performing Test CXX_HAS_WARNING-Wpointer-arith
-- Performing Test CXX_HAS_WARNING-Wpointer-arith - Success
-- Performing Test CXX_HAS_WARNING-Wshadow
-- Performing Test CXX_HAS_WARNING-Wshadow - Success
-- Performing Test CXX_HAS_WARNING-Wunused
-- Performing Test CXX_HAS_WARNING-Wunused - Success
-- Performing Test CXX_HAS_WARNING-Wwrite-strings
-- Performing Test CXX_HAS_WARNING-Wwrite-strings - Success
-- Performing Test CXX_HAS_WARNING-funit-at-a-time
-- Performing Test CXX_HAS_WARNING-funit-at-a-time - Success
-- Performing Test CXX_HAS_WARNING-Wno-strict-overflow
-- Performing Test CXX_HAS_WARNING-Wno-strict-overflow - Success
-- Performing Test CXX_HAS_WARNING-Wno-deprecated
-- Performing Test CXX_HAS_WARNING-Wno-deprecated - Success
-- Performing Test CXX_HAS_WARNING-Wno-invalid-offsetof
-- Performing Test CXX_HAS_WARNING-Wno-invalid-offsetof - Success
-- Performing Test CXX_HAS_WARNING-Wno-undefined-var-template
-- Performing Test CXX_HAS_WARNING-Wno-undefined-var-template - Failed
-- Performing Test CXX_HAS_WARNING-Woverloaded-virtual
-- Performing Test CXX_HAS_WARNING-Woverloaded-virtual - Success
-- Performing Test CXX_HAS_WARNING-Wstrict-null-sentinel
-- Performing Test CXX_HAS_WARNING-Wstrict-null-sentinel - Success
-- Performing Test C_HAS_WARNING-mtune_native
-- Performing Test C_HAS_WARNING-mtune_native - Success
-- Performing Test C_HAS_WARNING-march_corei7
-- Performing Test C_HAS_WARNING-march_corei7 - Success
-- Performing Test CXX_HAS_WARNING-mtune_native
-- Performing Test CXX_HAS_WARNING-mtune_native - Success
-- Performing Test CXX_HAS_WARNING-march_corei7
-- Performing Test CXX_HAS_WARNING-march_corei7 - Success
-- Performing Test have_gold
-- Performing Test have_gold - Success
-- Performing Test CXX_HAS_DISABLE_OPTIMIZATION_FLAG
-- Performing Test CXX_HAS_DISABLE_OPTIMIZATION_FLAG - Failed
CMake Error at CMake/ITKModuleRemote.cmake:12 (message):
  Failed to clone repository:
  'https://github.com/InsightSoftwareConsortium/ITKGenericLabelInterpolator.git'
Call Stack (most recent call first):
  CMake/ITKModuleRemote.cmake:134 (_git_clone)
  CMake/ITKModuleRemote.cmake:197 (_fetch_with_git)
  Modules/Remote/GenericLabelInterpolator.remote.cmake:1 (itk_fetch_module)
  Modules/Remote/CMakeLists.txt:6 (include)


-- Configuring incomplete, errors occurred!

What would be the best way to go about this?
Should I download and unpack some ITKGenericLabelInterpolator tarball explicitly? If so, then to where?
If not, perhaps the ITKGenericLabelInterpolator can be built separately and installed into the appropriate path. But I do not see how this can be done, since the module README only lists the ITK CMake flag as an install instruction.

@gdevenyi

This comment has been minimized.

Copy link
Contributor

commented Aug 13, 2019

@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

@gdevenyi I assume that, being a module, this software has no use cases which are independent of ITK. Is that correct?

@gdevenyi

This comment has been minimized.

Copy link
Contributor

commented Aug 13, 2019

Well, you could in theory use just this modules functions in writing a C++ program where everything else is hand coded or uses a different library, but this module still needs to be built against ITK, either internally via the module option, or externally by calling cmake appropriately.

@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

@gdevenyi Is there any particular commit of ITKGenericLabelInterpolator which is considered in sync with ITK 5.0.1 ? I also find it suspicious, that the ITKGenericLabelInterpolator CMake system specifies ITK 4.9 --- is this a problem?

@dzenanz

This comment has been minimized.

Copy link
Member

commented Aug 13, 2019

The in-sync version of remote modules can be found in their definition files, e.g. GenericLabelInterpolator.remote.cmake. Unzipping a tarball in ITK-Source/Modules/Remote/ might work in conjunction with ITK_FORBID_DOWNLOADS. Here is how directory structure might look like:

Dzenan@Ryzenator MINGW64 /c/Libs/ITK-5.0.1/Modules/Remote
$ ll
total 98
-rw-r--r-- 1 Dzenan 197121  300 Jul 23 15:56 AnalyzeObjectMapIO.remote.cmake
-rw-r--r-- 1 Dzenan 197121 1198 Jul 23 15:56 AnisotropicDiffusionLBR.remote.cmake
-rw-r--r-- 1 Dzenan 197121  413 Jul 23 15:56 BioCell.remote.cmake
-rw-r--r-- 1 Dzenan 197121  578 Jul 23 15:56 BoneMorphometry.remote.cmake
-rw-r--r-- 1 Dzenan 197121  255 Jul 23 15:56 BSplineGradient.cmake
-rw-r--r-- 1 Dzenan 197121  174 Jul 23 15:56 CMakeLists.txt
-rw-r--r-- 1 Dzenan 197121  971 Jul 23 15:56 Cuberille.remote.cmake
-rw-r--r-- 1 Dzenan 197121  271 Jul 23 15:56 FixedPointInverseDisplacementField.remote.cmake
-rw-r--r-- 1 Dzenan 197121  252 Jul 23 15:56 GenericLabelInterpolator.remote.cmake
-rw-r--r-- 1 Dzenan 197121  595 Jul 23 15:56 HigherOrderAccurateGradient.remote.cmake
-rw-r--r-- 1 Dzenan 197121  243 Jul 23 15:56 IOFDF.remote.cmake
drwxr-xr-x 1 Dzenan 197121    0 Aug  9 15:26 IOMeshSTL/
-rw-r--r-- 1 Dzenan 197121  374 Jul 23 15:56 IOMeshSTL.remote.cmake
-rw-r--r-- 1 Dzenan 197121  288 Jul 23 15:56 IOOpenSlide.remote.cmake
-rw-r--r-- 1 Dzenan 197121  219 Jul 23 15:56 IOScanco.remote.cmake
drwxr-xr-x 1 Dzenan 197121    0 Aug  9 15:27 IOTransformDCMTK/
-rw-r--r-- 1 Dzenan 197121  428 Jul 23 15:56 IOTransformDCMTK.remote.cmake
-rw-r--r-- 1 Dzenan 197121  651 Jul 23 15:56 IsotropicWavelets.remote.cmake
-rw-r--r-- 1 Dzenan 197121  582 Jul 23 15:56 LabelErodeDilate.remote.cmake
-rw-r--r-- 1 Dzenan 197121  258 Jul 23 15:56 LesionSizingToolkit.remote.cmake
-rw-r--r-- 1 Dzenan 197121  506 Jul 23 15:56 MeshNoise.remote.cmake
-rw-r--r-- 1 Dzenan 197121  182 Jul 23 15:56 MGHIO.remote.cmake
-rw-r--r-- 1 Dzenan 197121  301 Jul 23 15:56 MinimalPathExtraction.remote.cmake
-rw-r--r-- 1 Dzenan 197121  293 Jul 23 15:56 Montage.remote.cmake
-rw-r--r-- 1 Dzenan 197121  860 Jul 23 15:56 MorphologicalContourInterpolation.remote.cmake
-rw-r--r-- 1 Dzenan 197121  872 Jul 23 15:56 MultipleImageIterator.remote.cmake
-rw-r--r-- 1 Dzenan 197121  348 Jul 23 15:56 NeuralNetworks.remote.cmake
-rw-r--r-- 1 Dzenan 197121  470 Jul 23 15:56 ParabolicMorphology.remote.cmake
-rw-r--r-- 1 Dzenan 197121  718 Jul 23 15:56 PerformanceBenchmarking.remote.cmake
-rw-r--r-- 1 Dzenan 197121  542 Jul 23 15:56 PhaseSymmetry.remote.cmake
-rw-r--r-- 1 Dzenan 197121  548 Jul 23 15:56 PolarTransform.remote.cmake
-rw-r--r-- 1 Dzenan 197121  614 Jul 23 15:56 PrincipalComponentsAnalysis.remote.cmake
-rw-r--r-- 1 Dzenan 197121 1381 Jul 23 15:56 README.md
drwxr-xr-x 1 Dzenan 197121    0 Aug  9 15:27 RLEImage/
-rw-r--r-- 1 Dzenan 197121  528 Jul 23 15:56 RLEImage.remote.cmake
-rw-r--r-- 1 Dzenan 197121  216 Jul 23 15:56 RTK.remote.cmake
-rw-r--r-- 1 Dzenan 197121  198 Jul 23 15:56 SCIFIO.remote.cmake
-rw-r--r-- 1 Dzenan 197121  490 Jul 23 15:56 SimpleITKFilters.remote.cmake
-rw-r--r-- 1 Dzenan 197121  247 Jul 23 15:56 SkullStrip.remote.cmake
-rw-r--r-- 1 Dzenan 197121  501 Jul 23 15:56 SmoothingRecursiveYvvGaussianFilter.remote.cmake
-rw-r--r-- 1 Dzenan 197121  304 Jul 23 15:56 SphinxExamples.remote.cmake
-rw-r--r-- 1 Dzenan 197121  488 Jul 23 15:56 SplitComponents.remote.cmake
-rw-r--r-- 1 Dzenan 197121  562 Jul 23 15:56 Strain.remote.cmake
-rw-r--r-- 1 Dzenan 197121  454 Jul 23 15:56 SubdivisionQuadEdgeMeshFilter.remote.cmake
-rw-r--r-- 1 Dzenan 197121  735 Jul 23 15:56 TextureFeatures.remote.cmake
-rw-r--r-- 1 Dzenan 197121  267 Jul 23 15:56 Thickness3D.remote.cmake
-rw-r--r-- 1 Dzenan 197121  509 Jul 23 15:56 TotalVariation.remote.cmake
-rw-r--r-- 1 Dzenan 197121 1214 Jul 23 15:56 TwoProjectionRegistration.remote.cmake
-rw-r--r-- 1 Dzenan 197121  518 Jul 23 15:56 VariationalRegistration.remote.cmake
-rw-r--r-- 1 Dzenan 197121  261 Jul 23 15:56 WikiExamples.remote.cmake

Dzenan@Ryzenator MINGW64 /c/Libs/ITK-5.0.1/Modules/Remote
@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

@dzenanz, thank you :) is there any easy way to just “provide” the ITKGenericLabelInterpolator code so that git clone is not invoked, but I otherwise use the ITK CMake system?

@dzenanz

This comment has been minimized.

Copy link
Member

commented Aug 13, 2019

The code has to be there, either "manually" git cloned earlier, or just unzipped.

Are you already passing -DITK_FORBID_DOWNLOADS:BOOL=ON? If not, try that. If that does not help, you might need to modify ITKModuleRemote.cmake so it does not attempt to download stuff.

@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

@dzenanz ok, so I turned -DITK_FORBID_DOWNLOADS:BOOL=ON, and downloaded and copied the GenericLabelInterpolator code manually (see the ls calls following the log):

-- Performing Test C_HAS_WARNING-mtune_native - Success
-- Performing Test C_HAS_WARNING-march_corei7
-- Performing Test C_HAS_WARNING-march_corei7 - Success
-- Performing Test CXX_HAS_WARNING-mtune_native
-- Performing Test CXX_HAS_WARNING-mtune_native - Success
-- Performing Test CXX_HAS_WARNING-march_corei7
-- Performing Test CXX_HAS_WARNING-march_corei7 - Success
-- Performing Test have_gold
-- Performing Test have_gold - Success
-- Performing Test CXX_HAS_DISABLE_OPTIMIZATION_FLAG
-- Performing Test CXX_HAS_DISABLE_OPTIMIZATION_FLAG - Failed
CMake Error at CMake/ITKDownloadSetup.cmake:14 (message):
  Attempted to download Module_GenericLabelInterpolator when
  ITK_FORBID_DOWNLOADS is ON
Call Stack (most recent call first):
  CMake/ITKModuleRemote.cmake:169 (itk_download_attempt_check)
  Modules/Remote/GenericLabelInterpolator.remote.cmake:1 (itk_fetch_module)
  Modules/Remote/CMakeLists.txt:6 (include)


-- Found Git: /usr/bin/git (found version "2.22.0") 
CMake Error at CMake/ITKModuleRemote.cmake:12 (message):
  Failed to clone repository:
  '://github.com/InsightSoftwareConsortium/ITKGenericLabelInterpolator.git'
Call Stack (most recent call first):
  CMake/ITKModuleRemote.cmake:134 (_git_clone)
  CMake/ITKModuleRemote.cmake:197 (_fetch_with_git)
  Modules/Remote/GenericLabelInterpolator.remote.cmake:1 (itk_fetch_module)
  Modules/Remote/CMakeLists.txt:6 (include)


-- Configuring incomplete, errors occurred!
darkhost ~ # ls /var/tmp/portage/sci-libs/itk-5.0.1/work/InsightToolkit-5.0.1/Modules/Re
Registration/ Remote/       
darkhost ~ # ls /var/tmp/portage/sci-libs/itk-5.0.1/work/InsightToolkit-5.0.1/Modules/Remote/
AnalyzeObjectMapIO.remote.cmake                  MultipleImageIterator.remote.cmake
AnisotropicDiffusionLBR.remote.cmake             NeuralNetworks.remote.cmake
BSplineGradient.cmake                            ParabolicMorphology.remote.cmake
BioCell.remote.cmake                             PerformanceBenchmarking.remote.cmake
BoneMorphometry.remote.cmake                     PhaseSymmetry.remote.cmake
CMakeLists.txt                                   PolarTransform.remote.cmake
Cuberille.remote.cmake                           PrincipalComponentsAnalysis.remote.cmake
FixedPointInverseDisplacementField.remote.cmake  README.md
GenericLabelInterpolator.remote.cmake            RLEImage.remote.cmake
HigherOrderAccurateGradient.remote.cmake         RTK.remote.cmake
IOFDF.remote.cmake                               SCIFIO.remote.cmake
IOMeshSTL.remote.cmake                           SimpleITKFilters.remote.cmake
IOOpenSlide.remote.cmake                         SkullStrip.remote.cmake
IOScanco.remote.cmake                            SmoothingRecursiveYvvGaussianFilter.remote.cmake
IOTransformDCMTK.remote.cmake                    SphinxExamples.remote.cmake
ITKGenericLabelInterpolator                      SplitComponents.remote.cmake
IsotropicWavelets.remote.cmake                   Strain.remote.cmake
LabelErodeDilate.remote.cmake                    SubdivisionQuadEdgeMeshFilter.remote.cmake
LesionSizingToolkit.remote.cmake                 TextureFeatures.remote.cmake
MGHIO.remote.cmake                               Thickness3D.remote.cmake
MeshNoise.remote.cmake                           TotalVariation.remote.cmake
MinimalPathExtraction.remote.cmake               TwoProjectionRegistration.remote.cmake
Montage.remote.cmake                             VariationalRegistration.remote.cmake
MorphologicalContourInterpolation.remote.cmake   WikiExamples.remote.cmake
darkhost ~ # ls /var/tmp/portage/sci-libs/itk-5.0.1/work/InsightToolkit-5.0.1/Modules/Remote/ITKGenericLabelInterpolator/
CMakeLists.txt     LICENSE     appveyor.yml  include           setup.py  wrapping
CTestConfig.cmake  README.rst  document      itk-module.cmake  test

anything else I could try?

@dzenanz

This comment has been minimized.

Copy link
Member

commented Aug 13, 2019

Modify ITKModuleRemote.cmake so it instead of downloading it just reports error if the code is not already there.

@thewtex do you have any other suggestion Matt?

@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

@dzenanz so I should change the function to:

# Helper to perform the initial git clone and checkout.
function(_git_clone git_executable git_repository git_tag module_dir)
  if(NOT EXISTS module_dir)
    execute_process(
      COMMAND "${git_executable}" clone "${git_repository}" "${module_dir}"
      RESULT_VARIABLE error_code
      OUTPUT_QUIET
      ERROR_QUIET
      )
    if(error_code)
      message(FATAL_ERROR "Failed to clone repository: '${git_repository}'")
    endif()

    execute_process(
      COMMAND "${git_executable}" checkout ${git_tag}
      WORKING_DIRECTORY "${module_dir}"
      RESULT_VARIABLE error_code
      OUTPUT_QUIET
      ERROR_QUIET
      )
    if(error_code)
      message(FATAL_ERROR "Failed to checkout tag: '${git_tag}'")
    endif()

    execute_process(
      COMMAND "${git_executable}" submodule init
      WORKING_DIRECTORY "${module_dir}"
      RESULT_VARIABLE error_code
      )
    if(error_code)
      message(FATAL_ERROR "Failed to init submodules in: '${module_dir}'")
    endif()

    execute_process(
      COMMAND "${git_executable}" submodule update --recursive
      WORKING_DIRECTORY "${module_dir}"
      RESULT_VARIABLE error_code
      )
    if(error_code)
      message(FATAL_ERROR "Failed to update submodules in: '${module_dir}'")
    endif()
  endif()
endfunction()
@dzenanz

This comment has been minimized.

Copy link
Member

commented Aug 13, 2019

Maybe just remove this section?

if(Module_${_name})
itk_download_attempt_check(Module_${_name})
include(CMakeParseArguments)
cmake_parse_arguments(_fetch_options "" "GIT_REPOSITORY;GIT_TAG" "" ${ARGN})
find_package(Git)
if(NOT GIT_EXECUTABLE)
message(FATAL_ERROR "error: could not find git for clone of ${_name}")
endif()
execute_process(
COMMAND "${GIT_EXECUTABLE}" --version
OUTPUT_VARIABLE ov
OUTPUT_STRIP_TRAILING_WHITESPACE
)
string(REGEX REPLACE "^git version (.+)$" "\\1" _version "${ov}")
if("${_version}" VERSION_LESS 1.6.6)
message(FATAL_ERROR "Git version 1.6.6 or later is required.")
endif()
set(REMOTE_GIT_TAG "${_fetch_options_GIT_TAG}")
if( DEFINED REMOTE_GIT_TAG_${_name} AND NOT "${REMOTE_GIT_TAG_${_name}}" STREQUAL "${_fetch_options_GIT_TAG}")
set(REMOTE_GIT_TAG "${REMOTE_GIT_TAG_${_name}}")
message(STATUS "NOTE: Using override 'REMOTE_GIT_TAG_${_name}=${REMOTE_GIT_TAG}'\n"
" instead of value 'GIT_TAG=${_fetch_options_GIT_TAG}'\n"
" specified in file ${ITK_SOURCE_DIR}/Modules/Remote/${_name}.remote.cmake'")
endif()
set(REMOTE_GIT_TAG_${_name} "${REMOTE_GIT_TAG}" CACHE STRING "Override default GIT_TAG value for remote module ${_name}")
mark_as_advanced(REMOTE_GIT_TAG_${_name})
_fetch_with_git("${GIT_EXECUTABLE}"
"${_fetch_options_GIT_REPOSITORY}"
"${REMOTE_GIT_TAG}"
"${ITK_SOURCE_DIR}/Modules/Remote/${_name}"
)
endif()

@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

@dzenanz thank you so much :) that seems to have worked (and my initial suggestion didn't).

Do you think this is perhaps something that should be addressed upstream (i.e. with you) as well? after all, what sense does the -DITK_FORBID_DOWNLOADS:BOOL=ON option make, if the build system can't just use local sources instead of automatically failing?

@thewtex

This comment has been minimized.

Copy link
Member

commented Aug 13, 2019

As noted by @gdevenyi , to build without using the Remote Module Git clone, download the ITKGenericLabelInterpolator beforehand, e.g. with this link:

https://github.com/InsightSoftwareConsortium/ITKGenericLabelInterpolator/archive/master.zip

Then, unpack, create a build directory, and build it against the ITK build tree by passing ITK_DIR as a CMake configuration variable.

@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

@dzenanz ok, maybe I celebrated to soon, the build works fine with the modification suggested by you, but as soon as I run it with testing support , it fails: build.log

@thewtex ok, that would mean treating it as a separate package, rather than just a feature of ITK:

  1. How would I be able to test it?
  2. Also that brings up the question of versioning. I could version it by downloading the commits, and tracking them (the commits themselves contain no ordinal info) by using the version of the ITK associated with them. E.g. ITKGenericLabelInterpolator-187ab99b7d42718c99e5017f0acd3900d7469bd1 would be ITKGenericLabelInterpolator-5.0.1. This will work, but could break if you plan to introduce a (different) versioning scheme for the package. Do you plan to do that?
@dzenanz

This comment has been minimized.

Copy link
Member

commented Aug 13, 2019

CMake Error at /usr/share/cmake/Modules/ExternalData.cmake:1121 (message):
  Object MD5=7ccd2dcfe9736b82a696a6c71d4dfbb8 not found at:
    (No ExternalData_URL_TEMPLATES given)

I guess you also need to put test data for all the remote modules you need into the same place as the rest of ITK test data.

@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

@dzenanz I have looked around both in the ITKGenericLabelInterpolator repo, as well as in the ExternalData.cmake file of the ITK repo, as well as under the ITK GitHub account, and could not find a link to this external data. Where is it located?

@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

Also, I am unsure if the build system actually succeeded in generating the ITKGenericLabelInterpolator library. In the ITK files list I only see one entry with this name, which is /usr/lib/cmake/ITK-5.0/Modules/GenericLabelInterpolator.cmake --- I assumed there should also be a .h file?

@dzenanz

This comment has been minimized.

Copy link
Member

commented Aug 13, 2019

Any .h files belonging to GenericLabelInterpolator should be in /usr/include/ITK-5.0/, along all the other ITK's include files.

@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

I know, but there is no such file, as seen in the full files list :( I'll have the log ready by tomorrow (testing another issue atm). But do you maybe have any idea why this file might not have been generated?

@dzenanz

This comment has been minimized.

Copy link
Member

commented Aug 13, 2019

Test data is located here: https://data.kitware.com/#collection/57b5c9e58d777f126827f5a1/folder/57b5d5df8d777f10f269445b

It only seems to have 2 external test files. PR

@dzenanz

This comment has been minimized.

Copy link
Member

commented Aug 13, 2019

And it looks like GenericLabelInterpolator needs some general updating to build with version 5.x.

@dzenanz

This comment has been minimized.

Copy link
Member

commented Aug 13, 2019

/usr/include/ITK-5.0/itkLabelImageGenericInterpolateImageFunction.h and companions are in your list of files.

@thewtex

This comment has been minimized.

Copy link
Member

commented Aug 14, 2019

2. Also that brings up the question of versioning.

The Git commit hash of the ITKGenericLabelInterpolator repository could be used as the version.

@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 14, 2019

@thewtex I know, but that provides no sequential information. If one commit of ITKGenericLabelInterpolator is tied to one version of ITK, I was thinking maybe using the ITK version would be best. This is all done on my end, you don't have to worry about it, my question to you was: do you plan on introducing some other verioning scheme for ITKGenericLabelInterpolator which might conflict with this?

@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 14, 2019

@dzenanz sorry about the confusion regarding the .h file, yes, it is there, I just searched for the name of the package and not the file :)

Can I just download this ( https://data.kitware.com/api/v1/folder/57b5d5de8d777f10f269445a/download ) for the test data (the whole package seems very large)? At what place in the testing data hierarchy should I put this?

Using this CMake command:

cmake -C /var/tmp/portage/sci-libs/itk-5.0.1/work/itk-5.0.1_build/gentoo_common_config.cmake
    -G Unix Makefiles -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=ON 
    -DGDCM_USE_SYSTEM_OPENJPEG=ON -DITK_FORBID_DOWNLOADS:BOOL=ON 
    -DITK_USE_SYSTEM_DCMTK=ON -DITK_USE_SYSTEM_DOUBLECONVERSION=ON 
    -DITK_USE_SYSTEM_CASTXML=ON -DITK_USE_SYSTEM_HDF5=ON 
    -DITK_USE_SYSTEM_JPEG=ON -DITK_USE_SYSTEM_PNG=ON 
    -DITK_USE_SYSTEM_SWIG=ON -DITK_USE_SYSTEM_TIFF=ON 
    -DITK_USE_SYSTEM_ZLIB=ON -DITK_USE_KWSTYLE=OFF 
    -DITK_BUILD_DEFAULT_MODULES=ON -DITK_COMPUTER_MEMORY_SIZE=1 
    -DWRAP_ITK_JAVA=OFF -DWRAP_ITK_TCL=OFF -Ddouble-conversion_INCLUDE_DIRS=/usr/include/double-conversion
    -DExternalData_OBJECT_STORES=/var/tmp/portage/sci-libs/itk-5.0.1/work/InsightToolkit-5.0.1/.ExternalData
    -DModule_GenericLabelInterpolator:BOOL=ON -DBUILD_EXAMPLES=OFF 
    -DITK_USE_REVIEW=OFF -DITK_INSTALL_LIBRARY_DIR=lib64 -DITK_WRAP_PYTHON=OFF 
    -DCMAKE_BUILD_TYPE=Gentoo -DCMAKE_TOOLCHAIN_FILE=/var/tmp/portage/sci-libs/itk-5.0.1/work/itk-5.0.1_build/gentoo_toolchain.cmake
    /var/tmp/portage/sci-libs/itk-5.0.1/work/InsightToolkit-5.0.1

and having this work directory hierarchy:

darkhost /var/tmp/portage/sci-libs/itk-5.0.1/work # tree -a -L 3   
.
|-- GenericLabelInterpolator
|   `-- test
|       |-- Baseline
|       `-- Input
|-- ITKGenericLabelInterpolator-187ab99b7d42718c99e5017f0acd3900d7469bd1
|   |-- .circleci
|   |   `-- config.yml
|   |-- .gitignore
|   |-- .travis.yml
|   |-- CMakeLists.txt
|   |-- CTestConfig.cmake
|   |-- LICENSE
|   |-- README.rst
|   |-- appveyor.yml
|   |-- document
|   |   |-- Generic\ label\ interpolation.odt
|   |   `-- results
|   |-- include
|   |   |-- itkLabelImageGenericInterpolateImageFunction.h
|   |   |-- itkLabelImageGenericInterpolateImageFunction.hxx
|   |   |-- itkLabelSelectionImageAdaptor.h
|   |   `-- itkLabelSelectionPixelAccessor.h
|   |-- itk-module.cmake
|   |-- setup.py
|   |-- test
|   |   |-- Baseline
|   |   |-- CMakeLists.txt
|   |   |-- Docker
|   |   |-- Input
|   |   `-- RotateLabels.cxx
|   `-- wrapping
|       |-- CMakeLists.txt
|       `-- itkLabelImageGenericInterpolateImageFunction.wrap
`-- InsightToolkit-5.0.1
    |-- .ExternalData
    |   |-- MD5
    |   `-- README.rst
    |-- .circleci
    |   `-- config.yml
    |-- .clang-format
    |-- .editorconfig
    |-- CMake
    |   |-- CTestCustom.cmake.in
    |   |-- CppcheckTargets.cmake
    |   |-- FindFFTW.cmake
    |   |-- FindKWStyle.cmake
    |   |-- FindOpenCL.cmake
    |   |-- Findcppcheck.cmake
    |   |-- Findcppcheck.cpp
    |   |-- GetGitRevisionDescription.cmake
    |   |-- ITKConfig.cmake.in
    |   |-- ITKConfigVersion.cmake.in
    |   |-- ITKDownloadSetup.cmake
    |   |-- ITKExternalData.cmake
    |   |-- ITKGroups.cmake
    |   |-- ITKInitializeBuildType.cmake
    |   |-- ITKInternalConfig.cmake
    |   |-- ITKKWStyleConfig.cmake.in
    |   |-- ITKModuleAPI.cmake
    |   |-- ITKModuleCPPCheckTest.cmake
    |   |-- ITKModuleDoxygen.cmake
    |   |-- ITKModuleEnablement.cmake
    |   |-- ITKModuleExternal.cmake
    |   |-- ITKModuleHeaderTest.cmake
    |   |-- ITKModuleInfo.cmake.in
    |   |-- ITKModuleKWStyleTest.cmake
    |   |-- ITKModuleMacros.cmake
    |   |-- ITKModuleRemote.cmake
    |   |-- ITKModuleTest.cmake
    |   |-- ITKSetStandardCompilerFlags.cmake
    |   |-- ITK_CheckCCompilerFlag.cmake
    |   |-- ITK_CheckCXXCompilerFlag.cmake
    |   |-- InsightValgrind-RHEL6.supp
    |   |-- InsightValgrind-Ubuntu1804.supp
    |   |-- InsightValgrind.supp
    |   |-- PreventInBuildInstalls.cmake
    |   |-- PreventInSourceBuilds.cmake
    |   |-- RemoveTemporaryFiles.cmake.in
    |   |-- TopologicalSort.cmake
    |   |-- UseITK.cmake
    |   |-- itkApple.cmake
    |   |-- itkCheckSourceTree.cmake
    |   |-- itkCompilerChecks.cmake
    |   |-- itkExternal_FFTW.cmake
    |   |-- itkImageIOFactoryRegisterManager.h.in
    |   |-- itkMeshIOFactoryRegisterManager.h.in
    |   |-- itkOpenCL.cmake
    |   |-- itkSupportMacros.cmake
    |   |-- itkTargetLinkLibrariesWithDynamicLookup.cmake
    |   |-- itkTestObjCxxCompiler.mm
    |   |-- itkTransformIOFactoryRegisterManager.h.in
    |   |-- itkVersion.cmake
    |   `-- pre-commit
    |-- CMakeLists.txt
    |-- CODE_OF_CONDUCT.md
    |-- CONTRIBUTING.md
    |-- CTestConfig.cmake
    |-- Documentation
    |   |-- Art
    |   |-- CodeOfConduct
    |   |-- Data.md
    |   |-- Download.md
    |   |-- Doxygen
    |   |-- GitCheatSheet.pdf
    |   |-- GitCheatSheet.tex
    |   |-- GitHelp.md
    |   |-- ITK5MigrationGuide.md
    |   |-- Maintenance
    |   |-- ReleaseNotes
    |   |-- SupportedCompilers.md
    |   `-- UploadBinaryData.md
    |-- Examples
    |   |-- CMakeLists.txt
    |   |-- Data
    |   |-- DataRepresentation
    |   |-- Filtering
    |   |-- GUI
    |   |-- IO
    |   |-- Infrastructure
    |   |-- Installation
    |   |-- Iterators
    |   |-- Numerics
    |   |-- README.md
    |   |-- RegistrationITKv4
    |   |-- SearchScript.sh
    |   |-- Segmentation
    |   |-- SpatialObjects
    |   |-- Statistics
    |   |-- Visualization
    |   `-- Wrapping
    |-- GettingStarted.md
    |-- LICENSE
    |-- Modules
    |   |-- Bridge
    |   |-- Compatibility
    |   |-- Core
    |   |-- External
    |   |-- Filtering
    |   |-- IO
    |   |-- Nonunit
    |   |-- Numerics
    |   |-- Registration
    |   |-- Remote
    |   |-- Segmentation
    |   |-- ThirdParty
    |   `-- Video
    |-- NOTICE
    |-- README.md
    |-- Testing
    |   |-- ContinuousIntegration
    |   |-- Data
    |   `-- TestExternal
    |-- Utilities
    |   |-- Debugger
    |   |-- Doxygen
    |   |-- GitSetup
    |   |-- Hooks
    |   |-- ITKv5Preparation
    |   |-- InstallTest
    |   |-- KWStyle
    |   |-- Maintenance
    |   |-- SetupForDevelopment.sh
    |   `-- UploadBinaryData.sh
    `-- Wrapping
        |-- CMakeLists.txt
        |-- CMakeUtilityFunctions.cmake
        |-- ConfigureWrapping.cmake
        |-- Generators
        |-- Modules
        |-- TypedefMacros.cmake
        |-- WrapBasicTypes.cmake
        |-- WrapITKTypes.cmake
        |-- WrappingOptions.cmake
        `-- images

I assume I should just mv GenericLabelInterpolator InsightToolkit-5.0.1/.ExternalData, but that still fails with build.log.

@dzenanz

This comment has been minimized.

Copy link
Member

commented Aug 15, 2019

Yes, you could/should just download the files in this folder tree (a total of 2 files). The files should be named 861752fa4c3c619e09bba6daf17643e2 and 7ccd2dcfe9736b82a696a6c71d4dfbb8 (per their MD5 hashes) and reside in /var/tmp/portage/sci-libs/itk-5.0.1/work/InsightToolkit-5.0.1/.ExternalData/MD5/ dir.

@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 19, 2019

@dzenanz should I re-name the files? because when I download them they have names like gl_gaussian_3.mha.

@dzenanz

This comment has been minimized.

Copy link
Member

commented Aug 19, 2019

Yes, the files should be renamed from human-friendly into the MD5 hashes.

@TheChymera

This comment has been minimized.

Copy link
Author

commented Aug 21, 2019

@dzenanz thanks. Sadly, the build still fails with build.log and the following tree, which includes the files as named via checksums:

    for filename in ../GenericLabelInterpolator/test/*/*mha; do
        MD5=$(md5sum $filename) || die
        MD5=${MD5% *}
        cp "$filename" ".ExternalData/MD5/${MD5}" || die
    done
@dzenanz

This comment has been minimized.

Copy link
Member

commented Aug 21, 2019

Since the error is (No ExternalData_URL_TEMPLATES given), maybe try disabling ITK_FORBID_DOWNLOADS, or removing the specification? And I know this is the opposite of what I recommended earlier 😕

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.