ENH: Add TractographyTRX module

[mattcieslak]

This PR adds a TractographyTRX remote module for reading/writing/interacting with streamline data in the TRX format. It follows up on this discussion.

We'd be happy for feedback/suggestions on the module, or help getting it merge ready. Thanks in advance!

PR Checklist

• No API changes were made (or the changes have been approved)
• No major design changes were made (or the changes have been approved)
• Added test (or behavior not changed)
• Updated API documentation (or API not changed)
• Added license to new files (if any)
• Added Python wrapping to new files (if any) as described in ITK Software Guide Section 9.5
• Added ITK examples for all new major features (if any)

[dzenanz]

Please squash commits and force-push.

[thewtex]

@mattcieslak thanks!

One request inline.

[dzenanz]

As you are using FetchContent, I think that counts as no external dependencies.

As zlib is bundled with ITK (and cannot be disabled), why do you have this section?

[mattcieslak]

I had a hard time getting libzip (a trx-cpp dependency) compiled with the built in zlib. This is clunky but eventually got the CI tests passing on all the platforms.

IIRC there were was an issue where tthe header path changed between ITK versions (src/zlib.h to src/itkzlib-ng/zlib.h) and an issue with where windows was putting the library

[dzenanz]

@thewtex You did not post the in-line request 😄

[thewtex]

@thewtex You did not post the in-line request 😄

Added :-)

[dzenanz]

I tried this locally, and ran into:

Configuring done (5.6s)
CMake Error at C:/Misc/ITKTractographyTRX-vs22/_deps/trx_cpp-src/CMakeLists.txt:121 (TARGET_LINK_LIBRARIES):
  Target "trx" links to:

    Eigen3::Eigen

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

Generating done (0.5s)

VS projects were still generated. Trying to build resulted in:

Build started at 16:37...
1>------ Build started: Project: zip, Configuration: Release x64 ------
2>------ Build started: Project: castxml, Configuration: Release x64 ------
3>------ Build started: Project: itk-stub-files, Configuration: Release x64 ------
4>------ Build started: Project: swig, Configuration: Release x64 ------
5>------ Build started: Project: dist, Configuration: Release x64 ------
6>------ Build started: Project: TractographyTRXHeaderTestClean, Configuration: Release x64 ------
7>------ Skipped Build: Project: Experimental, Configuration: Release x64 ------
7>Project not selected to build for this solution configuration 
8>------ Skipped Build: Project: RUN_TESTS, Configuration: Release x64 ------
8>Project not selected to build for this solution configuration 
9>------ Skipped Build: Project: NightlyMemoryCheck, Configuration: Release x64 ------
9>Project not selected to build for this solution configuration 
10>------ Skipped Build: Project: Nightly, Configuration: Release x64 ------
10>Project not selected to build for this solution configuration 
11>------ Skipped Build: Project: Continuous, Configuration: Release x64 ------
11>Project not selected to build for this solution configuration 
12>------ Skipped Build: Project: list-fuzzers, Configuration: Release x64 ------
12>Project not selected to build for this solution configuration 
1>  zip_algorithm_deflate.c
1>  zip_dirent.c
3>  Generating .pyi files for Python wrapper interface
1>  zip_file_set_mtime.c
1>  zip_file_set_encryption.c
1>  zip_file_set_external_attributes.c
1>  zip_fopen_encrypted.c
1>  zip_fseek.c
1>  zip_hash.c
13>------ Skipped Build: Project: ITKHeaderTests, Configuration: Release x64 ------
13>Project not selected to build for this solution configuration 
1>  zip_fopen.c
1>  zip_get_name.c
1>  zip_fopen_index.c
1>C:\Misc\ITKTractographyTRX-vs22\_deps\zlib-src\zlib.h(34,10): error C1083: Cannot open include file: 'zconf.h': No such file or directory
1>C:\Misc\ITKTractographyTRX-vs22\_deps\zlib-src\zlib.h(34,10): error C1083: Cannot open include file: 'zconf.h': No such file or directory
1>  (compiling source file '../../libzip-src/lib/zip_dirent.c')
1>  (compiling source file '../../libzip-src/lib/zip_algorithm_deflate.c')
1>  zip_error_strerror.c
1>  zip_get_encryption_implementation.c
1>  zip_get_num_entries.c
1>  zip_get_file_comment.c
1>  zip_file_strerror.c
1>  zip_io_util.c
1>  zip_libzip_version.c
1>  zip_get_archive_flag.c
1>  zip_fread.c
1>  zip_ftell.c
1>  zip_get_archive_comment.c
1>  zip_fopen_index_encrypted.c
1>C:\Misc\ITKTractographyTRX-vs22\_deps\zlib-src\zlib.h(34,10): error C1083: Cannot open include file: 'zconf.h': No such file or directory
1>  (compiling source file '../../libzip-src/lib/zip_error_strerror.c')
1>C:\Misc\ITKTractographyTRX-vs22\_deps\zlib-src\zlib.h(34,10): error C1083: Cannot open include file: 'zconf.h': No such file or directory
1>  (compiling source file '../../libzip-src/lib/zip_io_util.c')
1>  zip_get_num_files.c
3>CUSTOMBUILD : warning : index file  is missing,  Python stub hints will not be generated for this file.
3>  Exception: No index files were found in directory 'C:/Dev/ITK-vs22/Wrapping/Generators/Python/itk-pkl'
3>  Traceback (most recent call last):
3>    File "C:\Dev\ITK-git\Wrapping\Generators\Python\itk\pyi_generator.py", line 570, in <module>
3>      raise Exception(except_comment)
3>  Exception: No index files were found in directory 'C:/Dev/ITK-vs22/Wrapping/Generators/Python/itk-pkl'
3>C:\Program Files\Microsoft Visual Studio\18\Community\MSBuild\Microsoft\VC\v180\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\Misc\ITKTractographyTRX-vs22\CMakeFiles\bac0ec94f44e9e0378c091e26551b5cb\itk-stub-files.rule' exited with code 1.
14>------ Build started: Project: trx, Configuration: Release x64 ------
15>------ Build started: Project: zip_read_file_fuzzer, Configuration: Release x64 ------
16>------ Build started: Project: zip_write_encrypt_pkware_file_fuzzer, Configuration: Release x64 ------
17>------ Build started: Project: zip_write_encrypt_aes256_file_fuzzer, Configuration: Release x64 ------
18>------ Build started: Project: zip_read_fuzzer, Configuration: Release x64 ------
18>LINK : fatal error LNK1181: cannot open input file '..\lib\Release\zip.lib'
17>LINK : fatal error LNK1181: cannot open input file '..\lib\Release\zip.lib'
14>  dtype_helpers.cpp
16>LINK : fatal error LNK1181: cannot open input file '..\lib\Release\zip.lib'
15>LINK : fatal error LNK1181: cannot open input file '..\lib\Release\zip.lib'
14>  trx.cpp
14>C:\Misc\ITKTractographyTRX-vs22\_deps\trx_cpp-src\include\trx\trx.h(4,10): error C1083: Cannot open include file: 'Eigen/Core': No such file or directory
14>C:\Misc\ITKTractographyTRX-vs22\_deps\trx_cpp-src\src\trx.cpp(1,10): error C1083: Cannot open include file: 'Eigen/Core': No such file or directory
14>  (compiling source file '../trx_cpp-src/src/detail/dtype_helpers.cpp')
19>------ Build started: Project: TractographyTRX, Configuration: Release x64 ------
20>------ Skipped Build: Project: distcheck, Configuration: Release x64 ------
20>Project not selected to build for this solution configuration 
19>  itkTrxParcellationLabeler.cxx
19>  itkTrxStreamWriter.cxx
19>  itkTrxStreamlineData.cxx
19>C:\Misc\ITKTractographyTRX-vs22\_deps\trx_cpp-src\include\trx\trx.h(4,10): error C1083: Cannot open include file: 'Eigen/Core': No such file or directory
19>  (compiling source file '../../ITKTractographyTRX/src/itkTrxStreamWriter.cxx')
19>C:\Misc\ITKTractographyTRX-vs22\_deps\trx_cpp-src\include\trx\trx.h(4,10): error C1083: Cannot open include file: 'Eigen/Core': No such file or directory
19>  (compiling source file '../../ITKTractographyTRX/src/itkTrxStreamlineData.cxx')
19>C:\Misc\ITKTractographyTRX-vs22\_deps\trx_cpp-src\include\trx\trx.h(4,10): error C1083: Cannot open include file: 'Eigen/Core': No such file or directory
19>  (compiling source file '../../ITKTractographyTRX/src/itkTrxParcellationLabeler.cxx')
21>------ Build started: Project: TractographyTRXTestDriver, Configuration: Release x64 ------
22>------ Build started: Project: TractographyTRXHeaderTest1, Configuration: Release x64 ------
22>  Generating test/TractographyTRXHeaderTest1.cxx
21>  itkTrxGroupTest.cxx
21>  itkTrxGroupTdiMapperTest.cxx
21>  itkTrxCoverageTest.cxx
22>  TractographyTRXHeaderTest1.cxx
21>C:\Misc\ITKTractographyTRX-vs22\_deps\trx_cpp-src\include\trx\trx.h(4,10): error C1083: Cannot open include file: 'Eigen/Core': No such file or directory
21>  (compiling source file '../../ITKTractographyTRX/test/itkTrxGroupTest.cxx')
21>C:\Misc\ITKTractographyTRX-vs22\_deps\trx_cpp-src\include\trx\trx.h(4,10): error C1083: Cannot open include file: 'Eigen/Core': No such file or directory
21>  (compiling source file '../../ITKTractographyTRX/test/itkTrxCoverageTest.cxx')
21>C:\Misc\ITKTractographyTRX-vs22\_deps\trx_cpp-src\include\trx\trx.h(4,10): error C1083: Cannot open include file: 'Eigen/Core': No such file or directory
21>  (compiling source file '../../ITKTractographyTRX/test/itkTrxGroupTdiMapperTest.cxx')
22>LINK : fatal error LNK1181: cannot open input file 'C:\Dev\ITK-vs22\lib\Release\itkTractographyTRX-6.0.lib'
23>------ Skipped Build: Project: INSTALL, Configuration: Release x64 ------
23>Project not selected to build for this solution configuration 
========== Build: 4 succeeded, 10 failed, 4 up-to-date, 9 skipped ==========
========== Build completed at 16:37 and took 04.022 seconds ==========

[blowekamp]

Eigen3::Eigen

There is an open PR to refactor ITK's embedded Eigen to conform with existing pattern in ITK. Usage such as the include path may change soon.

This looks like similar usage to the TotalVariations remove module where there is a FetchConent project that also needs to use Eigen.

[mattcieslak]

@dzenanz I updated the build based on TotalVariation's, could you please try again? I also set the shasum to the updated commit.

[blowekamp]

Hello,

I'm not the sure of the full implication for this module but there are a couple PR related. An update to they Eigen3 library handled in ITK is here: #5831 I'm not sure how much of a difference it will make with your remote, but it removes the need for a lot of special handling of the Eigen library in the CMake code.

And an update an clean update of the CMake code for TotalVariations: InsightSoftwareConsortium/ITKTotalVariation#57 But this does other simiplifications and updates to get the module working smoothly with ITKv6.

I'm going to work on the Eigen library update today hopefully the results will allow some simplification with your usage,

[dzenanz]

It now builds and tests pass. Wrapping generates an error, but that is expected given its complete absence:

Build started at 09:14...
1>------ Build started: Project: castxml, Configuration: Release x64 ------
2>------ Build started: Project: itk-stub-files, Configuration: Release x64 ------
3>------ Build started: Project: swig, Configuration: Release x64 ------
4>------ Build started: Project: dist, Configuration: Release x64 ------
5>------ Build started: Project: TractographyTRXHeaderTestClean, Configuration: Release x64 ------
6>------ Skipped Build: Project: Experimental, Configuration: Release x64 ------
6>Project not selected to build for this solution configuration 
7>------ Skipped Build: Project: RUN_TESTS, Configuration: Release x64 ------
7>Project not selected to build for this solution configuration 
8>------ Skipped Build: Project: NightlyMemoryCheck, Configuration: Release x64 ------
8>Project not selected to build for this solution configuration 
9>------ Skipped Build: Project: Nightly, Configuration: Release x64 ------
9>Project not selected to build for this solution configuration 
10>------ Skipped Build: Project: Continuous, Configuration: Release x64 ------
10>Project not selected to build for this solution configuration 
11>------ Skipped Build: Project: list-fuzzers, Configuration: Release x64 ------
11>Project not selected to build for this solution configuration 
2>  Generating .pyi files for Python wrapper interface
2>CUSTOMBUILD : warning : index file  is missing,  Python stub hints will not be generated for this file.
2>  Exception: No index files were found in directory 'C:/Dev/ITK-vs22/Wrapping/Generators/Python/itk-pkl'
2>  Traceback (most recent call last):
2>    File "C:\Dev\ITK-git\Wrapping\Generators\Python\itk\pyi_generator.py", line 570, in <module>
2>      raise Exception(except_comment)
2>  Exception: No index files were found in directory 'C:/Dev/ITK-vs22/Wrapping/Generators/Python/itk-pkl'
2>C:\Program Files\Microsoft Visual Studio\18\Community\MSBuild\Microsoft\VC\v180\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\Misc\ITKTractographyTRX-vs22\CMakeFiles\bac0ec94f44e9e0378c091e26551b5cb\itk-stub-files.rule' exited with code 1.
12>------ Skipped Build: Project: INSTALL, Configuration: Release x64 ------
12>Project not selected to build for this solution configuration 
13>------ Skipped Build: Project: ITKHeaderTests, Configuration: Release x64 ------
13>Project not selected to build for this solution configuration 
14>------ Skipped Build: Project: distcheck, Configuration: Release x64 ------
14>Project not selected to build for this solution configuration 
========== Build: 4 succeeded, 1 failed, 17 up-to-date, 9 skipped ==========
========== Build completed at 09:14 and took 01.110 seconds ==========

I would imagine it is not hard to write those wrapping specifications, as there are not a lot of classes, and they don't have a lot of template parameters either. But I assume you already have it on your TODO list?

[thewtex]

Thanks!

Please create new PRs to update the GIT_TAG as improvements land.