Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Update external Eigen from release 3.4 to master #4265

Merged

Conversation

phcerdan
Copy link
Contributor

@phcerdan phcerdan commented Oct 22, 2023

ENH: Update Eigen from 3.4 to master

@github-actions github-actions bot added type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots type:Enhancement Improvement of existing methods or implementation area:Python wrapping Python bindings for a class area:ThirdParty Issues affecting the ThirdParty module labels Oct 22, 2023
@phcerdan phcerdan changed the title ENH: Update external Eigen from release 3.4 to master WIP: ENH: Update external Eigen from release 3.4 to master Oct 22, 2023
@phcerdan
Copy link
Contributor Author

Test is passing. The failure in Windows is unrelated, it was failing for other PR's as well.

@N-Dekker
Copy link
Contributor

N-Dekker commented Oct 23, 2023

Thanks @phcerdan but would it still be an option to use a tagged revision of Eigen? Looks like 3.4.1 is quite a recent version: https://gitlab.com/libeigen/eigen/-/commits/3.4.1


Oh, now I see, 3.4.1 is just a branch, not (yet?) a tag.

@phcerdan
Copy link
Contributor Author

phcerdan commented Oct 23, 2023

Good spotting @N-Dekker, it would be good to discuss what we prefer, this 3.4.1 branch or a commit from master.

We could discuss this in the open issue: #3546

@github-actions github-actions bot removed the type:Enhancement Improvement of existing methods or implementation label Oct 24, 2023
@phcerdan phcerdan changed the title WIP: ENH: Update external Eigen from release 3.4 to master ENH: Update external Eigen from release 3.4 to master Oct 24, 2023
@github-actions github-actions bot added the type:Enhancement Improvement of existing methods or implementation label Oct 24, 2023
Copy link
Member

@thewtex thewtex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

@thewtex thewtex requested a review from jcfr October 24, 2023 16:50
@thewtex
Copy link
Member

thewtex commented Oct 24, 2023

@jcfr may have suggestions on how to make the mangling name configurable

@phcerdan
Copy link
Contributor Author

@jcfr happy to hear suggestions for making the mangling name configurable. If you can provide a link to some existing CMake project with the options, I can copy it here.

This PR introduces a BOOL variable ITK_EIGEN_MANGLE, and if that's defined, we mangle with #define Eigen itkeigen (this is a header-only library). I kept the same approach than VTK (they mangle with vtkeigen).

@jcfr
Copy link
Contributor

jcfr commented Oct 30, 2023

re: mangling

See this relevant comment:

Associated discussion for generalizing the support currently specific to Slicer12 is discussed in https://discourse.itk.org/t/adding-support-for-customizing-itk-namespace/5170

Footnotes

  1. https://github.com/Slicer/ITK/commits/slicer-v5.3rc04-2022-09-19-62eb5ca

  2. https://github.com/Slicer/ITK/commit/69e52f05c482012b95228c3fe46f299b1735ca99 (COMP: Add support for customizing ITK namespace (draft))

@jcfr
Copy link
Contributor

jcfr commented Oct 30, 2023

This PR introduces a BOOL variable ITK_EIGEN_MANGLE

I am not sure we want a CMake option to control this as we will always want the function name to be mangled if building against the version bundled in ITK.

@phcerdan
Copy link
Contributor Author

phcerdan commented Oct 30, 2023

I am not sure we want a CMake option to control this as we will always want the function name to be mangled if building against the version bundled in ITK.

Not mangling Eigen was proposed to facilitate external ITK modules (see ITKTotalVariation for example) using third-party libraries (libproxTV) that use Eigen to not having to fork the proxTV library and replace all Eigen references with itkeigen, or having to setup a new Eigen for that external module.
Eigen is so common in C++ ecosystem that it seemed a good helper for external modules.

However, I understand mangling is better for the general case, and we should turn it ON by default. But I would appreciate if we keep the option to turn it off, to reuse the internal Eigen on those external modules that use it.

Slicer@69e52f0 (COMP: Add support for customizing ITK namespace (draft))

Thanks for the links, amazing work!
As a side note and different topic, I think we have to keep all those changes of mangling to third party code in our own fork of the third-party, instead of handling them in ITK history. We should have an ITK/HDF5 fork and a branch with those changes on top.
We share the same tech with VTK to update-from-stream, but we don't seem to comply with their preconditions. Changes to third-party code should be handled in a third-party-fork, not in ITK. We face git history pollution, and merge conflicts because of it (see #2314).

CMake/itkExternal_Eigen3.cmake Outdated Show resolved Hide resolved
CMake/itkExternal_Eigen3.cmake Outdated Show resolved Hide resolved
CMake/itkExternal_Eigen3.cmake Outdated Show resolved Hide resolved
CMake/itkExternal_Eigen3.cmake Outdated Show resolved Hide resolved
CMake/itkExternal_Eigen3.cmake Show resolved Hide resolved
CMake/itkExternal_Eigen3.cmake Outdated Show resolved Hide resolved
CMake/itkExternal_Eigen3.cmake Outdated Show resolved Hide resolved
Modules/ThirdParty/Eigen3/src/itk_eigen.h.in Outdated Show resolved Hide resolved
@jcfr
Copy link
Contributor

jcfr commented Oct 30, 2023

I also suggest to split this pull request and contribute the support for disabling the mangling of the Eigen include directory into its own pull request.

@jcfr
Copy link
Contributor

jcfr commented Oct 30, 2023

Not mangling Eigen was proposed to facilitate external ITK modules (see ITKTotalVariation for example) using third-party libraries (libproxTV) that use Eigen to not having to fork the proxTV library and replace all Eigen references with itkeigen, or having to setup a new Eigen for that external module.

I indeed remember the time we worked on this together. If I recall it is available at https://github.com/phcerdan/proxTV/commits/use_eigen

For reference, see also https://github.com/InsightSoftwareConsortium/ITKTotalVariation/blob/master/CMakeLists.txt

@jcfr
Copy link
Contributor

jcfr commented Nov 7, 2023

@phcerdan and I reviewed this pull request and concluded that updating Eigen without adding extra logic for namespacing was a sensible first step.

Approach for supporting including multiple Eigen "header only" library in one project will be evaluated and discussed later.

kwrobot and others added 2 commits November 8, 2023 16:42
Code extracted from:

    https://github.com/InsightSoftwareConsortium/eigen

at commit 2a86e97ca9f0213df35bcaa7d7c3315ae5aa9ab9 (for/itk-20231108-master-4d54c43d).
# By Eigen Upstream
* upstream-Eigen3:
  Eigen3 2023-11-08 (2a86e97c)
@phcerdan phcerdan requested a review from jcfr November 8, 2023 15:45
@phcerdan phcerdan removed the area:Python wrapping Python bindings for a class label Nov 8, 2023
@phcerdan
Copy link
Contributor Author

phcerdan commented Nov 8, 2023

Tested against current ITKTotalVariation external module, which uses the internal ITK eigen, and all green.

@phcerdan
Copy link
Contributor Author

Not in a hurry, but could we move forward with this? I integrated all the suggestions from @jcfr.
Removed any reference to mangling.

Wrapping the "mangling" conversation. (Note, mangling refers to symbols, there are no symbols in a header-only library).
The simple #define Eigen itkeigen (borrowed from VTK), is not enough to mangle make this header not colliding with third-party installations of Eigen. That define will also modify third-party Eigen.

Our protection is changing the name (the path) of the eigen files.
This protects ITK from picking an external Eigen (for example one installed in the system).

If a third party linking to ITK uses Eigen, the standard procedure, as with many other dependencies is to use their own Eigen and share it with ITK (ITK_USE_SYSTEM_EIGEN), and with their software. Or, which I think is great, reuse the internal Eigen from ITK and use that for their own library without the need of using any ITK macro. See ITKTotalVariation for that: https://github.com/InsightSoftwareConsortium/ITKTotalVariation/blob/b615ce13394fb5fb16bf8abc2390f5dab73ac27e/CMakeLists.txt

  if(NOT ITK_USE_SYSTEM_EIGEN)
    # Set Eigen3_DIR to the internal Eigen3 used in ITK
    set(Eigen3_DIR "${ITKInternalEigen3_DIR}")
    message(STATUS "ITKTotalVariation: Using internal ITK Eigen Config found in: ${Eigen3_DIR}")
  endif()
  find_package(Eigen3 REQUIRED)
  set(${PROJECT_NAME}_EXPORT_CODE_INSTALL
"${${PROJECT_NAME}_EXPORT_CODE_INSTALL}
set(Eigen3_DIR \"${Eigen3_DIR}\")
find_package(Eigen3 REQUIRED CONFIG)
")
  set(${PROJECT_NAME}_EXPORT_CODE_BUILD
"${${PROJECT_NAME}_EXPORT_CODE_BUILD}
if(NOT ITK_BINARY_DIR)
  set(Eigen3_DIR \"${Eigen3_DIR}\")
  find_package(Eigen3 REQUIRED CONFIG)
endif()
")

@dzenanz
Copy link
Member

dzenanz commented Nov 13, 2023

@jcfr Review/merge?

CMake/itkExternal_Eigen3.cmake Outdated Show resolved Hide resolved
CMake/itkExternal_Eigen3.cmake Outdated Show resolved Hide resolved
@github-actions github-actions bot added the area:Python wrapping Python bindings for a class label Nov 13, 2023
Conditionally pass CMAKE_SH if CMAKE_VERSION is less than 3.17.0

Co-authored-by: Jean-Christophe Fillion-Robin <jchris.fillionr@kitware.com>
@phcerdan
Copy link
Contributor Author

This is ready, all @jcfr suggestions have been integrated (thanks!)

Copy link
Member

@thewtex thewtex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💖

@phcerdan phcerdan merged commit 5d494b0 into InsightSoftwareConsortium:master Nov 19, 2023
11 of 12 checks passed
@phcerdan phcerdan deleted the update_eigen_to_master branch November 19, 2023 08:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:Python wrapping Python bindings for a class area:ThirdParty Issues affecting the ThirdParty module type:Enhancement Improvement of existing methods or implementation type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants