Skip to content

ENH: Add Modules/Beta/ container (infra-only for stacked beta-module ingests)#6085

Closed
hjmjohnson wants to merge 1 commit intoInsightSoftwareConsortium:mainfrom
hjmjohnson:wip-inline-beta-modules
Closed

ENH: Add Modules/Beta/ container (infra-only for stacked beta-module ingests)#6085
hjmjohnson wants to merge 1 commit intoInsightSoftwareConsortium:mainfrom
hjmjohnson:wip-inline-beta-modules

Conversation

@hjmjohnson
Copy link
Copy Markdown
Member

@hjmjohnson hjmjohnson commented Apr 19, 2026

Infrastructure-only scaffold: adds Modules/Beta/ as the in-tree home
for modules formerly fetched via Modules/Remote/*.remote.cmake. No
modules are ingested by this PR — see the follow-on PR for the actual
ingests.

Stack: this is the base of a two-PR stack.

  • PR 6085 (this PR)Modules/Beta/ container only.
  • PR 6086 — grafts the 10 beta modules on top, with STYLE: cleanup
    and per-module COMP: Fix pre-commit hook failures for <Module>
    commits. Full upstream git history and blame preserved via
    git merge --allow-unrelated-histories --no-ff of filter-repo'd
    histories.

Closes prior rejected approach in #6061; cross-linked to issue #6060.

What this PR adds
  • Modules/Beta/CMakeLists.txt — declares itk_beta_module_manifest()
    (no-op CMake function) and globs Modules/Beta/*.beta.cmake
    provenance manifests so they are valid CMake and can be include()-d
    with no build-time side effects. Modules grafted under
    Modules/Beta/<Name>/ are auto-discovered by the existing
    itk-module.cmake DAG scan in CMake/ITKModuleEnablement.cmake.
  • Wires add_subdirectory(Modules/Beta) into the root CMakeLists.txt
    directly after Modules/Remote.

35 insertions across 2 files. No modules activated.

Why this shape — reviewer-facing strategy

Reviewers on #6061 rejected the "consolidate remote modules into a
category repo" approach. This restart inverts the strategy:

  1. Graft remote modules in-tree at Modules/Beta/<Name>/, bringing
    their full upstream git history with them (so git blame still walks
    back to the original authors).
  2. Use a provenance bridge manifest at
    Modules/Beta/<Name>.beta.cmake that records UPSTREAM_URL,
    UPSTREAM_SHA, and INGEST_DATE — the bridge lets tooling audit what
    was grafted without parsing README prose.
  3. Separate the infra (this PR) from the ingests (follow-on PR) so the
    container mechanism can land and be reviewed independently of any
    per-module content.
Remote-module categorization (response to @blowekamp)

The original domain-based grouping lives in
#6060
(IO / Filtering / Mesh / Analysis / Registration). That axis sorts
modules by what they compute. @blowekamp asked for an orthogonal
integration-difficulty axis — what makes a module easy or hard to
ingest into ITK proper. The two axes are complementary; ingest
ordering should follow the difficulty axis (easiest first).

Tier Definition Examples (from current ~60 remote modules) Ingest priority
A. Header-only / pure-ITK New filters or iterators built only on itk:: types; no third-party libraries; no executables. Cleanest mechanical ingest. AnisotropicDiffusionLBR, FastBilateral, LabelErodeDilate, MorphologicalContourInterpolation, MultipleImageIterator, ParabolicMorphology, PhaseSymmetry, SmoothingRecursiveYvvGaussianFilter, SplitComponents, MeshNoise, SubdivisionQuadEdgeMeshFilter First — this PR's beta scope
B. Header-only with auxiliary helpers Pure-ITK plus small in-tree helpers (e.g. trivial mesh I/O); no third-party libraries. IOMeshSTL, IOMeshSWC, MeshToPolyData, BSplineGradient, Cuberille, HigherOrderAccurateGradient, GenericLabelInterpolator, RLEImage After (A) lands
C. Third-party or auxiliary library dependency Requires bundled or system third-party code (DCMTK, OpenSlide, Scanco SDK, MGH formats, libsvm, etc.). Each adds a ThirdParty/ subtree, license review, and a find_package story. AnalyzeObjectLabelMap, IOFDF, IOMeshMZ3, IOOpenSlide, IOScanco, IOTransformDCMTK, MGHIO, IOOMEZarrNGFF, SCIFIO, TotalVariation, VkFFTBackend Defer; tackle one TPL at a time
D. Executables / performance harness Ships CLI tools or benchmark drivers, not just library code. Adds install/packaging surface and CI runtime. PerformanceBenchmarking, IsotropicWavelets (CLI), Strain (CLI), MinimalPathExtraction (CLI), SkullStrip (CLI) Defer; coordinate with packaging
E. Coupled to an external release cycle Maintained or consumed by a sibling project that pins to specific releases — Elastix, SimpleITK, RTK. In-tree ingest forces ITK's release cadence onto them (or vice versa). RTK, CudaCommon (RTKConsortium), SimpleITKFilters (SimpleITK), ITKElastix-tracked modules Stay independent until consumers opt in
F. Application-oriented / multi-component Larger than a single filter; bundles algorithm + config + reference data + sometimes a UI. Higher review burden, longer compile, more LOC to maintain. SkullStrip, LesionSizingToolkit, Montage, TubeTK, Ultrasound, HASI, BoneMorphometry, BoneEnhancement Case-by-case; some may stay external
G. External-org or specialized maintainer Maintained by an outside group (SCIInstitute, ntustison, RTKConsortium, etc.). In-tree ingest changes their contribution model. Cleaver, AdaptiveDenoising, SCIFIO, Shape, TractographyTRX, CudaCommon Stay independent; mirror only with maintainer consent
H. Documentation / build-only Not user-facing code; build infrastructure or doc generators. SphinxExamples, WebAssemblyInterface (toolchain) Stay independent; ingest only if it simplifies the build
I. Legacy / deprecation candidate Likely to be removed or superseded; ingestion would lock in dead code. BioCell, IOOMEZarrNGFF (defer), TotalVariation (defer) Skip; address via deprecation path

This PR's Modules/Beta/ scope = Tier A only. The 10 modules
ingested in the follow-on PR #6086 are all Tier A (pure-ITK, no third-
party deps, no executables, no external release cycle, no large
application surface). Tiers B–I will be handled later, each with its
own per-tier scoping discussion. The provenance manifest at
Modules/Beta/<Name>.beta.cmake records UPSTREAM_URL/SHA so a future
move from Beta back out (e.g., promoting to a stable module group, or
demoting to remote because of newly added third-party deps) is fully
auditable.

@hjmjohnson hjmjohnson force-pushed the wip-inline-beta-modules branch from afbf5a2 to 88522d6 Compare April 19, 2026 12:06
@github-actions github-actions bot added the type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots label Apr 19, 2026
@hjmjohnson hjmjohnson changed the title WIP: ENH: Ingest remote modules inline under Modules/Beta/ (supersedes #6061) WIP: ENH: Add Modules/Beta/ container (infra-only for stacked beta-module ingests) Apr 19, 2026
#
# In-tree home for modules that were formerly configure-time remote
# fetches (Modules/Remote/*.remote.cmake). Each module lives at
# Modules/Beta/<Name>/ with its own itk-module.cmake and is discovered
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would welcome suggestions for the naming of this new subdirectory. I chose "Beta," but I don't have a strong affinity for it. Perhaps "Community", "PreRelease", "Contributed", .... ?

I like the overall simplicity of this scheme. It's extensible, provides containment for items we don't yet want to put into the primary core, and provides historical context as we move some elements into the core.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are they not added to their appropriate group / module according to their topic? We want to make modules simpler and more maintainable. And easier for libraries clients to use and find content.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thewtex

TLDR; Shortest path to reduced maintenance burden.

======

I was trying to keep the scope of this initial step small. The primary concern at the moment was reducing the maintainer burden by moving all code for modules primarily maintained by the core ITK developers into ITK. Thus, greatly reduces the number of repositories that need to be touched for every style, code, or CI change.

Using this beta directory adjacent to the review directory allows for the minimum amount of infrastructure change to achieve the goal of reducing maintenance burden.

Moving them from Remote or Beta to a location where clients can more easily find their content will likely involve extensive community discussions about each module: about testing code coverage, code quality, whether they have broad community appeal, ITK style conformance, and other concerns that will likely take a longer time frame to resolve per module.

This step is nearly all mechanical. I was hoping that this would allow 1/2 - 3/4 of the current remote modules to be sucked into ITK's primary repo quickly. The build behavior does not change, and it should be backward compatible with external tools that currently enable a subset of Remote modules.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thewtex If you think the correct path is to move directly from remote modules to direct ITK integration, would you recommend a single PR per remote module, 1 per day over the next month or two?

I'll make a trial remote module -> Direct ITK inclusion draft PR for Cuberille as a discussion point to contrast with the proposed method here.

Copy link
Copy Markdown
Member

@thewtex thewtex Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The primary concern at the moment was reducing the maintainer burden by moving all code for modules primarily maintained by the core ITK developers into ITK.

I agree with this goal.

Using this beta directory adjacent to the review directory allows for the minimum amount of infrastructure change to achieve the goal of reducing maintenance burden.

This duplicates code, in an unstable location, and at location that is not logically connected with existing groups and modules. This is increasing maintenance burden.

Many of the remote modules have not changed for years, have at least decent test coverage, and are not going to change. Some are easier to identify than others on which Group and Module (or a new module) they belong in. Some are trivial to locate. I do not agree that another beta group is better than putting it in the appropriate location.

@hjmjohnson I agree with you that a single PR per remote module is what is going to improve maintenance burden. It does not have to be all of them -- just the most obvious ones will make a difference on the maintenance burden and beneficial impact on the community. Then mark the remote module repository Archived on GitHub.

I'll make a trial remote module -> Direct ITK inclusion draft PR for Cuberille as a discussion point to contrast with the proposed method here.

Thank you!

@hjmjohnson hjmjohnson changed the title WIP: ENH: Add Modules/Beta/ container (infra-only for stacked beta-module ingests) ENH: Add Modules/Beta/ container (infra-only for stacked beta-module ingests) Apr 19, 2026
@hjmjohnson hjmjohnson marked this pull request as ready for review April 20, 2026 10:54
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Apr 20, 2026

Greptile Summary

Adds Modules/Beta/ as the in-tree container for modules formerly fetched via Modules/Remote/*.remote.cmake, wiring it into the root build and providing a no-op itk_beta_module_manifest() function for provenance manifests. No modules are activated by this PR — it is purely scaffolding for the follow-on ingest PR.

Confidence Score: 5/5

Safe to merge — infra-only, no modules activated, consistent with the existing Remote pattern.

The only finding is a P2 style suggestion (adding CONFIGURE_DEPENDS to the glob) that does not affect correctness at this stage since there are no .beta.cmake files yet.

No files require special attention.

Important Files Changed

Filename Overview
CMakeLists.txt Adds add_subdirectory(Modules/Beta) in the correct position between Modules/Remote and ITKModuleEnablement; no other changes.
Modules/Beta/CMakeLists.txt New file establishing the Beta module container: defines a no-op itk_beta_module_manifest() function and globs/includes *.beta.cmake provenance manifests; only minor improvement possible (CONFIGURE_DEPENDS).

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[root CMakeLists.txt] --> B[add_subdirectory Modules/Remote]
    A --> C[add_subdirectory Modules/Beta NEW]
    A --> D[include ITKModuleEnablement]

    B --> B1["file(GLOB *.remote.cmake)\nforeach → include()"]
    C --> C1["define itk_beta_module_manifest() no-op"]
    C --> C2["file(GLOB *.beta.cmake)\nforeach → include() manifests"]

    D --> E["GLOB_RECURSE */*/*/itk-module.cmake\n(discovers Modules/Beta/Name/ automatically)"]
    E --> F[Module DAG build + enablement]
Loading

Reviews (1): Last reviewed commit: "ENH: Add Modules/Beta/ container for inl..." | Re-trigger Greptile

Comment thread Modules/Beta/CMakeLists.txt Outdated
@blowekamp
Copy link
Copy Markdown
Member

blowekamp commented Apr 20, 2026

This may be been done some place, but I'd like to see a categorization of remote module with considerations to integration.

Potential things to consider:

  • Header only components (easiest to integrate)
  • Third-party or auxiliary dependency ( these are much hard )
  • Executes or performance tests
  • Actively using a separate release cycle to uses with external projects ( SimpleITK, Elastix)
  • Larger application oriented module ( e.g. Skull stripper)

@hjmjohnson
Copy link
Copy Markdown
Member Author

categorization of remote module with considerations

A partial categorization list exists in #6060. You bring up some good things to consider in expanding the 6060 preliminary categorization.

Introduces Modules/Beta/ as the in-tree home for modules that were
formerly configure-time remote fetches (Modules/Remote/*.remote.cmake).

Modules/Beta/CMakeLists.txt defines a no-op itk_beta_module_manifest()
function so that provenance files (Modules/Beta/<Name>.beta.cmake) are
valid CMake and can be include()-d without build-time side effects.
Actual modules grafted under Modules/Beta/<Name>/ are discovered via
the existing itk-module.cmake DAG scan in ITKModuleEnablement.cmake.

No modules are ingested by this commit; this only wires the container.
@hjmjohnson hjmjohnson force-pushed the wip-inline-beta-modules branch from 88522d6 to 3f2a96a Compare April 20, 2026 14:22
@github-actions github-actions bot added type:Enhancement Improvement of existing methods or implementation area:Remotes Issues affecting the Remote module labels Apr 20, 2026
@hjmjohnson
Copy link
Copy Markdown
Member Author

hjmjohnson commented Apr 20, 2026

@blowekamp good points — added an integration-difficulty axis as a complement to the domain axis in #6060. The full table is also in the PR body's Remote-module categorization <details> block; pasting it inline here for visibility:

Tier Definition Examples (from current ~60 remote modules) Ingest priority
A. Header-only / pure-ITK New filters or iterators built only on itk:: types; no third-party libraries; no executables. Cleanest mechanical ingest. AnisotropicDiffusionLBR, FastBilateral, LabelErodeDilate, MorphologicalContourInterpolation, MultipleImageIterator, ParabolicMorphology, PhaseSymmetry, SmoothingRecursiveYvvGaussianFilter, SplitComponents, MeshNoise, SubdivisionQuadEdgeMeshFilter First — this PR's beta scope
B. Header-only with auxiliary helpers Pure-ITK plus small in-tree helpers (e.g. trivial mesh I/O); no third-party libraries. IOMeshSTL, IOMeshSWC, MeshToPolyData, BSplineGradient, Cuberille, HigherOrderAccurateGradient, GenericLabelInterpolator, RLEImage After (A) lands
C. Third-party or auxiliary library dependency Requires bundled or system third-party code (DCMTK, OpenSlide, Scanco SDK, MGH formats, libsvm, etc.). Each adds a ThirdParty/ subtree, license review, and a find_package story. AnalyzeObjectLabelMap, IOFDF, IOMeshMZ3, IOOpenSlide, IOScanco, IOTransformDCMTK, MGHIO, IOOMEZarrNGFF, SCIFIO, TotalVariation, VkFFTBackend Defer; tackle one TPL at a time
D. Executables / performance harness Ships CLI tools or benchmark drivers, not just library code. Adds install/packaging surface and CI runtime. PerformanceBenchmarking, IsotropicWavelets (CLI), Strain (CLI), MinimalPathExtraction (CLI), SkullStrip (CLI) Defer; coordinate with packaging
E. Coupled to an external release cycle Maintained or consumed by a sibling project that pins to specific releases — Elastix, SimpleITK, RTK. In-tree ingest forces ITK's release cadence onto them (or vice versa). RTK, CudaCommon (RTKConsortium), SimpleITKFilters (SimpleITK), ITKElastix-tracked modules Stay independent until consumers opt in
F. Application-oriented / multi-component Larger than a single filter; bundles algorithm + config + reference data + sometimes a UI. Higher review burden, longer compile, more LOC to maintain. SkullStrip, LesionSizingToolkit, Montage, TubeTK, Ultrasound, HASI, BoneMorphometry, BoneEnhancement Case-by-case; some may stay external
G. External-org or specialized maintainer Maintained by an outside group (SCIInstitute, ntustison, RTKConsortium, etc.). In-tree ingest changes their contribution model. Cleaver, AdaptiveDenoising, SCIFIO, Shape, TractographyTRX, CudaCommon Stay independent; mirror only with maintainer consent
H. Documentation / build-only Not user-facing code; build infrastructure or doc generators. SphinxExamples, WebAssemblyInterface (toolchain) Stay independent; ingest only if it simplifies the build
I. Legacy / deprecation candidate Likely to be removed or superseded; ingestion would lock in dead code. BioCell, IOOMEZarrNGFF (defer), TotalVariation (defer) Skip; address via deprecation path

The 10 modules slated for ingest in #6086 are all Tier A (pure-ITK, no third-party deps, no CLI tools), which is what makes the Modules/Beta/ mechanism in this PR a tractable starting point. Tiers B–I would each get their own scoping discussion before any further ingest.

@blowekamp
Copy link
Copy Markdown
Member

The current "Review" module is under a "NonUnit" group. I believe the this imposes certain restrictions such as not ITK module can depend on the Review Module, and it may add all or certain dependencies.

Is this a need to codify a similar restriction for the Beta group?

ITK's cmake is in a bit of a different world now with the dependencies being specified directly in CMake now as interface libraries, and not just at the module level.

@hjmjohnson
Copy link
Copy Markdown
Member Author

ITK's cmake is in a bit of a different world now with the dependencies being specified directly in CMake now as interface libraries, and not just at the module level.

Just wanted to shout out to you with a Thank You! for all your work to make that a reality!

@thewtex
Copy link
Copy Markdown
Member

thewtex commented Apr 20, 2026

"Review" module is under a "NonUnit" group

I think we can all agree that we want to avoid the Review / NonUnit situation, if possible.

@hjmjohnson
Copy link
Copy Markdown
Member Author

This option is also rejected. Going straight to direct integration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Remotes Issues affecting the Remote module type:Enhancement Improvement of existing methods or implementation type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants