ENH: Add Utilities/Maintenance/RemoteModuleIngest tooling

[hjmjohnson]

Separates the remote-module ingestion tooling from any specific module ingest (see #6093 for the first concrete use case). This PR adds a new directory Utilities/Maintenance/RemoteModuleIngest/ containing the driver script, strategy documents, and AI-agent guidance. Relates to #6060.

Directory layout

Utilities/Maintenance/RemoteModuleIngest/
├── ingest-remote-module.sh   # v3 whitelist-based driver (executable, 280 lines)
├── README.md                 # Human-facing quickstart + convention
├── AGENTS.md                 # AI-agent pre-flight + decision tree + escalation triggers
├── INGESTION_STRATEGY.md     # Policy: whitelist, modes, thresholds, CID normalization, examples policy
├── AUDIT_DESIGN.md           # Pre-ingest audit pipeline + recommend_mode() pseudocode
└── CLEANUP_CHECKLIST.md      # Safety-net list for the post-merge STYLE commit

What the script does

1. Clones upstream (git clone --no-local --mirror).
2. Runs git filter-repo --paths include/ src/ test/ wrapping/ CMakeLists.txt itk-module.cmake --to-subdirectory-filter Modules/<Group>/<Module> --prune-empty always.
3. A second filter-repo pass strips CTestConfig.cmake (always points at a standalone CDash project, never applies in-tree).
4. Reports a content-link inventory (.md5 / .shaNNN / .cid) and warns if non-.cid links remain.
5. In non-dry-run mode: merges the filter-repo'd upstream into the current ITK branch with --allow-unrelated-histories --no-ff, auto-populating Co-authored-by: trailers from the upstream log.
6. Prints the recommended follow-on commits (README, remote.cmake delete, CI opt-in, optional CID norm).

--dry-run --keep-tempdir lets reviewers preview the post-filter tree before committing anything.

Consensus decisions captured in the strategy doc

• Whitelist, not blacklist (agreed with @dzenanz on ENH: Ingest AnisotropicDiffusionLBR into Modules/Filtering #6093): only code, headers, tests, wrapping, and module build descriptors cross the merge boundary. Everything else stays in the archived upstream repo.
• CID normalization mandatory (agreed with @dzenanz on ENH: Ingest AnisotropicDiffusionLBR into Modules/Filtering #6093): every .md5 / .shaNNN content-link converts to .cid before merge. The tree-wide sweep PR path is preferred over per-module conversions, following the precedent at ITK commit f3899ce8c6.
• Examples go to top-level (agreed with @dzenanz on ENH: Ingest AnisotropicDiffusionLBR into Modules/Filtering #6093): per-module examples/ directories are NOT ingested inline; they either stay in the archived upstream (default) or relocate to Examples/ via a separate follow-up PR.
• Commit count is not a gate (agreed with @dzenanz on ENH: Ingest AnisotropicDiffusionLBR into Modules/Filtering #6093): with the whitelist in place, surviving commits reflect real upstream authorship and should not push a module into squash. Only size metrics (pack delta, largest blob) are mode-selection criteria.
• Bloat protection via whitelist (agreed with @blowekamp on ENH: Ingest AnisotropicDiffusionLBR into Modules/Filtering #6093): Old/, paper/, docs/figures/, demos, and CI scaffolding can never enter ITK's pack because they are outside the whitelist by construction.
• Primary + co-authors preserved in all modes: driver auto-derives the author list from the filter-repo'd git log and appends Co-authored-by: trailers to the merge commit body.

How this PR relates to #6093

• ENH: Ingest AnisotropicDiffusionLBR into Modules/Filtering #6093 — ingests ITKAnisotropicDiffusionLBR as the first module using this tooling (re-ingested on 2026-04-22 with the v3 approach).
• This PR — lands the tooling so subsequent ingests reviewers can see the script being invoked, not rediscovered per PR.

Reviewers can land this PR first (it's small, self-contained, touches only Utilities/Maintenance/), then the follow-on module ingest PRs can reference Utilities/Maintenance/RemoteModuleIngest/ingest-remote-module.sh <Module> <Group> as the reproducible command that produced the merge commit.

[dzenanz]

Looks good on a glance.

[dzenanz]

The described changes sound good. I did not take a good look at new CID-related scripts.

[greptile-apps]

Greptile Summary

This PR adds Utilities/Maintenance/RemoteModuleIngest/ — a self-contained tooling directory with a whitelist-based git filter-repo driver script and companion strategy/agent guidance documents for reproducibly ingesting ITK remote modules into the main source tree.

• P1 — ingest-remote-module.sh line 273: git fetch ingest-src-tmp main:ingest-src-tmp-main hardcodes the branch name main; many older remote-module repos still default to master, causing the fetch (and therefore the entire ingest) to fail silently under "Merge failed" even when the filter-repo pass succeeds.
• P2 — exit code table (ingest-remote-module.sh lines 43–48): documented codes 2 = filter-repo failure and 3 = merge failure don't match the implementation, where both error paths call die (exit 1) and code 2 is actually the CID-normalization gate.
• P2 — cid-normalize.sh line 190: trap ... RETURN has no effect outside a shell function body; temp file cleanup on unexpected exits is unreliable.

Confidence Score: 4/5

The P1 hardcoded main branch will break ingests from any master-default upstream repo; fix before merging to avoid silent failures in the first real use case.

One real P1 defect (hardcoded branch name) that will break the script for any upstream using a non-main default branch. All other findings are P2. The documentation and strategy files are thorough and well-structured. Score reflects a single blocking issue that is easy to fix.

Utilities/Maintenance/RemoteModuleIngest/ingest-remote-module.sh — hardcoded main branch name (line 273) and exit code table mismatch.

Important Files Changed

Filename	Overview
Utilities/Maintenance/RemoteModuleIngest/ingest-remote-module.sh	Main driver script (280 lines): hardcodes branch name main when fetching from the filter-repo'd mirror (P1 – breaks ingests from master-default repos); exit code comments document codes 2 and 3 that don't match the implementation (P2).
Utilities/Maintenance/RemoteModuleIngest/cid-normalize.sh	Content-link converter: trap ... RETURN used outside a function has no effect, leaving temp files unclean on unexpected exits (P2); logic and fallback chain are otherwise correct.
Utilities/Maintenance/RemoteModuleIngest/verify-cid-access.sh	Pre-push CID gateway checker: correct flow; uses printf "$g_fmt" "$cid" with a known-safe base32 CID (SC2059 suppressed intentionally); all exit paths covered.
Utilities/Maintenance/RemoteModuleIngest/AGENTS.md	AI-agent guidance document: thorough decision tree, escalation triggers, and post-merge validation steps; no code issues.
Utilities/Maintenance/RemoteModuleIngest/README.md	Human-facing quickstart and convention overview; documentation only.
Utilities/Maintenance/RemoteModuleIngest/INGESTION_STRATEGY.md	Policy document capturing whitelist rationale, CID-normalization mandate, examples routing, and author-preservation decisions; documentation only.
Utilities/Maintenance/RemoteModuleIngest/AUDIT_DESIGN.md	Audit pipeline design and audit.json schema; documentation only.
Utilities/Maintenance/RemoteModuleIngest/CLEANUP_CHECKLIST.md	Post-merge style-commit safety checklist; documentation only.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A([ingest-remote-module.sh]) --> B[Preflight checks\ngit-filter-repo, clean tree, ITK root]
    B --> C[Step 1: git clone --mirror upstream]
    C --> D[Step 2: filter-repo whitelist pass\ninclude/ src/ test/ wrapping/\nCMakeLists.txt itk-module.cmake\n--to-subdirectory-filter Modules/Group/Module]
    D --> E{CTestConfig.cmake\nin result?}
    E -- yes --> F[Step 3: filter-repo invert-paths\nstrip CTestConfig.cmake]
    E -- no --> G
    F --> G[Step 4: inventory\n.md5 / .shaNNN / .cid counts]
    G --> H{--dry-run?}
    H -- yes --> I([Exit 0: report only])
    H -- no --> J[Step 6: collect authors\nbuild Co-authored-by trailers]
    J --> K[Step 7: git merge --allow-unrelated-histories\n⚠ hardcodes branch 'main']
    K --> L[Print follow-up commit checklist]
    L --> M{non-.cid\nlinks remain?}
    M -- no --> N([Exit 0: success])
    M -- yes --> O([Exit 2: ACTION REQUIRED\nrun cid-normalize.sh])

Loading

_{Reviews (1): Last reviewed commit: "DOC: Add strategy + agent guidance for R..." | Re-trigger Greptile}