ENH: Generate SPDX 2.3 Software Bill of Materials at configure time#5817
ENH: Generate SPDX 2.3 Software Bill of Materials at configure time#5817
Conversation
|
The VTK modular system uses arguments to the main module macro for the license information: https://docs.vtk.org/en/latest/api/cmake/vtkModule.html#vtk-module-file-contents I think it is worth a human conversation to decide if this information should have a public interface with the main module macro or if it should be separate. The man macro certainly could call a private function such as these. I was just looking into moving FFTW into a third party library structure. With the varied backed-ends supported it is a dynamic case to support, and very relevant to license requirements. Having all the module information defined, along with the license information maybe beneficial to keep some information consistent, and apply certain logic or enforcement such as defaulting internal module to ITK license while requiring remote modules to have these attributes. |
|
Copilot's first PR seems better to me (easier to maintain), with license information distributed in |
|
@copilot Make license and other per-module information distributed in |
|
More importantly, I am unfamiliar with SPDX and therefore will not be able to tell whether Copilot does a great or terrible job (or somewhere in between). |
b6e40e9 to
bc0d0a2
Compare
|
@copilot Make license and other per-module information distributed in |
bc0d0a2 to
8fbee2b
Compare
|
@copilot Make license and other per-module information distributed in |
Done in 8692d2d and 50daeeb. SPDX metadata is now declared directly in each |
50daeeb to
7efe75e
Compare
Implement build-time generation of a Software Bill of Materials (SBOM) in SPDX 2.3 JSON format. The SBOM documents ITK and all enabled third-party dependencies with their names, licenses, and dependency relationships. - Add CMake/ITKSBOMGeneration.cmake with SBOM generation logic - Add ITK_GENERATE_SBOM option (default ON) to CMakeLists.txt - Generate sbom.spdx.json in the build directory at configure time - Install sbom.spdx.json alongside LICENSE, NOTICE, and README.md - Support FFTW as optional GPL dependency in SBOM - Provide itk_sbom_register_package() for remote modules to extend Co-authored-by: dzenanz <1792121+dzenanz@users.noreply.github.com>
3e1311b to
3902e8a
Compare
Move SPDX license and per-module metadata from the centralized lookup table in ITKSBOMGeneration.cmake into each ThirdParty module's itk-module.cmake file for co-location and easier maintenance. - Extend itk_module() macro to accept SPDX_LICENSE, SPDX_DOWNLOAD_LOCATION, SPDX_COPYRIGHT, SPDX_CUSTOM_LICENSE_TEXT, and SPDX_CUSTOM_LICENSE_NAME - Add SPDX metadata to all 23 ThirdParty itk-module.cmake files - Simplify ITKSBOMGeneration.cmake to read from module declarations Co-authored-by: dzenanz <1792121+dzenanz@users.noreply.github.com>
Custom license names and extracted texts were written directly into the JSON output without escaping. Any text containing quotes, backslashes, or newlines would produce invalid JSON. Apply _itk_sbom_json_escape() to both the name and extractedText fields before embedding.
Add SPDX_VERSION to the itk_module() macro for declaring the version of each vendored third-party dependency. Populate versions for 11 modules extractable from source headers or UpdateFromUpstream.sh tags: Eigen3 (3.4.90), Expat (2.7.4), GoogleTest (1.17.0), HDF5 (1.14.5), JPEG/libjpeg-turbo (3.0.4), MINC (2.4.06), NIFTI (3.0.0), OpenJPEG (2.5.4), PNG (1.6.54), TIFF (4.7.0), ZLIB (2.2.5) Also fix Eigen3 license from "MPL-2.0" to "MPL-2.0 OR Apache-2.0" (dual-licensed) and update NIFTI download URL to the current GitHub repository.
Add ITKSBOMValidation test that validates the generated sbom.spdx.json at test time: checks JSON syntax, required SPDX 2.3 fields, at least one package, DESCRIBES relationship, unique SPDX IDs, and extracted license entries. Runs only when ITK_GENERATE_SBOM=ON and BUILD_TESTING=ON.
Add VerifySPDXVersions.py that cross-checks SPDX_VERSION declared in each itk-module.cmake against the tag in UpdateFromUpstream.sh. For modules with parseable version tags (v1.2.3, R_2_7_4, hdf5_1.14.5, bare semver), the test verifies consistency. Modules tracking master, commit SHAs, or custom ITK tags are skipped. This catches the case where a vendored dependency is updated via UpdateFromUpstream.sh but SPDX_VERSION is not bumped. Currently validates 8 of 16 modules with UpdateFromUpstream.sh scripts.
3902e8a to
bc8d90b
Compare
Add AUTHOR_WARNING during SBOM generation for enabled ThirdParty modules that have no SPDX_LICENSE declared in their itk-module.cmake. This encourages remote module and ThirdParty maintainers to add SPDX metadata for supply chain compliance.
|
| Filename | Overview |
|---|---|
| CMake/ITKSBOMGeneration.cmake | New file: core SBOM generation. P1 — _pkg_copyright, _pkg_download, _pkg_license, and _pkg_version are inserted into JSON without escaping (unlike _pkg_description). P2 — foreach over unquoted list variable; hardcoded licenseListVersion. |
| CMake/ITKSBOMValidation.cmake | New file: CTest validation and version-consistency tests. P1 — ${Python3_EXECUTABLE} is used without a guard; missing Python3 silently registers a broken test. |
| CMake/ITKModuleMacros.cmake | Adds six new SPDX keyword arguments to itk_module(); regex and elseif chain updated consistently; no issues found. |
| CMakeLists.txt | Adds ITK_GENERATE_SBOM option (default ON), wires up itk_generate_sbom() and ITKSBOMValidation includes after module enablement, and adds install rule for sbom.spdx.json; straightforward. |
| Modules/ThirdParty/PNG/itk-module.cmake | P2 — SPDX_LICENSE "Libpng-2.0" is not a recognised SPDX identifier for libpng 1.6.x; the correct identifier is libpng. |
| Utilities/Maintenance/VerifySPDXVersions.py | New helper: cross-checks SPDX_VERSION in itk-module.cmake against UpdateFromUpstream.sh tags for 8 modules; version normalisation patterns look correct; modules with SHA/master/custom tags are skipped. |
| Modules/ThirdParty/TIFF/itk-module.cmake | Adds SPDX metadata; "libtiff" is a valid SPDX identifier for the libtiff licence; no issues. |
| Modules/ThirdParty/Eigen3/itk-module.cmake | Adds SPDX metadata; MPL-2.0 OR Apache-2.0 licence expression is correct for Eigen 3.4.x; no issues. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[CMake configure] --> B{ITK_GENERATE_SBOM?}
B -- OFF --> Z[skip]
B -- ON --> C[include ITKSBOMGeneration]
C --> D[itk_generate_sbom]
D --> E[Loop over ITK_MODULES_ENABLED]
E --> F{SPDX_LICENSE set?}
F -- NO, ThirdParty --> G[AUTHOR_WARNING]
F -- NO, non-ThirdParty --> H[skip]
F -- YES --> I[Append package JSON
⚠️ no escape on copyright/download/license]
D --> J{ITK_USE_FFTW?}
J -- YES --> K[Append FFTW package]
D --> L[get_property ITK_SBOM_EXTRA_PACKAGES
foreach unquoted ⚠️]
L --> M[Append remote module packages]
D --> N[Build relationships array]
D --> O[Build hasExtractedLicensingInfo]
D --> P[file WRITE sbom.spdx.json]
P --> Q[include ITKSBOMValidation]
Q --> R{BUILD_TESTING?}
R -- NO --> Z
R -- YES --> S{Python3_EXECUTABLE set?
⚠️ not checked}
S --> T[add_test ITKSBOMValidation
Python JSON validation]
S --> U[add_test ITKSBOMVersionConsistency
VerifySPDXVersions.py]
Reviews (1): Last reviewed commit: "ENH: Warn when ThirdParty modules lack S..." | Re-trigger Greptile
|
Addressing the discussion points from @blowekamp and @dzenanz: Distributed metadata (dzenanz's request): Done — commit 8692d2d moved all SPDX metadata into each VTK-style integration with module macro (blowekamp's suggestion): This is exactly what was implemented — the SPDX parameters are arguments to the existing FFTW licensing (blowekamp): FFTW is handled as a special case in Greptile findings: All 5 inline findings (2 P1, 3 P2) have been reviewed and acknowledged — JSON escaping gaps, missing Python3 guard, |
Address all findings from greptile code review: P1: Escape all user-supplied fields (version, download location, license, copyright) through _itk_sbom_json_escape() before JSON interpolation — both in itk_generate_sbom() and itk_sbom_register_package(). Previously only description was escaped. P1: Add Python3_EXECUTABLE guard in ITKSBOMValidation.cmake to skip SBOM validation tests when Python3 is not available. P2: Use foreach(... IN LISTS ...) instead of unquoted variable expansion to prevent re-splitting on embedded semicolons. P2: Fix PNG SPDX license identifier from "Libpng-2.0" to "Libpng" (the valid SPDX license list entry for libpng 1.6.x). P2: Expose licenseListVersion as ITK_SBOM_SPDX_LICENSE_LIST_VERSION cache variable (default "3.25") instead of hardcoding "3.22".
|
All 5 greptile findings fixed in 6d31421. Fix summary
|
Add SPDX 2.3 SBOM generation at CMake configure time, producing
sbom.spdx.jsonin the build directory with all enabled ThirdParty modules and their metadata. Per-module SPDX metadata is co-located in each module'sitk-module.cmakefile. Fixes #4302.Commit breakdown (7 independent commits)
ITKSBOMGeneration.cmakewithitk_generate_sbom(),itk_sbom_register_package()for remote modules, FFTW GPL tracking, install ruleitk_module()withSPDX_LICENSE,SPDX_DOWNLOAD_LOCATION,SPDX_COPYRIGHT,SPDX_CUSTOM_LICENSE_TEXT/NAME; populate all 23 ThirdParty moduleshasExtractedLicensingInfowere not escapeditk_module()parameter; versions for 11 modules (Eigen3 3.4.90, Expat 2.7.4, GoogleTest 1.17.0, HDF5 1.14.5, JPEG/libjpeg-turbo 3.0.4, MINC 2.4.06, NIFTI 3.0.0, OpenJPEG 2.5.4, PNG 1.6.54, TIFF 4.7.0, ZLIB 2.2.5); fix Eigen3 license toMPL-2.0 OR Apache-2.0ITKSBOMValidationCTest checks JSON syntax, SPDX 2.3 fields, package uniquenessVerifySPDXVersions.pycross-checksSPDX_VERSIONagainstUpdateFromUpstream.shtags (8 of 16 modules with parseable version tags)AUTHOR_WARNINGfor enabled ThirdParty modules lackingSPDX_LICENSERemote module extensibility
Known limitations
SPDX_VERSION(trackmaster, commit SHAs, or custom tags inUpdateFromUpstream.sh)BSD-3-Clausebut contain sub-components with different licenses)NOTICEfile is not yet deprecated/replaced