Capture and report CDX SBOM header information from a load_sbom pipeline

[mjherzog]

Somewhat like a Docker image, an SBOM has important "header" information that should be displayed in SCIO as Project Data.
For CycloneDX v1.5 JSON format the sections/fields are:

• bomFormat (required)
• specVersion (required(
• serialNumber
• version
• metadata
  • timestamp
  • lifecycles
  • tools
  • authors
  • component (NB: SCIO currently reports this in Packages, but it should be separate from the packages reported from the components section
  • manufacture (NB: deprecated in CDX v1.6)
  • supplier
  • licenses
  • properties
    Some of the data like metadata/licenses could be voluminous, but we rarely see metadata/licenses data in recent practice.
    We should also add this feature for SPDX Documents - https://spdx.github.io/spdx-spec/v2.3/document-creation-information/ - but that is a lower priority.

[tdruez]

Implementation in progress at #1266
Support for CycloneDX and SPDX headers added, as an example, loading the following SBOM files:

• https://github.com/CycloneDX/bom-examples/blob/master/SBOM/laravel-7.12.0/bom.1.4.json
• https://github.com/CycloneDX/bom-examples/blob/master/SBOM/juice-shop/v11.1.2/bom.json#L3
• https://github.com/nexB/scancode.io/blob/main/scanpipe/tests/data/asgiref-3.3.0.spdx.json

The Project extra_data contains the following, @mjherzog could you confirm this would work for you?

sboms_headers:
  asgiref-3.3.0.spdx.json:
    name: scancodeio_asgiref
    SPDXID: SPDXRef-DOCUMENT
    comment: |
      Generated with ScanCode.io and provided on an "AS IS" BASIS, WITHOUT WARRANTIES
      OR CONDITIONS OF ANY KIND, either express or implied.
      No content created from ScanCode.io should be considered or used as legal advice.
      Consult an Attorney for any legal advice.
      ScanCode.io is a free software code scanning tool from nexB Inc. and others
      licensed under the Apache License version 2.0.
      ScanCode is a trademark of nexB Inc.
      Visit https://github.com/nexB/scancode.io for support and download.
    dataLicense: CC0-1.0
    spdxVersion: SPDX-2.3
    creationInfo:
      created: '2000-01-01T01:02:03Z'
      creators:
        - 'Tool: ScanCode.io-31.0.0'
      licenseListVersion: '3.20'
    documentNamespace: https://scancode.io/spdxdocs/8d3058f3-ec1f-487d-8c5f-b2d3b26cda3e
  juice-shop-v11.1.2.bom.json:
    version: 1
    metadata:
      tools:
        - name: Node.js module
          vendor: CycloneDX
          version: 2.0.0
      component:
        name: juice-shop
        purl: pkg:npm/juice-shop@11.1.2
        type: library
        bom-ref: pkg:npm/juice-shop@11.1.2
        version: 11.1.2
        licenses:
          - license:
              id: MIT
        description: Probably the most modern and sophisticated insecure web application
        externalReferences:
          - url: https://owasp-juice.shop
            type: website
          - url: https://github.com/bkimminich/juice-shop/issues
            type: issue-tracker
          - url: git+https://github.com/bkimminich/juice-shop.git
            type: vcs
      timestamp: '2020-08-03T03:20:53.771Z'
    bomFormat: CycloneDX
    specVersion: '1.2'
    serialNumber: urn:uuid:1f860713-54b9-4253-ba5a-9554851904af
  laravel-7.12.0.bom.1.4.json:
    version: 1
    metadata:
      tools:
        - name: cyclonedx-php-composer
          vendor: cyclonedx
          version: in-dev
          externalReferences:
            - url: ../..
              type: distribution
            - url: https://github.com/CycloneDX/cyclonedx-php-composer/#readme
              type: website
              comment: as detected from Composer manifest 'homepage'
            - url: https://github.com/CycloneDX/cyclonedx-php-composer/issues
              type: issue-tracker
              comment: as detected from Composer manifest 'support.issues'
            - url: https://github.com/CycloneDX/cyclonedx-php-composer/
              type: vcs
              comment: as detected from Composer manifest 'support.source'
      component:
        name: cyclonedx-php-composer-demo
        purl: pkg:composer/cyclonedx/cyclonedx-php-composer-demo@dev-master
        type: application
        group: cyclonedx
        author: Jan Kowalleck
        bom-ref: cyclonedx/cyclonedx-php-composer-demo-dev-master
        version: dev-master
        properties:
          - name: cdx:composer:package:type
            value: project
        description: demo of cyclonedx/cyclonedx-php-composer with a pinned version of laravel/framework
    bomFormat: CycloneDX
    specVersion: '1.4'

[mjherzog]

This looks good to me - thank you. One concern is keeping up with the relevant spec changes for this data, especially for CDX where there seem to be a faster rate of change for this data than SPDX.

[tdruez]

@mjherzog That should not be a major problem.

The current implementation is based on simple lists of the first-level properties that we want to extract.
See https://github.com/nexB/scancode.io/pull/1266/files#diff-ab05a75105e46033069d1b0bb4b067d7f6fb7083da20fac160a9289e606c853fR337-R354
It will be easy enough to add any new fields added to the SPECs that we would want to capture.

[tdruez]

Merged and deployed.