pip_audit: initial SBOM support via CycloneDX

[woodruffw]

WIP; just pushing something up for visibility.

TODO:

• Unit tests
• Support for both XML and JSON SBOM outputs
• Verify that vulnerability information is included in SBOMs

Relevant upstream tracking issues and PRs:

• Downstream type hints? CycloneDX/cyclonedx-python-lib#52
• Vulnerability model information in the JSON output format? CycloneDX/cyclonedx-python-lib#62
• model/vulnerability: fix optional type CycloneDX/cyclonedx-python-lib#61

Closes #77.

[woodruffw]

XML support is "in", but is currently failing with:

Traceback (most recent call last):
  File "/Users/william/devel/pip-audit/env/bin/pip-audit", line 33, in <module>
    sys.exit(load_entry_point('pip-audit', 'console_scripts', 'pip-audit')())
  File "/Users/william/devel/pip-audit/pip_audit/cli.py", line 215, in audit
    print(formatter.format(result))
  File "/Users/william/devel/pip-audit/pip_audit/format/cyclonedx.py", line 80, in format
    return formatter.output_as_string()  # type: ignore
  File "/Users/william/devel/pip-audit/env/lib/python3.6/site-packages/cyclonedx/output/xml.py", line 59, in output_as_string
    vulnerability=vulnerability))
  File "/Users/william/devel/pip-audit/env/lib/python3.6/site-packages/cyclonedx/output/xml.py", line 146, in _get_vulnerability_as_xml_element
    if vulnerability.has_ratings():
  File "/Users/william/devel/pip-audit/env/lib/python3.6/site-packages/cyclonedx/model/vulnerability.py", line 251, in has_ratings
    return len(self.get_ratings()) > 0
TypeError: object of type 'NoneType' has no len()

...so I'm guessing there's a part of the Vulnerability model that needs to be more initialized.

Edit: Looks like a type error in CycloneDX. Creating a PR.

[woodruffw]

Upstream bug: CycloneDX/cyclonedx-python-lib#61

[woodruffw]

Example XML output with an intentionally vulnerable requirements file:

<?xml version="1.0" encoding="UTF-8"?>
<bom serialNumber="urn:uuid:5f03d520-4ca9-4025-8781-b0711cc98cda" version="1" xmlns="http://cyclonedx.org/schema/bom/1.3" xmlns:v="http://cyclonedx.org/schema/ext/vulnerability/1.0">
  <metadata>
    <timestamp>2021-11-09T18:46:43.718858+00:00</timestamp>
    <tools>
      <tool>
        <vendor>CycloneDX</vendor>
        <name>cyclonedx-python-lib</name>
        <version>0.10.0</version>
      </tool>
    </tools>
  </metadata>
  <components>
    <component bom-ref="pkg:pypi/docker-py@1.2.3" type="library">
      <name>docker-py</name>
      <version>1.2.3</version>
      <purl>pkg:pypi/docker-py@1.2.3</purl>
    </component>
    <component bom-ref="pkg:pypi/certifi@2021.10.8" type="library">
      <name>certifi</name>
      <version>2021.10.8</version>
      <purl>pkg:pypi/certifi@2021.10.8</purl>
    </component>
    <component bom-ref="pkg:pypi/charset-normalizer@2.0.7" type="library">
      <name>charset-normalizer</name>
      <version>2.0.7</version>
      <purl>pkg:pypi/charset-normalizer@2.0.7</purl>
    </component>
    <component bom-ref="pkg:pypi/idna@3.3" type="library">
      <name>idna</name>
      <version>3.3</version>
      <purl>pkg:pypi/idna@3.3</purl>
    </component>
    <component bom-ref="pkg:pypi/pip@21.3.1" type="library">
      <name>pip</name>
      <version>21.3.1</version>
      <purl>pkg:pypi/pip@21.3.1</purl>
    </component>
    <component bom-ref="pkg:pypi/requests@2.26.0" type="library">
      <name>requests</name>
      <version>2.26.0</version>
      <purl>pkg:pypi/requests@2.26.0</purl>
    </component>
    <component bom-ref="pkg:pypi/setuptools@40.6.2" type="library">
      <name>setuptools</name>
      <version>40.6.2</version>
      <purl>pkg:pypi/setuptools@40.6.2</purl>
    </component>
    <component bom-ref="pkg:pypi/six@1.16.0" type="library">
      <name>six</name>
      <version>1.16.0</version>
      <purl>pkg:pypi/six@1.16.0</purl>
    </component>
    <component bom-ref="pkg:pypi/urllib3@1.26.7" type="library">
      <name>urllib3</name>
      <version>1.26.7</version>
      <purl>pkg:pypi/urllib3@1.26.7</purl>
    </component>
    <component bom-ref="pkg:pypi/pyyaml@5.3" type="library">
      <name>pyyaml</name>
      <version>5.3</version>
      <purl>pkg:pypi/pyyaml@5.3</purl>
      <v:vulnerabilities>
        <v:vulnerability ref="pkg:pypi/pyyaml@5.3">
          <v:id>PYSEC-2020-96</v:id>
          <v:description>A vulnerability was discovered in the PyYAML library in versions before 5.3.1, where it is susceptible to arbitrary code execution when it processes untrusted YAML files through the full_load method or with the FullLoader loader. Applications that use the library to process untrusted input may be vulnerable to this flaw. An attacker could use this flaw to execute arbitrary code on the system by abusing the python/object/new constructor.</v:description>
          <v:recommendations>
            <v:recommendation>Upgrade</v:recommendation>
          </v:recommendations>
          <v:advisories>
            <v:advisory>Upgrade: 5.3.1</v:advisory>
          </v:advisories>
        </v:vulnerability>
        <v:vulnerability ref="pkg:pypi/pyyaml@5.3">
          <v:id>PYSEC-2021-142</v:id>
          <v:description>A vulnerability was discovered in the PyYAML library in versions before 5.4, where it is susceptible to arbitrary code execution when it processes untrusted YAML files through the full_load method or with the FullLoader loader. Applications that use the library to process untrusted input may be vulnerable to this flaw. This flaw allows an attacker to execute arbitrary code on the system by abusing the python/object/new constructor. This flaw is due to an incomplete fix for CVE-2020-1747.</v:description>
          <v:recommendations>
            <v:recommendation>Upgrade</v:recommendation>
          </v:recommendations>
          <v:advisories>
            <v:advisory>Upgrade: 5.4</v:advisory>
          </v:advisories>
        </v:vulnerability>
      </v:vulnerabilities>
    </component>
  </components>
</bom>

[woodruffw]

One notable thing: the CycloneDX JSON serialization format doesn't seem to include the vulnerability extension. I'm not sure what's up with that -- if it's a bug in the Python library or because the representation is only well specified in the XML schema(s). Will follow up with an upstream issue.

Edit: Upstream: CycloneDX/cyclonedx-python-lib#62

[woodruffw]

Turns out vulnerability information is intentionally not supported in the current JSON output format. Version 1.4 of CycloneDX will resolve that by promoting vulnerability information from an extension to a full-fledged part of the spec.

[tetsuo-cpp]

Awesome! LGTM.

[woodruffw]

The only remaining blocker here is CycloneDX/cyclonedx-python-lib#61. Once that's in, I'll merge this as an initial version and we can iterate on adding even more metadata.