Skip to content

Commit

Permalink
Support for CycloneDX schema version 1.4 (#108)
Browse files Browse the repository at this point in the history 
BREAKING CHANGE: Support for CycloneDX 1.4. This includes:
- Support for `tools` having `externalReferences`
- Allowing `version` for a `Component` to be optional in 1.4
- Support for `releaseNotes` per `Component`
- Support for the core schema implementation of Vulnerabilities (VEX)

Other changes included in this PR:
- Unit tests now include schema validation (we've left schema validation out of the core library due to dependency bloat)
- Fixes to ensure schema is adhered to in 1.0
- URI's are now used throughout the library through a new `XsUri` class to provide URI validation
- Documentation is now hosted on readthedocs.org (https://cyclonedx-python-library.readthedocs.io/)
- `$schema` is now included in JSON BOMs
- Concrete Parsers how now been moved into downstream projects to keep this libraries focus on modelling and outputting CycloneDX - see https://github.com/CycloneDX/cyclonedx-python
- Added reference to release of this library on Anaconda

Signed-off-by: Paul Horton <phorton@sonatype.com>

Signed-off-by: Jan Kowalleck <jan.kowalleck@gmail.com>

Co-authored-by: Paul Horton <phorton@sonatype.com>

Co-authored-by: Jan Kowalleck <jan.kowalleck@gmail.com>
  • Loading branch information
@madpah @jkowalleck
madpah and jkowalleck committed Jan 13, 2022
1 parent 3058afc commit 7fb6da9
Show file tree
Hide file tree
Showing 100 changed files with 10,510 additions and 3,735 deletions.
34 changes: 0 additions & 34 deletions .github/workflows/docs.yml
View file

This file was deleted.

30 changes: 13 additions & 17 deletions .github/workflows/manual-release-candidate.yml
View file
Original file line number Diff line number Diff line change
@@ -1,22 +1,17 @@
name: Manual Release Candidate
name: Manual Pre Release Publish

on:
workflow_dispatch:
inputs:
branch:
description: 'Branch to produce Release Candidate from'
release_candidate_suffix:
description: 'RC Suffix e.g. rc0, beta1, alpha2. Do not include a leading hyphen.'
required: true
type: string
rc_number:
description: 'RC Number'
default: 0
required: true
type: number

jobs:
release:
release_candidate:
runs-on: ubuntu-latest
concurrency: release
concurrency: release_candidate
steps:
- name: Checkout code
uses: actions/checkout@v2
Expand All @@ -31,12 +26,13 @@ jobs:
poetry config virtualenvs.create false
poetry install
python -m pip install python-semantic-release
- name: Determine RC candidate version
- name: Apply Pre Release Version
run: |
RC_VERSION="$(python-semantic-release --noop --major print-version)rc${{ inputs.rc_number }})"
RC_VERSION="$(semantic-release --noop --major print-version)-${{ github.event.inputs.release_candidate_suffix }}"
echo "RC Version will be: ${RC_VERSION}"
# - name: Python Semantic Release
# uses: relekang/python-semantic-release@master
# with:
# github_token: ${{ secrets.GITHUB_TOKEN }}
# pypi_token: ${{ secrets.PYPI_TOKEN }}
poetry version ${RC_VERSION}
poetry build
- name: Publish Pre Release 📦 to PyPI
uses: pypa/gh-action-pypi-publish@master
with:
password: ${{ secrets.PYPI_TOKEN }}
3 changes: 3 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,6 @@ html/

# mypy caches
/.mypy_cache

# Exlude built docs
docs/_build
45 changes: 45 additions & 0 deletions .readthedocs.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# encoding: utf-8

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0

# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

# Set the version of Python and other tools you might need
build:
os: ubuntu-20.04
tools:
python: "3.9"
# You can also specify other tool versions:
# nodejs: "16"
# rust: "1.55"
# golang: "1.17"

# Build documentation in the docs/ directory with Sphinx
sphinx:
configuration: docs/conf.py

# Formats
formats: all

# Optionally declare the Python requirements required to build your docs
python:
install:
- method: pip
path: .
- requirements: docs/requirements.txt
199 changes: 5 additions & 194 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

[![shield_gh-workflow-test]][link_gh-workflow-test]
[![shield_pypi-version]][link_pypi]
[![shield_conda-forge-version]][link_conda-forge]
[![shield_license]][license_file]
[![shield_website]][link_website]
[![shield_slack]][link_slack]
Expand All @@ -22,199 +23,7 @@ Additionally, you can use this module yourself in your application to programmat

CycloneDX is a lightweight BOM specification that is easily created, human-readable, and simple to parse.

## Installation

Install from pypi.org as you would any other Python module:

```shell
pip install cyclonedx-python-lib
```

## Architecture

This module break out into three key areas:

1. **Parser**: Use a parser that suits your needs to automatically gather information about your environment or
application
2. **Model**: Internal models used to unify data from different parsers
3. **Output**: Choose and configure an output which allows you to define output format as well as the CycloneDX schema
version

### Parsing

You can use one of the parsers to obtain information about your project or environment. Available parsers:

| Parser | Class / Import | Description |
| ------- | ------ | ------ |
| CondaListJsonParser | `from cyclonedx.parser.conda import CondaListJsonParser` | Parses input provided as a `str` that is output from `conda list --json` |
| CondaListExplicitParser | `from cyclonedx.parser.conda import CondaListExplicitParser` | Parses input provided as a `str` that is output from `conda list --explicit` or `conda list --explicit --md5` |
| Environment | `from cyclonedx.parser.environment import EnvironmentParser` | Looks at the packaged installed in your current Python environment. |
| PipEnvParser | `from cyclonedx.parser.pipenv import PipEnvParser` | Parses `Pipfile.lock` content passed in as a string. |
| PipEnvFileParser | `from cyclonedx.parser.pipenv import PipEnvFileParser` | Parses the `Pipfile.lock` file at the supplied path. |
| PoetryParser | `from cyclonedx.parser.poetry import PoetryParser` | Parses `poetry.lock` content passed in as a string. |
| PoetryFileParser | `from cyclonedx.parser.poetry import PoetryFileParser` | Parses the `poetry.lock` file at the supplied path. |
| RequirementsParser | `from cyclonedx.parser.requirements import RequirementsParser` | Parses a multiline string that you provide that conforms to the `requirements.txt` [PEP-508] standard. |
| RequirementsFileParser | `from cyclonedx.parser.requirements import RequirementsFileParser` | Parses a file that you provide the path to that conforms to the `requirements.txt` [PEP-508] standard. |

#### Example

```py
from cyclonedx.parser.environment import EnvironmentParser

parser = EnvironmentParser()
```

#### Notes on Requirements parsing

CycloneDX software bill-of-materials require pinned versions of requirements. If your `requirements.txt` does not have
pinned versions, warnings will be recorded and the dependencies without pinned versions will be excluded from the
generated CycloneDX. CycloneDX schemas (from version 1.0+) require a component to have a version when included in a
CycloneDX bill of materials (according to schema).

If you need to use a `requirements.txt` in your project that does not have pinned versions an acceptable workaround
might be to:

```shell
pip install -r requirements.txt
pip freeze > requirements-frozen.txt
```

You can then feed in the frozen requirements from `requirements-frozen.txt` _or_ use the `Environment` parser one you
have `pip install`ed your dependencies.

### Modelling

You can create a BOM Model from either a Parser instance or manually using the methods avaialbel directly on the `Bom` class.

The model also supports definition of vulnerabilities for output using the CycloneDX schema extension for
[Vulnerability Disclosures](https://cyclonedx.org/use-cases/#vulnerability-disclosure) as of version 0.3.0.

**Note:** Known vulnerabilities associated with Components can be sourced from various data sources, but this library
will not source them for you. Perhaps look at [Jake][jake] if you're interested in this.

#### Example from a Parser

```py
from cyclonedx.model.bom import Bom
from cyclonedx.parser.environment import EnvironmentParser

parser = EnvironmentParser()
bom = Bom.from_parser(parser=parser)
```

### Generating Output

Once you have an instance of a `Bom` you can produce output in either `JSON` or `XML` against any of the supporting CycloneDX schema versions as you require.

We provide two helper methods:

* Output to string (for you to do with as you require)
* Output directly to a filename you provide

#### Example as JSON

```py
from cyclonedx.output import get_instance, OutputFormat

outputter = get_instance(bom=bom, output_format=OutputFormat.JSON)
outputter.output_as_string()
```

#### Example as XML

```py
from cyclonedx.output import get_instance, SchemaVersion

outputter = get_instance(bom=bom, schema_version=SchemaVersion.V1_2)
outputter.output_to_file(filename='/tmp/sbom-v1.2.xml')
```

## Library API Documentation

The Library API Documentation is available online at [https://cyclonedx.github.io/cyclonedx-python-lib/](https://cyclonedx.github.io/cyclonedx-python-lib/).

## Schema Support

This library is a work in progress and complete support for all parts of the CycloneDX schema will come in future releases.

Here is a summary of the parts of the schema supported by this library:

_Note: We refer throughout using XPath, but the same is true for both XML and JSON output formats._

<table width="100%">
<thead>
<tr>
<th>XPath</th>
<th>Support v1.3</th>
<th>Support v1.2</th>
<th>Support v1.1</th>
<th>Support v1.0</th>
<th>Notes</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>/bom</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>Y</td>
<td>
This is the root element and is supported with all it's defined attributes.
</td>
</tr>
<tr>
<td><code>/bom/metadata</code></td>
<td>Y</td><td>Y</td><td>N/A</td><td>N/A</td>
<td>
<code>timestamp</code> and <code>tools</code> are currently supported
</td>
</tr>
<tr>
<td><code>/bom/components</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>Y</td>
<td>&nbsp;</td>
</tr>
<tr>
<th colspan="6"><strong><code>/bom/components/component</code></strong></th>
</tr>
<tr>
<td><code>./author</code></td>
<td>Y</td><td>Y</td><td>N/A</td><td>N/A</td>
<td>&nbsp;</td>
</tr>
<tr>
<td><code>./name</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>Y</td>
<td>&nbsp;</td>
</tr>
<tr>
<td><code>./version</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>Y</td>
<td>&nbsp;</td>
</tr>
<tr>
<td><code>./purl</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>Y</td>
<td>&nbsp;</td>
</tr>
<tr>
<td><code>./externalReferences</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>N/A</td>
<td>Not all Parsers have this information. It will be populated where there is information available.</td>
</tr>
<tr>
<td><code>./hashes</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>Y</td>
<td>
These are supported when programmatically creating a <code>Bom</code> - these will not currently be
automatically populated when using a <code>Parser</code>.
</td>
</tr>
</tbody>
</table>

### Notes on Schema Support

* N/A is where the CycloneDX standard does not include this
* If the table above does not refer to an element, it is not currently supported
View our documentation [here](https://cyclonedx-python-library.readthedocs.io/).

## Python Support

Expand Down Expand Up @@ -244,14 +53,16 @@ See the [LICENSE][license_file] file for the full license.
[contributing_file]: https://github.com/CycloneDX/cyclonedx-python-lib/blob/master/CONTRIBUTING.md

[shield_gh-workflow-test]: https://img.shields.io/github/workflow/status/CycloneDX/cyclonedx-python-lib/Python%20CI/main?logo=GitHub&logoColor=white "build"
[shield_pypi-version]: https://img.shields.io/pypi/v/cyclonedx-python-lib?logo=Python&logoColor=white&label=PyPI "PyPI"
[shield_pypi-version]: https://img.shields.io/pypi/v/cyclonedx-python-lib?logo=pypi&logoColor=white&label=PyPI "PyPI"
[shield_conda-forge-version]: https://img.shields.io/conda/vn/conda-forge/cyclonedx-python-lib?logo=anaconda&logoColor=white&label=conda-forge "conda-forge"
[shield_license]: https://img.shields.io/github/license/CycloneDX/cyclonedx-python-lib "license"
[shield_website]: https://img.shields.io/badge/https://-cyclonedx.org-blue.svg "homepage"
[shield_slack]: https://img.shields.io/badge/slack-join-blue?logo=Slack&logoColor=white "slack join"
[shield_groups]: https://img.shields.io/badge/discussion-groups.io-blue.svg "groups discussion"
[shield_twitter-follow]: https://img.shields.io/badge/Twitter-follow-blue?logo=Twitter&logoColor=white "twitter follow"
[link_gh-workflow-test]: https://github.com/CycloneDX/cyclonedx-python-lib/actions/workflows/poetry.yml?query=branch%3Amain
[link_pypi]: https://pypi.org/project/cyclonedx-python-lib/
[link_conda-forge]: https://anaconda.org/conda-forge/cyclonedx-python-lib
[link_website]: https://cyclonedx.org/
[link_slack]: https://cyclonedx.org/slack/invite
[link_discussion]: https://groups.io/g/CycloneDX
Expand Down
Loading

0 comments on commit 7fb6da9

Please sign in to comment.