Feat: omit development dependencies from SBOM results

[jkowalleck]

Some tools distinguish between regular runtime dependencies and other dependencies that are only needed during development phases (like building, linting, testing, etc.).

There should be an option to omit those non-runtime-dependencies from the SBOM.

This feature is seen as essential for this very tool, since this very tool analyses the source information in the first place, so it already should know the group/category of a dependency.

Scope:

• poetry - knows a "dev" category
• pipenv - knows a "develop" group

Out of scope:

• environment - no possible way to distinguish purpose of installation
• requirements - the file is already a whitelist. It's in it what is in it.
• conda - does not support dev dependencies. Or does it?

---

Solution:

• CLI gets a new option --omit which accepts (choices) currently only one value: dev.
  As more such options might come later, this option can be used multiple times (use parser.add_argument(..., action='append'))
  State that the "omit" might not be fully supported by all information sources.
• Implementation
  • CycloneDxCmd
    • Is responsible for parsing the option for omit. Is also responsible for translating this option to the correct values of the respective *Parser classes.
    • creates instances of PoetryParser with the according value for omit_category. If the option for omit contained dev then, dev is added to the omit_group.
    • creates instances of PipEnvParser with the according value for omit_group. If the option for omit contained dev then, develop is added to the omit_group.
  • PoetryParser
    • has a new omit_category: Set[str] parameter.
    • filter packages based on omit_category.
    • See also [POETRY] option to omit dev-requirements #374
  • PipEnvParser
    • has a new omit_group: Set[str] parameter.
    • filter packages based on omit_group.
• Docs (README.md & ./docs/**)
  • new CLI switch was added to "usage"

Requirements to the tests:

• the existing test-fixtures were extended by development requirements.
• existing tests were extended to expect the development requirements.
• new tests were written, that use the new omit filters, and the results tests, that the unexpected results are not included.

---

Supersedes #374
Supersedes #529

[tngraf]

Very good description on what needs to be done.
But there is one unanswered question: the current implementation of PoetryParser includes by default the dev dependencies. The current implementation of PipEnvParser does not include the dev dependencies.
So if we add omit_group to PipEnvParser and no omit value is given, do we then want to include the dev dependencies? This would break the current behavior.

Proposal:

• for version 3.x we do never include dev dependencies, to be fully compatible
• for version 4.x - if you agree - we include dev dependencies by default and allow to ignore them via --omit dev.

[jkowalleck]

alternative:
how does this sound?

1. add the ability to the parses to detect the groups/category and add them to the result elements as a property, so that the result set could be filtered on the property of the result items.
   The property could be property ala cdx:poetry:component:category = whatever-it-is. if applied, register to https://github.com/CycloneDX/cyclonedx-property-taxonomy/blob/main/cdx/poetry.md
   The property could be property ala cdx:poetry:pipenv:group= whatever-it-is. if ap[plied, register a new taxonomy https://github.com/CycloneDX/cyclonedx-property-taxonomy
2. add an option to the core program to filter the parser's results, based on "omit" settings - filter via predicate logic.

Breaking changes? Then we just release a new major version. Embrace change, drive forward.

[JCHacking]

Note that poetry now includes dependency groups, in fact the development dependencies option is deprecated in favor of using dependency groups
https://python-poetry.org/docs/master/managing-dependencies/#dependency-groups

[jkowalleck]

noted, and already properly implmented:

cyclonedx-python/cyclonedx_py/parser/poetry.py

Lines 36 to 41 in e277819

	class PoetryGroupWellknown(Enum):
	"""Wellknown Poetry groups.
	See https://python-poetry.org/docs/managing-dependencies/#dependency-groups
	"""
	Main = "main"
	Dev = "dev"

cyclonedx-python/cyclonedx_py/parser/poetry.py

Lines 76 to 79 in e277819

	if package_category:
	component.properties.add(Property(
	name=PoetryProps.PackageGroup.value,
	value=package_category))

[jkowalleck]

fixed by #605

[jkowalleck]

This feature will be part of the next/upcoming major release.
Changelog: see #605
Install via: pip install cyclonedx-bom==4.0.0rc1