Feat: dependency graph in SBOM result

[r614]

Will there be added support for the dependency graph extension, or will that be a separate library?

[stevespringett]

Support for dependency graphs should be included in every official CycloneDX implementation, including this one. Currently, only the Maven plugin supports it.

I'm relying heavily on the community for these types of enhancements. PR's are highly encouraged.

[sbs2001]

How does this look

<?xml version="1.0" encoding="UTF-8"?>
<bom xmlns="http://cyclonedx.org/schema/bom/1.0" version="1">
    <components>
        <component type="library">
            <publisher>the purl authors</publisher>
            <name>packageurl_python</name>
            <version>0.9.3</version>
            <description>A "purl" aka. Package URL parser and builder</description>
            <hashes>
                <hash alg="MD5">d051230d016990f856c14ceb6ec7836c</hash>
                <hash alg="SHA-256">0682b2eddab16151da5bd4ef38081e9b27f8eb33cd29baf41f4996d4e88e6e70</hash>
            </hashes>
            <licenses>
                <license>
                    <name>MIT</name>
                </license>
            </licenses>
            <purl>pkg:pypi/packageurl-python@0.9.3</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Kenneth Reitz</publisher>
            <name>requests</name>
            <version>2.25.0</version>
            <description>Python HTTP for Humans.</description>
            <hashes>
                <hash alg="MD5">2966d68a5a4e6832d967763d41f48d04</hash>
                <hash alg="SHA-256">e786fa28d8c9154e6a4de5d46a1d921b8749f8b74e28bde23768e5e16eece998</hash>
            </hashes>
            <licenses>
                <license>
                    <name>Apache 2.0</name>
                </license>
            </licenses>
            <purl>pkg:pypi/requests@2.25.0</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Davide Brunato</publisher>
            <name>xmlschema</name>
            <version>1.2.5</version>
            <description>An XML Schema validator and decoder</description>
            <hashes>
                <hash alg="MD5">7a5623bbe80f43d96b1a77a8cdd95619</hash>
                <hash alg="SHA-256">7c528e0ec3eac97276491e7657d843f6090cbc2ea9216eb4398553623859a23f</hash>
            </hashes>
            <licenses>
                <license>
                    <name>MIT</name>
                </license>
            </licenses>
            <purl>pkg:pypi/xmlschema@1.2.5</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Python Packaging Authority</publisher>
            <name>setuptools</name>
            <version>50.3.2</version>
            <description>Easily download, build, install, upgrade, and uninstall Python packages</description>
            <hashes>
                <hash alg="MD5">079395a567856392c1445a76a2833370</hash>
                <hash alg="SHA-256">2c242a0856fbad7efbe560df4a7add9324f340cf48df43651e9604924466794a</hash>
            </hashes>
            <purl>pkg:pypi/setuptools@50.3.2</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>David Fischer</publisher>
            <name>requirements_parser</name>
            <version>0.2.0</version>
            <description>Parses Pip requirement files</description>
            <hashes>
                <hash alg="MD5">611b0cab139e9a35363ec4ffa1fe6c8c</hash>
                <hash alg="SHA-256">76650b4a9d98fc65edf008a7920c076bb2a76c08eaae230ce4cfc6f51ea6a773</hash>
            </hashes>
            <licenses>
                <license>
                    <name>BSD</name>
                </license>
            </licenses>
            <purl>pkg:pypi/requirements-parser@0.2.0</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Donald Stufft and individual contributors</publisher>
            <name>packaging</name>
            <version>20.7</version>
            <description>Core utilities for Python packages</description>
            <hashes>
                <hash alg="MD5">da81732f29c8f3d3bd3ff16f85c42b7c</hash>
                <hash alg="SHA-256">eb41423378682dadb7166144a4926e443093863024de508ca5c9737d6bc08376</hash>
            </hashes>
            <purl>pkg:pypi/packaging@20.7</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Daniel Blanchard</publisher>
            <name>chardet</name>
            <version>3.0.4</version>
            <description>Universal encoding detector for Python 2 and 3</description>
            <hashes>
                <hash alg="MD5">0004b00caff7bb543a1d0d0bd0185a03</hash>
                <hash alg="SHA-256">fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691</hash>
            </hashes>
            <licenses>
                <license>
                    <name>LGPL</name>
                </license>
            </licenses>
            <purl>pkg:pypi/chardet@3.0.4</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Holger Krekel, Bruno Oliveira, Ronny Pfannschmidt, Floris Bruynooghe, Brianna Laugher, Florian Bruhin and others</publisher>
            <name>pytest</name>
            <version>6.1.2</version>
            <description>pytest: simple powerful testing with Python</description>
            <hashes>
                <hash alg="MD5">4b715c5f2f17acc462c992839e1811af</hash>
                <hash alg="SHA-256">4288fed0d9153d9646bfcdf0c0428197dba1ecb27a33bb6e031d002fa88653fe</hash>
            </hashes>
            <licenses>
                <license>
                    <name>MIT</name>
                </license>
            </licenses>
            <purl>pkg:pypi/pytest@6.1.2</purl>
            <modified>false</modified>
        </component>
        <component type="library">
            <publisher>Julian Berman</publisher>
            <name>jsonschema</name>
            <version>3.2.0</version>
            <description>An implementation of JSON Schema validation for Python</description>
            <hashes>
                <hash alg="MD5">7617cd8e4a79ba49cfd602eb921b08d8</hash>
                <hash alg="SHA-256">4e5b3cf8216f577bee9ce139cbe72eca3ea4f292ec60928ff24758ce626cd163</hash>
            </hashes>
            <purl>pkg:pypi/jsonschema@3.2.0</purl>
            <modified>false</modified>
        </component>
    </components>
    <dg:dependencies>
        <dg:dependency ref="pkg:pypi/packageurl-python@0.9.3" />
        <dg:dependency ref="pkg:pypi/requests@2.25.0">
            <dg:dependency ref="pkg:pypi/idna@2.10" />
            <dg:dependency ref="pkg:pypi/urllib3@1.26.3" />
            <dg:dependency ref="pkg:pypi/certifi@2020.12.5" />
            <dg:dependency ref="pkg:pypi/chardet@3.0.4" />
        </dg:dependency>
        <dg:dependency ref="pkg:pypi/xmlschema@1.2.5">
            <dg:dependency ref="pkg:pypi/elementpath@2.2.0" />
        </dg:dependency>
        <dg:dependency ref="pkg:pypi/setuptools@50.3.2" />
        <dg:dependency ref="pkg:pypi/requirements-parser@0.2.0" />
        <dg:dependency ref="pkg:pypi/packaging@20.7">
            <dg:dependency ref="pkg:pypi/pyparsing@2.4.7" />
        </dg:dependency>
        <dg:dependency ref="pkg:pypi/chardet@3.0.4" />
        <dg:dependency ref="pkg:pypi/pytest@6.1.2" />
        <dg:dependency ref="pkg:pypi/jsonschema@3.2.0">
            <dg:dependency ref="pkg:pypi/six@1.15.0" />
            <dg:dependency ref="pkg:pypi/setuptools@47.1.0" />
            <dg:dependency ref="pkg:pypi/pyrsistent@0.17.3" />
            <dg:dependency ref="pkg:pypi/attrs@20.3.0" />
        </dg:dependency>
    </dg:dependencies>
</bom>

I have rough implementation of this, will open a PR soon.

This is in reference to https://cyclonedx.org/ext/dependency-graph/ . That page says "It has been incorporated (with minor changes) into CycloneDX v1.2 and higher. #"

Could someone point me to the "minor changes" ?

[stevespringett]

The dependency graph extension should not be used. Rather, the built-in dependency graph elements should be used instead. This is going to require #9 to be implemented.

The 'minor change' is actually with regard to #9 - the metadata section. The dependency graph example provided in #40 (comment) is not capable of describing direct vs transitive relationships. The 'minor change' in v1.2 is that the dependency graph can now make that distinction. Refer to https://cyclonedx.org/use-cases/#dependency-graph

[sbs2001]

@stevespringett thanks for the links . Correct me if I am wrong: to translate the v1.2 spec to python world, the setup.py or something top level would need to be parsed. That would be enough to make the metadata node. All the things in the requirements.txt would be treated as it's direct dependencies. And their subsequent (2-degree dependency) would be the transitive deps.

Also could you elaborate

The dependency graph extension should not be used. Rather, the built-in dependency graph elements should be used instead

does that simply mean <dg:dependency> gets changed to <dependency> ?

[stevespringett]

does that simply mean dg:dependency gets changed to ?

Correct

[mgrajesh1]

@sbs2001 , were you able to raise PR for above? Thanks

[madpah]

Leaving this open as some work may be required in this application for outputting dependency graphs once cyclonedx-python-lib successfully supports bom.dependencies - see CycloneDX/cyclonedx-python-lib#7.

[KramNamez]

I'm struggling to understand the current state of this feature. It seems like the library supports this now, but there's still some work left on this side of things, correct?

If so, would it be a good idea to work on #303 and go from there, or does it need a new attempt? I'd love for this feature to finally make it into the tool and am willing to help out.

[jkowalleck]

re: #40 (comment)
looks the same to me.
CycloneDX lib got support, but there is nothing done in the actual detection of dependencies.

[KramNamez]

I have had reason and time to look at this again, and now I understand how... thorny this can be.

I think I can port the work from #303 onto a newer version of the source, so that the poetry and environment parsers will support it, and I have a PoC for how to do it from a requirements.txt that has been generated by pip-compile.

Should I just open a branch with all of these changes, as a proposal? Or should I leave the pip-compile-based one out, since that is a specific variant of the regular requirements.txt?

[jkowalleck]

related: #487 (comment)

this shows an idea how to create a dep tree from requirements.txt

@madpah FYI

[jkowalleck]

will be part of upcoming v4
at least for some sources

[jkowalleck]

fixed by #605

[jkowalleck]

This feature will be part of the next/upcoming major release.
Changelog: see #605
Install via: pip install cyclonedx-bom==4.0.0rc1