Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add cyclonedx schema version selection #2123

Merged
merged 9 commits into from
Sep 13, 2023
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module github.com/anchore/syft
go 1.21.0

require (
github.com/CycloneDX/cyclonedx-go v0.7.1
github.com/CycloneDX/cyclonedx-go v0.7.2
github.com/Masterminds/semver v1.5.0
github.com/Masterminds/sprig/v3 v3.2.3
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
Expand Down
6 changes: 4 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03
github.com/BurntSushi/toml v0.4.1/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ=
github.com/BurntSushi/toml v1.2.1/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/CycloneDX/cyclonedx-go v0.7.1 h1:5w1SxjGm9MTMNTuRbEPyw21ObdbaagTWF/KfF0qHTRE=
github.com/CycloneDX/cyclonedx-go v0.7.1/go.mod h1:N/nrdWQI2SIjaACyyDs/u7+ddCkyl/zkNs8xFsHF2Ps=
github.com/CycloneDX/cyclonedx-go v0.7.2 h1:kKQ0t1dPOlugSIYVOMiMtFqeXI2wp/f5DBIdfux8gnQ=
github.com/CycloneDX/cyclonedx-go v0.7.2/go.mod h1:K2bA+324+Og0X84fA8HhN2X066K7Bxz4rpMQ4ZhjtSk=
github.com/DataDog/datadog-go v3.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ=
github.com/DataDog/zstd v1.4.5 h1:EndNeuB0l9syBZhut0wns3gV1hL8zX8LIu6ZiVHWLIQ=
github.com/DataDog/zstd v1.4.5/go.mod h1:1jcaCB/ufaK+sKp1NBhlGmpz41jOoPQ35bpF36t7BBo=
Expand Down Expand Up @@ -683,6 +683,8 @@ github.com/sylabs/sif/v2 v2.11.5 h1:7ssPH3epSonsTrzbS1YxeJ9KuqAN7ISlSM61a7j/mQM=
github.com/sylabs/sif/v2 v2.11.5/go.mod h1:GBoZs9LU3e4yJH1dcZ3Akf/jsqYgy5SeguJQC+zd75Y=
github.com/sylabs/squashfs v0.6.1 h1:4hgvHnD9JGlYWwT0bPYNt9zaz23mAV3Js+VEgQoRGYQ=
github.com/sylabs/squashfs v0.6.1/go.mod h1:ZwpbPCj0ocIvMy2br6KZmix6Gzh6fsGQcCnydMF+Kx8=
github.com/terminalstatic/go-xsd-validate v0.1.5 h1:RqpJnf6HGE2CB/lZB1A8BYguk8uRtcvYAPLCF15qguo=
github.com/terminalstatic/go-xsd-validate v0.1.5/go.mod h1:18lsvYFofBflqCrvo1umpABZ99+GneNTw2kEEc8UPJw=
github.com/therootcompany/xz v1.0.1 h1:CmOtsn1CbtmyYiusbfmhmkpAAETj0wBIH6kCYaX+xzw=
github.com/therootcompany/xz v1.0.1/go.mod h1:3K3UH1yCKgBneZYhuQUvJ9HPD19UEXEI0BWbMn8qNMY=
github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
Expand Down
2 changes: 1 addition & 1 deletion schema/cyclonedx/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# CycloneDX Schemas

`syft` generates a CycloneDX BOm output. We want to be able to validate the CycloneDX schemas
`syft` generates a CycloneDX Bom output. We want to be able to validate the CycloneDX schemas
(and dependent schemas) against generated syft output. The best way to do this is with `xmllint`,
however, this tool does not know how to deal with references from HTTP, only the local filesystem.
For this reason we've included a copy of all schemas needed to validate `syft` output, modified
Expand Down
35 changes: 32 additions & 3 deletions syft/formats/cyclonedxjson/encoder.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,40 @@ import (
"github.com/anchore/syft/syft/sbom"
)

func encoder(output io.Writer, s sbom.SBOM) error {
func encoderV1_0(output io.Writer, s sbom.SBOM) error {
enc, bom := buildEncoder(output, s)
return enc.EncodeVersion(bom, cyclonedx.SpecVersion1_0)
}

func encoderV1_1(output io.Writer, s sbom.SBOM) error {
enc, bom := buildEncoder(output, s)
return enc.EncodeVersion(bom, cyclonedx.SpecVersion1_1)
}

func encoderV1_2(output io.Writer, s sbom.SBOM) error {
enc, bom := buildEncoder(output, s)
return enc.EncodeVersion(bom, cyclonedx.SpecVersion1_2)
}

func encoderV1_3(output io.Writer, s sbom.SBOM) error {
enc, bom := buildEncoder(output, s)
return enc.EncodeVersion(bom, cyclonedx.SpecVersion1_3)
}

func encoderV1_4(output io.Writer, s sbom.SBOM) error {
enc, bom := buildEncoder(output, s)
return enc.EncodeVersion(bom, cyclonedx.SpecVersion1_4)
}

func encoderV1_5(output io.Writer, s sbom.SBOM) error {
enc, bom := buildEncoder(output, s)
return enc.EncodeVersion(bom, cyclonedx.SpecVersion1_5)
}

func buildEncoder(output io.Writer, s sbom.SBOM) (cyclonedx.BOMEncoder, *cyclonedx.BOM) {
bom := cyclonedxhelpers.ToFormatModel(s)
enc := cyclonedx.NewBOMEncoder(output, cyclonedx.BOMFileFormatJSON)
enc.SetPretty(true)
enc.SetEscapeHTML(false)
err := enc.Encode(bom)
return err
return enc, bom
}
58 changes: 55 additions & 3 deletions syft/formats/cyclonedxjson/format.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,62 @@ import (

const ID sbom.FormatID = "cyclonedx-json"

func Format() sbom.Format {
var Format = Format1_4

func Format1_0() sbom.Format {
Copy link
Contributor

@kzantow kzantow Sep 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be nice to programmatically build these formats, something like this could live in the common/cyclonedxhelpers package:

func Formats(format cyclonedx.BOMFileFormat, ids ...sbom.FormatID) (out []sbom.Format) {
  versions := []cyclonedx.SpecVersion{
    cyclonedx.SpecVersion1_0,
    cyclonedx.SpecVersion1_1,
    cyclonedx.SpecVersion1_2,
    cyclonedx.SpecVersion1_3,
    cyclonedx.SpecVersion1_4,
    cyclonedx.SpecVersion1_5,
  }
  for i := range versions {
    version := versions[i]
    encoder := func(output io.Writer, s sbom.SBOM) error {
      return encodeCycloneDX(output, s, format, version)
    }
    if version == cyclonedx.SpecVersion1_4 {
      // I had another comment about maybe adding a `Default` field to the format struct, which seems to have been lost.
      // assuming that this was something we wanted to do, it could look like this without a huge refactor: 
      out = append(out, sbom.NewFormatDefault(
		version.String(),
		encoder,
		cyclonedxhelpers.GetDecoder(format),
		cyclonedxhelpers.GetValidator(format),
		ids...,
	))
    } else {
      out = append(out, sbom.NewFormat(
		version.String(),
		encoder,
		cyclonedxhelpers.GetDecoder(format),
		cyclonedxhelpers.GetValidator(format),
		ids...,
	))
    }
  }
}

This way there only needs to be a single encodeCycloneDX() function and adding new versions is just updating the list, which would update both XML and JSON variants. E.g. the XML variant would have:

func Formats() []sbom.Format {
  return cyclonedxhelpers.Formats(cyclonedx.BOMFileFormatXML, ID, "cyclonedx", "cyclone")
}

Copy link
Contributor Author

@spiffcs spiffcs Sep 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good suggestion, but I think we wanted to save this kind of work for when we go back and take a look at Formats before syft v1.0. The purpose of this PR was to just keep this as close to the same pattern as we have with SPDX. Getting creative here and trying to build these programmatically (different from the current SPDX pattern) would eventually get swallowed by that new work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed to the both of you -- it's a good suggestion, but it would be better to be consistent with the current pattern first, then refactor to "the next" pattern.

return sbom.NewFormat(
cyclonedx.SpecVersion1_0.String(),
encoderV1_0,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatJSON),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatJSON),
ID,
)
}

func Format1_1() sbom.Format {
return sbom.NewFormat(
cyclonedx.SpecVersion1_1.String(),
encoderV1_1,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatJSON),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatJSON),
ID,
)
}

func Format1_2() sbom.Format {
return sbom.NewFormat(
cyclonedx.SpecVersion1_2.String(),
encoderV1_2,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatJSON),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatJSON),
ID,
)
}

func Format1_3() sbom.Format {
return sbom.NewFormat(
cyclonedx.SpecVersion1_3.String(),
encoderV1_3,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatJSON),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatJSON),
ID,
)
}

func Format1_4() sbom.Format {
return sbom.NewFormat(
cyclonedx.SpecVersion1_4.String(),
encoderV1_4,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatJSON),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatJSON),
ID,
)
}

func Format1_5() sbom.Format {
return sbom.NewFormat(
sbom.AnyVersion,
encoder,
cyclonedx.SpecVersion1_5.String(),
encoderV1_5,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatJSON),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatJSON),
ID,
Expand Down
34 changes: 34 additions & 0 deletions syft/formats/cyclonedxjson/format_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
package cyclonedxjson

import (
"testing"

"github.com/CycloneDX/cyclonedx-go"
)

func TestFormatVersions(t *testing.T) {
tests := []struct {
name string
expectedVersion string
}{
{

"cyclonedx-json should default to v1.4",
cyclonedx.SpecVersion1_4.String(),
},
}

for _, c := range tests {
c := c
t.Run(c.name, func(t *testing.T) {
sbomFormat := Format()
if sbomFormat.ID() != ID {
t.Errorf("expected ID %q, got %q", ID, sbomFormat.ID())
}

if sbomFormat.Version() != c.expectedVersion {
t.Errorf("expected version %q, got %q", c.expectedVersion, sbomFormat.Version())
}
})
}
}
37 changes: 33 additions & 4 deletions syft/formats/cyclonedxxml/encoder.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,40 @@ import (
"github.com/anchore/syft/syft/sbom"
)

func encoder(output io.Writer, s sbom.SBOM) error {
func encoderV1_0(output io.Writer, s sbom.SBOM) error {
enc, bom := buildEncoder(output, s)
return enc.EncodeVersion(bom, cyclonedx.SpecVersion1_0)
}

func encoderV1_1(output io.Writer, s sbom.SBOM) error {
enc, bom := buildEncoder(output, s)
return enc.EncodeVersion(bom, cyclonedx.SpecVersion1_1)
}

func encoderV1_2(output io.Writer, s sbom.SBOM) error {
enc, bom := buildEncoder(output, s)
return enc.EncodeVersion(bom, cyclonedx.SpecVersion1_2)
}

func encoderV1_3(output io.Writer, s sbom.SBOM) error {
enc, bom := buildEncoder(output, s)
return enc.EncodeVersion(bom, cyclonedx.SpecVersion1_3)
}

func encoderV1_4(output io.Writer, s sbom.SBOM) error {
enc, bom := buildEncoder(output, s)
return enc.EncodeVersion(bom, cyclonedx.SpecVersion1_4)
}

func encoderV1_5(output io.Writer, s sbom.SBOM) error {
enc, bom := buildEncoder(output, s)
return enc.EncodeVersion(bom, cyclonedx.SpecVersion1_5)
}

func buildEncoder(output io.Writer, s sbom.SBOM) (cyclonedx.BOMEncoder, *cyclonedx.BOM) {
bom := cyclonedxhelpers.ToFormatModel(s)
enc := cyclonedx.NewBOMEncoder(output, cyclonedx.BOMFileFormatXML)
enc.SetPretty(true)

err := enc.Encode(bom)
return err
enc.SetEscapeHTML(false)
return enc, bom
}
58 changes: 55 additions & 3 deletions syft/formats/cyclonedxxml/format.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,62 @@ import (

const ID sbom.FormatID = "cyclonedx-xml"

func Format() sbom.Format {
var Format = Format1_4

func Format1_0() sbom.Format {
return sbom.NewFormat(
cyclonedx.SpecVersion1_0.String(),
encoderV1_0,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatXML),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatXML),
ID, "cyclonedx", "cyclone",
)
}

func Format1_1() sbom.Format {
return sbom.NewFormat(
cyclonedx.SpecVersion1_1.String(),
encoderV1_1,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatXML),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatXML),
ID, "cyclonedx", "cyclone",
)
}

func Format1_2() sbom.Format {
return sbom.NewFormat(
cyclonedx.SpecVersion1_2.String(),
encoderV1_2,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatXML),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatXML),
ID, "cyclonedx", "cyclone",
)
}

func Format1_3() sbom.Format {
return sbom.NewFormat(
cyclonedx.SpecVersion1_3.String(),
encoderV1_3,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatXML),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatXML),
ID, "cyclonedx", "cyclone",
)
}

func Format1_4() sbom.Format {
return sbom.NewFormat(
cyclonedx.SpecVersion1_4.String(),
encoderV1_4,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatXML),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatXML),
ID, "cyclonedx", "cyclone",
)
}

func Format1_5() sbom.Format {
return sbom.NewFormat(
sbom.AnyVersion,
encoder,
cyclonedx.SpecVersion1_5.String(),
encoderV1_5,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatXML),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatXML),
ID, "cyclonedx", "cyclone",
Expand Down
32 changes: 26 additions & 6 deletions syft/formats/formats.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,17 +26,27 @@ import (
func Formats() []sbom.Format {
return []sbom.Format{
syftjson.Format(),
cyclonedxxml.Format(),
cyclonedxjson.Format(),
github.Format(),
table.Format(),
text.Format(),
template.Format(),
cyclonedxxml.Format1_0(),
cyclonedxxml.Format1_1(),
cyclonedxxml.Format1_2(),
cyclonedxxml.Format1_3(),
cyclonedxxml.Format1_4(),
cyclonedxxml.Format1_5(),
cyclonedxjson.Format1_0(),
cyclonedxjson.Format1_1(),
cyclonedxjson.Format1_2(),
cyclonedxjson.Format1_3(),
cyclonedxjson.Format1_4(),
cyclonedxjson.Format1_5(),
spdxtagvalue.Format2_1(),
spdxtagvalue.Format2_2(),
spdxtagvalue.Format2_3(),
spdxjson.Format2_2(),
spdxjson.Format2_3(),
table.Format(),
text.Format(),
template.Format(),
}
}

Expand All @@ -55,7 +65,7 @@ func Identify(by []byte) sbom.Format {

// ByName accepts a name@version string, such as:
//
// spdx-json@2.1 or cyclonedx@2
// spdx-json@2.1 or cyclonedx@1.5
func ByName(name string) sbom.Format {
parts := strings.SplitN(name, "@", 2)
version := sbom.AnyVersion
Expand All @@ -71,6 +81,16 @@ func ByNameAndVersion(name string, version string) sbom.Format {
for _, f := range Formats() {
for _, n := range f.IDs() {
if cleanFormatName(string(n)) == name && versionMatches(f.Version(), version) {
// if the version is not specified and the format is cyclonedx, then we want to return the most recent version up to 1.4
// If more aliases like cdx are added this will not catch those - we want to eventually provide a way for
// formats to inform this function what their default version is
// TODO: remove this check when 1.5 is stable or default formats are designed. PR below should be merged.
// https://github.com/CycloneDX/cyclonedx-go/pull/90
if version == sbom.AnyVersion && strings.Contains(string(n), "cyclonedx") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking for the string cyclonedx but we might introduce an alias like cdx. I think this should more formally ask the format for the latest supported version.

At the same time, I feel that this format object is about to be rethought anyway. I think there are strong signs to split up for encoding vs decoding use cases, which means trying to settle this detail in this PR wouldn't be worth it. I think it's worth at least adding a comment here about the limitations of this approach and a TODO to refactor later.

Copy link
Contributor

@wagoodman wagoodman Sep 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, how does ByName work for spdx for the latest version (I didn't read up on that yet)?

Copy link
Contributor Author

@spiffcs spiffcs Sep 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

 I feel that this format object is about to be rethought anyway. 
I think there are strong signs to split up for encoding vs decoding use cases, 
which means trying to settle this detail in this PR wouldn't be worth it.

^ Yea that sounds good to me

I think this should more formally ask the format for the latest supported version

I had tried just setting up a default version for each format rather than latest so we had more flexibility, but that would constitute more of a refactor for the format object that I didn't want to include in this PR.

The comment for this line should be below it I can move above and add a TODO:

// if the version is not specified and the format is cyclonedx, then we want to return the most recent version up to 1.4
// https://github.com/CycloneDX/cyclonedx-go/pull/90

For SPDX if you pass in syft -o spdx alpine:latest then no version is detected. mostRecentFormat will eventually become 2.3

Copy link
Contributor

@kzantow kzantow Sep 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would probably like to add a Default field to the format object. It could be accomplished without a huge refactor (see related comment below). At some point in the future, when we support SPDX 3, we will probably also want to continue to default to SPDX 2.3.

Copy link
Contributor Author

@spiffcs spiffcs Sep 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can follow this PR up with a stab at that. I really want to keep this PR small and focused on integrating the released cyclonedx library. Other changes on how we handle formats, their defaults or how the latest is selected can be their own change set. The reason this line had to be included was because we don't necessarily want to be on latest for cyclonedx at this moment. Rather than rewrite ByNameAndVersion into a totally new thing I thought a simple edge case with comment would be the correct change here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check should probably be strings.Contains(string(n), "cyclone") to use the same default when -o cyclone is used.

if f.Version() == "1.5" {
continue
}
}
if mostRecentFormat == nil || f.Version() > mostRecentFormat.Version() {
mostRecentFormat = f
}
Expand Down
1 change: 0 additions & 1 deletion syft/formats/formats_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,6 @@ func TestFormats_EmptyInput(t *testing.T) {
}

func TestByName(t *testing.T) {

tests := []struct {
name string
want sbom.FormatID
Expand Down
33 changes: 33 additions & 0 deletions syft/formats/syftjson/format_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
package syftjson

import (
"testing"

"github.com/anchore/syft/internal"
)

func TestFormat(t *testing.T) {
tests := []struct {
name string
version string
}{
{
name: "default version should use latest internal version",
version: "",
},
}

for _, c := range tests {
c := c
t.Run(c.name, func(t *testing.T) {
sbomFormat := Format()
if sbomFormat.ID() != ID {
t.Errorf("expected ID %q, got %q", ID, sbomFormat.ID())
}

if sbomFormat.Version() != internal.JSONSchemaVersion {
t.Errorf("expected version %q, got %q", c.version, sbomFormat.Version())
}
})
}
}