Support CycloneDX evidence for file locations

[Sirdorblu]

i'v created a sbo-generator for uninstalled deb pkg, for example, what i get:

      "bom-ref": "644d44b4eb7e02495e40d0b69ff8ae953007b437e9c768f1c1fff33f2877024f",
      "type": "library",
      "supplier": {
        "name": "Ubuntu Developers \u003cubuntu-devel-discuss@lists.ubuntu.com\u003e"
      },
      "name": "zeromq3",
      "version": "4.3.2-2ubuntu1.20.04.1~esm2",
      "description": "lightweight messaging kernel (shared library)\nØMQ is a library which extends the standard socket interfaces with features\ntraditionally provided by specialised messaging middleware products.\n.\nØMQ sockets provide an abstraction of asynchronous message queues, multiple\nmessaging patterns, message filtering (subscriptions), seamless access to\nmultiple transport protocols and more.\n.\nThis package contains the libzmq shared library.",
      "hashes": [
        {
          "alg": "SHA-256",
          "content": "644d44b4eb7e02495e40d0b69ff8ae953007b437e9c768f1c1fff33f2877024f"
        }
      ],
      "cpe": "cpe:2.3:a:ubuntu:zeromq3:4_3_2_2ubuntu1_20_04_1~esm2:*:*:*:*:*:*:*",
      "purl": "pkg:deb/ubuntu/zeromq3@4.3.2-2ubuntu1.20.04.1~esm2?arch=amd64",
      "externalReferences": [
        {
          "url": "https://www.zeromq.org/",
          "type": "website"
        }
      ],
      "evidence": {
        "occurrences": [
          {
            "location": "packages\libzmq5_4.3.2-2ubuntu1.20.04.1~esm2_amd64.deb"
          }
        ]
      }
    
But doesnt see a location, why?
   "vulnerability": {
    "id": "CVE-2021-20234",
    "dataSource": "https://ubuntu.com/security/CVE-2021-20234",
    "namespace": "ubuntu:distro:ubuntu:20.04",
    "severity": "Low",
    "urls": [
     "https://ubuntu.com/security/CVE-2021-20234"
    ],
    "cvss": [],
    "fix": {
     "versions": [],
     "state": "not-fixed"
    },
    "advisories": []
   },
   "relatedVulnerabilities": [
    {
     "id": "CVE-2021-20234",
     "dataSource": "https://nvd.nist.gov/vuln/detail/CVE-2021-20234",
     "namespace": "nvd:cpe",
     "severity": "Medium",
     "urls": [
      "https://bugzilla.redhat.com/show_bug.cgi?id=1921972",
      "https://github.com/zeromq/libzmq/security/advisories/GHSA-wfr2-29gj-5w87",
      "https://bugzilla.redhat.com/show_bug.cgi?id=1921972",
      "https://github.com/zeromq/libzmq/security/advisories/GHSA-wfr2-29gj-5w87"
     ],
     "description": "An uncontrolled resource consumption (memory leak) flaw was found in the ZeroMQ client in versions before 4.3.3 in src/pipe.cpp. This issue causes a client that connects to multiple malicious or compromised servers to crash. The highest threat from this vulnerability is to system availability.",
     "cvss": [
      {
       "source": "nvd@nist.gov",
       "type": "Primary",
       "version": "2.0",
       "vector": "AV:N/AC:M/Au:N/C:N/I:N/A:P",
       "metrics": {
        "baseScore": 4.3,
        "exploitabilityScore": 8.6,
        "impactScore": 2.9
       },
       "vendorMetadata": {}
      },
      {
       "source": "nvd@nist.gov",
       "type": "Primary",
       "version": "3.1",
       "vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H",
       "metrics": {
        "baseScore": 6.5,
        "exploitabilityScore": 2.8,
        "impactScore": 3.6
       },
       "vendorMetadata": {}
      }
     ]
    }
   ],
   "matchDetails": [
    {
     "type": "exact-direct-match",
     "matcher": "dpkg-matcher",
     "searchedBy": {
      "distro": {
       "type": "ubuntu",
       "version": "20.04"
      },
      "namespace": "ubuntu:distro:ubuntu:20.04",
      "package": {
       "name": "zeromq3",
       "version": "4.3.2-2ubuntu1.20.04.1~esm2"
      }
     },
     "found": {
      "versionConstraint": "none (deb)",
      "vulnerabilityID": "CVE-2021-20234"
     }
    }
   ],
   "artifact": {
    "id": "763aa19b328d5d18",
    "name": "zeromq3",
    "version": "4.3.2-2ubuntu1.20.04.1~esm2",
    "type": "deb",
    "locations": [],
    "language": "",
    "licenses": [],
    "cpes": [
     "cpe:2.3:a:ubuntu:zeromq3:4_3_2_2ubuntu1_20_04_1~esm2:*:*:*:*:*:*:*"
    ],
    "purl": "pkg:deb/ubuntu/zeromq3@4.3.2-2ubuntu1.20.04.1~esm2?arch=amd64",
    "upstreams": []
   }
  }

[Sirdorblu]

package main

import (
	"archive/tar"
	"compress/gzip"
	"crypto/sha256"
	"encoding/json"
	"flag"
	"fmt"
	"html"
	"io"
	"log"
	"os"
	"path/filepath"
	"strings"

	cyclonedx "github.com/CycloneDX/cyclonedx-go"
	"github.com/blakesmith/ar"
	"github.com/google/uuid"
	"github.com/xi2/xz"
)


type Occurrence struct {
	Location string `json:"location"`
}


type Evidence struct {
	Occurrences []Occurrence `json:"occurrences"`
}


type MyComponent struct {
	cyclonedx.Component
	Evidence *Evidence `json:"evidence,omitempty"`
}


type MyBOM struct {
	BomFormat    string        `json:"bomFormat"`
	SpecVersion  string        `json:"specVersion"`
	SerialNumber string        `json:"serialNumber"`
	Version      int           `json:"version"`
	Components   []MyComponent `json:"components"`
}


func processDebFile(debPath string, defaultVendor string) *MyComponent {
	file, err := os.Open(debPath)
	if err != nil {
		log.Printf("error open  %s: %v", debPath, err)
		return nil
	}
	defer file.Close()

	arReader := ar.NewReader(file)
	var controlData map[string]string

	
	for {
		header, err := arReader.Next()
		if err == io.EOF {
			break
		}
		if err != nil {
			log.Printf("error read %s: %v", debPath, err)
			return nil
		}

		if strings.HasPrefix(header.Name, "control.tar") {
			var tarReader *tar.Reader
			if strings.HasSuffix(header.Name, ".gz") {
				gzipReader, err := gzip.NewReader(arReader)
				if err != nil {
					log.Printf("error unpack zip to  %s: %v", debPath, err)
					return nil
				}
				defer gzipReader.Close()
				tarReader = tar.NewReader(gzipReader)
			} else if strings.HasSuffix(header.Name, ".xz") {
				xzReader, err := xz.NewReader(arReader, 0)
				if err != nil {
					log.Printf("error unpack tar %s: %v", debPath, err)
					return nil
				}
				tarReader = tar.NewReader(xzReader)
			} else {
				tarReader = tar.NewReader(arReader)
			}

			for {
				tarHeader, err := tarReader.Next()
				if err == io.EOF {
					break
				}
				if err != nil {
					log.Printf("error read tar from %s: %v", debPath, err)
					return nil
				}

				if tarHeader.Name == "./control" || tarHeader.Name == "control" {
					controlBytes, err := io.ReadAll(tarReader)
					if err != nil {
						log.Printf("error read control from %s: %v", debPath, err)
						return nil
					}
					controlData = parseControlData(string(controlBytes))
					break
				}
			}
			break
		}
	}

	if controlData == nil {
		log.Printf("cant find control %s", debPath)
		return nil
	}

	
	packageName := strings.TrimSpace(controlData["Package"])
	sourceName := strings.TrimSpace(controlData["Source"])
	ver := strings.TrimSpace(controlData["Version"])
	desc := strings.TrimSpace(controlData["Description"])
	homepage := strings.TrimSpace(controlData["Homepage"])
	lic := strings.TrimSpace(controlData["License"])
	arch := strings.TrimSpace(controlData["Architecture"])
	maintainer := strings.TrimSpace(controlData["Maintainer"])

	
	maintainer = html.UnescapeString(maintainer)
	desc = html.UnescapeString(desc)

	
	var name string
	if sourceName != "" {
		if idx := strings.Index(sourceName, " ("); idx != -1 {
			sourceName = sourceName[:idx]
		}
		name = sourceName
	} else {
		name = packageName
	}

	
	vendor := defaultVendor
	if vendor == "" {
		if maintainer != "" {
			vendor = extractVendorFromMaintainer(maintainer)
		} else {
			vendor = "ubuntu"
		}
	}

	
	purl := fmt.Sprintf("pkg:deb/%s/%s@%s?arch=%s", vendor, name, ver, arch)

	
	hashValue, err := calculateFileHash(debPath)
	if err != nil {
		log.Printf("error hash %s: %v", debPath, err)
		return nil
	}

	
	baseComponent := cyclonedx.Component{
		Type:        cyclonedx.ComponentTypeLibrary,
		Name:        name,
		Version:     ver,
		Description: desc,
		PackageURL:  purl,
		BOMRef:      hashValue,
	}
	if lic != "" {
		baseComponent.Licenses = &cyclonedx.Licenses{
			cyclonedx.LicenseChoice{
				License: &cyclonedx.License{
					Name: lic,
				},
			},
		}
	}
	if maintainer != "" {
		baseComponent.Supplier = &cyclonedx.OrganizationalEntity{
			Name: maintainer,
		}
	}
	if homepage != "" {
		baseComponent.ExternalReferences = &[]cyclonedx.ExternalReference{
			{
				Type: cyclonedx.ERTypeWebsite,
				URL:  homepage,
			},
		}
	}
	cpe := generateCPE(vendor, name, ver)
	if cpe != "" {
		baseComponent.CPE = cpe
	}
	baseComponent.Hashes = &[]cyclonedx.Hash{
		{
			Algorithm: cyclonedx.HashAlgoSHA256,
			Value:     hashValue,
		},
	}


	evidence := Evidence{
		Occurrences: []Occurrence{
			{Location: debPath},
		},
	}

	myComp := MyComponent{
		Component: baseComponent,
		Evidence:  &evidence,
	}

	return &myComp
}


func extractVendorFromMaintainer(maintainer string) string {
	idx := strings.Index(maintainer, "<")
	if idx != -1 {
		return strings.TrimSpace(maintainer[:idx])
	}
	return strings.TrimSpace(maintainer)
}


func generateCPE(vendor, packageName, version string) string {
	vendor = sanitizeCPEString(vendor)
	product := sanitizeCPEString(packageName)
	version = sanitizeCPEString(version)
	cpe := fmt.Sprintf("cpe:2.3:a:%s:%s:%s:*:*:*:*:*:*:*", vendor, product, version)
	return cpe
}


func sanitizeCPEString(s string) string {
	s = strings.ToLower(s)
	s = strings.ReplaceAll(s, " ", "_")
	s = strings.ReplaceAll(s, ".", "_")
	s = strings.ReplaceAll(s, "-", "_")
	s = strings.ReplaceAll(s, "/", "_")
	s = strings.ReplaceAll(s, "\\", "_")
	s = strings.ReplaceAll(s, ":", "_")
	return s
}


func parseControlData(data string) map[string]string {
	controlInfo := make(map[string]string)
	lines := strings.Split(data, "\n")
	var currentKey string
	for _, line := range lines {
		if line == "" {
			continue
		}
		if strings.HasPrefix(line, " ") || strings.HasPrefix(line, "\t") {
			controlInfo[currentKey] += "\n" + strings.TrimSpace(line)
		} else {
			if sepIndex := strings.Index(line, ":"); sepIndex != -1 {
				key := line[:sepIndex]
				value := strings.TrimSpace(line[sepIndex+1:])
				controlInfo[key] = value
				currentKey = key
			}
		}
	}
	return controlInfo
}


func calculateFileHash(filePath string) (string, error) {
	file, err := os.Open(filePath)
	if err != nil {
		return "", err
	}
	defer file.Close()
	hasher := sha256.New()
	if _, err := io.Copy(hasher, file); err != nil {
		return "", err
	}
	return fmt.Sprintf("%x", hasher.Sum(nil)), nil
}

func main() {
	var vendorArg string
	flag.StringVar(&vendorArg, "vendor", "", "vendor for comp )")
	flag.Parse()

	if flag.NArg() != 1 {
		fmt.Println("usage generator  [options] path/*deb")
		flag.PrintDefaults()
		os.Exit(1)
	}
	debDir := flag.Arg(0)

	var myComponents []MyComponent
	err := filepath.Walk(debDir, func(path string, info os.FileInfo, err error) error {
		if err != nil {
			return err
		}
		if filepath.Ext(path) == ".deb" {
			component := processDebFile(path, vendorArg)
			if component != nil {
				myComponents = append(myComponents, *component)
			}
		}
		return nil
	})
	if err != nil {
		log.Fatalf("error read folder: %v", err)
	}

	bom := MyBOM{
		BomFormat:    "CycloneDX",
		SpecVersion:  "1.6",
		SerialNumber: "urn:uuid:" + uuid.New().String(),
		Version:      1,
		Components:   myComponents,
	}

	bomBytes, err := json.MarshalIndent(bom, "", "  ")
	if err != nil {
		log.Fatalf("error BOM: %v", err)
	}

	err = os.WriteFile("sbom.json", bomBytes, 0644)
	if err != nil {
		log.Fatalf("error SBOM: %v", err)
	}

	fmt.Println("SBOM saved sbom.json")
}

[kzantow]

Hi @Sirdorblu -- I'm having a hard time understanding what the problem is; would you be able to summarize in a few sentences what you expect to happen and what's happening? I don't see Grype being used or a sample SBOM that you've generated -- only code that you may have used to generate an SBOM, which I can't really run if I wanted to. It would be much more helpful to provide the actual SBOM that you generated and indicate what Grype did or did not do that you expected it to do differently.

[kzantow]

Re-reading this, it seems like the ask is to support the CycloneDX evidence for file locations. Since this would be a Syft SBOM decoding concern, I've re-titled it and moved it.

[Sirdorblu]

Re-reading this, it seems like the ask is to support the CycloneDX evidence for file locations. Since this would be a Syft SBOM decoding concern, I've re-titled it and moved it.

Sorry it took me a while to get back to you.
I noticed two issues:
1)Syft doesn’t recognise all CycloneDX 1.6 fields(For example,filed "location". for test, i created a sbom by grype for notinstalled rpm's packages, instead of using standard syntax cyclondx, grype added a things like syft:location:0:path:...... ).
2)Grype can’t scan Debian packages that haven’t been installed yet.
I’d like to thank you for already supporting un-installed RPMs—that’s invaluable for teams that maintain their own internal repositories, because it lets us index those packages and track vulnerabilities before they ever reach production. It also makes it easy to add security quality-gates to our CI/CD pipelines.
Having the same capability for .deb files would be a huge help.

Thanks again for all your hard work on this project!!!!

[Sirdorblu]

Hi @Sirdorblu -- I'm having a hard time understanding what the problem is; would you be able to summarize in a few sentences what you expect to happen and what's happening? I don't see Grype being used or a sample SBOM that you've generated -- only code that you may have used to generate an SBOM, which I can't really run if I wanted to. It would be much more helpful to provide the actual SBOM that you generated and indicate what Grype did or did not do that you expected it to do differently.

it's example of code that unzip a deb pkg then read meta info and create standard cyclondx 1.6 sbom, and grype cant read a location because i should use syft syntax (I don't remember exactly, but something like "syft:location:0:path:......")

[kzantow]

Hey @Sirdorblu -- it looks like support to read .deb files was added recently and should be in the upcoming release: #3704

[anthonydahanne]

hello all! reviving this issue! there were actually 2 issues at hand:

• support to read of .deb content (fixed in feat: add Debian archive (.deb) file cataloger #3704 )
• "syft should file in "evidence" > "occurrences" >"location" fields

has support ever been implemented for the latter? still up for grabs?
thanks!
PS: for context, Dependency Track will support this cyclonedx field in their next major release, see DependencyTrack/hyades#1691