Skip to content

Commit

Permalink
feat: add support for VEX (Fixes #1570) (#1583)
Browse files Browse the repository at this point in the history
  • Loading branch information
@anthonyharrison
anthonyharrison committed Mar 30, 2022
1 parent 84715ba commit 7f327c9
Show file tree
Hide file tree
Showing 9 changed files with 618 additions and 2 deletions.
12 changes: 11 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ cve-bin-tool --input-file <filename>
```

Note that the `--input-file` option can also be used to add extra triage data like remarks, comments etc. while scanning a directory so that output will reflect this triage data and you can save time of re-triaging (Usage: `cve-bin-tool -i=test.csv /path/to/scan`).
A VEX file (which may be created using the `--vex` command line option) can also be used as a triage file. A VEX file
is detected if the file suffix is '.vex'.

### Scanning an SBOM file for known vulnerabilities

Expand All @@ -91,6 +93,11 @@ Valid SBOM types are [SPDX](https://spdx.dev/specifications/),

The CVE Binary Tool provides console-based output by default. If you wish to provide another format, you can specify this and a filename on the command line using `--format`. The valid formats are CSV, JSON, console, HTML and PDF. The output filename can be specified using the `--output-file` flag.

The reported vulnerabilities can additionally be reported in the
Vulnerability Exchange (VEX) format by specifying `--vex` command line option.
The generated VEX file can then be used as an `--input-file` to support
a triage process.

## Full option list

Usage:
Expand Down Expand Up @@ -142,9 +149,12 @@ Usage:
minimum CVE severity to report (default: low)
--report Produces a report even if there are no CVE for the
respective output format
--affected-versions Lists versions of product affected by a given CVE (to facilitate upgrades)
-A [<distro_name>-<distro_version_name>], --available-fix [<distro_name>-<distro_version_name>]
Lists available fixes of the package from Linux distribution
-b [<distro_name>-<distro_version_name>], --backport-fix [<distro_name>-<distro_version_name>]
Lists backported fixes if available from Linux distribution
--affected-versions Lists versions of product affected by a given CVE (to facilitate upgrades)
--vex VEX Provide vulnerability exchange (vex) filename

Merge Report:
-a INTERMEDIATE_PATH, --append INTERMEDIATE_PATH
Expand Down
8 changes: 8 additions & 0 deletions cve_bin_tool/cli.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,13 @@ def main(argv=None):
help="Lists versions of product affected by a given CVE (to facilitate upgrades)",
)

output_group.add_argument(
"--vex",
action="store",
help="Provide vulnerability exchange (vex) filename",
default="",
)

parser.add_argument(
"-e",
"--exclude",
Expand Down Expand Up @@ -614,6 +621,7 @@ def main(argv=None):
append=args["append"],
merge_report=merged_reports,
affected_versions=args["affected_versions"],
vex_filename=args["vex"],
)

if not args["quiet"]:
Expand Down
62 changes: 62 additions & 0 deletions cve_bin_tool/input_engine.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@
import csv
import json
import os
import re
from collections import defaultdict
from logging import Logger
from typing import Any, DefaultDict, Dict, Iterable, Set, Union

from cve_bin_tool.cvedb import CVEDB
from cve_bin_tool.error_handler import (
ErrorHandler,
ErrorMode,
Expand All @@ -33,6 +35,8 @@ def __init__(
self.logger = logger or LOGGER.getChild(self.__class__.__name__)
self.error_mode = error_mode
self.parsed_data = defaultdict(dict)
# Connect to the database
self.cvedb = CVEDB(version_check=False)

def parse_input(self) -> DefaultDict[ProductInfo, TriageData]:
if not os.path.isfile(self.filename):
Expand All @@ -42,6 +46,8 @@ def parse_input(self) -> DefaultDict[ProductInfo, TriageData]:
self.input_csv()
elif self.filename.endswith(".json"):
self.input_json()
elif self.filename.endswith(".vex"):
self.input_vex()
return self.parsed_data

def input_csv(self) -> None:
Expand All @@ -62,6 +68,62 @@ def input_json(self) -> None:

self.parse_data(set(json_data[0].keys()), json_data)

def validate_product(self, product: str) -> bool:
"""
Ensure product name conforms to CPE 2.3 standard.
See https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd for naming specification
"""
cpe_regex = r"\A([A-Za-z0-9\._\-~ %])+\Z"
return re.search(cpe_regex, product) is not None

def input_vex(self) -> None:
analysis_state = {
"under_review": Remarks.Unexplored,
"in_triage": Remarks.Unexplored,
"exploitable": Remarks.Confirmed,
"not_affected": Remarks.Mitigated,
}
with open(self.filename) as json_file:
json_data = json.load(json_file)
# Only handle CycloneDX VEX file format
if json_data["bomFormat"] == "CycloneDX":
for vulnerability in json_data["vulnerabilities"]:
id = vulnerability["id"]
analysis_status = vulnerability["analysis"]["state"].lower()
state = Remarks.Unexplored
if analysis_status in analysis_state:
state = analysis_state[analysis_status]
comments = vulnerability["analysis"]["detail"]
for rating in vulnerability["ratings"]:
severity = rating["severity"]
for affect in vulnerability["affects"]:
ref = affect["ref"]
version = None
vendor = None
if "#" in ref:
# Extract product information after # delimiter
p = ref.split("#")[1]
# Last element is version, rest is product which may include -
elements = p.rsplit("-", 1)
product = elements[0]
version = elements[1]
# Now find vendor
vendor_package_pair = self.cvedb.get_vendor_product_pairs(
product
)
if vendor_package_pair != []:
vendor = vendor_package_pair[0]["vendor"]
if version is not None and self.validate_product(product):
product_info = ProductInfo(
vendor.strip(), product.strip(), version.strip()
)
self.parsed_data[product_info][id.strip() or "default"] = {
"remarks": state,
"comments": comments.strip(),
"severity": severity.strip(),
}
self.parsed_data[product_info]["paths"] = {""}

def parse_data(self, fields: Set[str], data: Iterable) -> None:
required_fields = {"vendor", "product", "version"}
missing_fields = required_fields - fields
Expand Down
90 changes: 89 additions & 1 deletion cve_bin_tool/output_engine/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from ..cvedb import CVEDB
from ..error_handler import ErrorHandler, ErrorMode
from ..log import LOGGER
from ..util import ProductInfo
from ..util import ProductInfo, Remarks
from ..version import VERSION
from . import pdfbuilder
from .console import output_console
Expand Down Expand Up @@ -305,6 +305,7 @@ def __init__(
merge_report: Union[None, List[str]] = None,
affected_versions: int = 0,
all_cve_version_info=None,
vex_filename: str = "",
):
self.logger = logger or LOGGER.getChild(self.__class__.__name__)
self.all_cve_version_info = all_cve_version_info
Expand All @@ -321,6 +322,7 @@ def __init__(
self.merge_report = merge_report
self.affected_versions = affected_versions
self.all_cve_data = all_cve_data
self.vex_filename = vex_filename

def output_cves(self, outfile, output_type="console"):
"""Output a list of CVEs
Expand Down Expand Up @@ -372,6 +374,92 @@ def output_cves(self, outfile, output_type="console"):
)
self.logger.info(f"Output stored at {self.append}")

if self.vex_filename != "":
self.generate_vex(self.all_cve_data, self.vex_filename)

def generate_vex(self, all_cve_data: Dict[ProductInfo, CVEData], filename: str):
analysis_state = {
Remarks.NewFound: "under_review",
Remarks.Unexplored: "under_review",
Remarks.Confirmed: "exploitable",
Remarks.Mitigated: "not_affected",
Remarks.Ignored: "not_affected",
}
response_state = {
Remarks.NewFound: "Outstanding",
Remarks.Unexplored: "Not defined",
Remarks.Confirmed: "Upgrade required",
Remarks.Mitigated: "Resolved",
Remarks.Ignored: "No impact",
}
# Generate VEX file
vex_output = {"bomFormat": "CycloneDX", "specVersion": "1.4", "version": 1}
# Extra info considered useful
# "creationInfo": {
# "created": datetime.now().strftime("%Y-%m-%dT%H-%M-%SZ"),
# "creators": ["Tool: cve_bin_tool", "Version:" + VERSION],
# },
# "documentDescribes": ["VEX_File"],
# "externalDocumentRefs": [{
# "sbomDocument": "<FILENAME>"
# }],
# }
vuln_entry = []
for product_info, cve_data in all_cve_data.items():
for cve in cve_data["cves"]:
# Create vulnerability entry. Contains id, scoring, analysis and affected component
vulnerability = dict()
vulnerability["id"] = cve.cve_number
vulnerability["source"] = {
"name": "NVD",
"url": "https://nvd.nist.gov/vuln/detail/" + cve.cve_number,
}
if cve.cvss_version == 3:
url = f"v3-calculator?name={cve.cve_number}&vector={cve.cvss_vector}&version=3.1"
else:
url = f"v2-calculator?name={cve.cve_number}&vector={cve.cvss_vector}&version=2.0"
ratings = [
{
"source": {
"name": "NVD",
"url": "https://nvd.nist.gov/vuln-metrics/cvss/" + url,
},
"score": str(cve.score),
"severity": cve.severity,
"method": "CVSSv" + str(cve.cvss_version),
"vector": cve.cvss_vector,
}
]
vulnerability["ratings"] = ratings
vulnerability["cwes"] = []
vulnerability["description"] = cve.description
vulnerability["recommendation"] = ""
vulnerability["advisories"] = []
vulnerability["created"] = "NOT_KNOWN"
vulnerability["published"] = "NOT_KNOWN"
vulnerability["updated"] = "NOT_KNOWN"
analysis = {
"state": analysis_state[cve.remarks],
"response": response_state[cve.remarks],
"justification": "",
"detail": cve.comments,
}
vulnerability["analysis"] = analysis
bom_urn = "NOTKNOWN"
bom_version = 1
vulnerability["affects"] = [
{
"ref": f"urn:cdx:{bom_urn}/{bom_version}#{product_info.product}-{product_info.version}",
}
]
vuln_entry.append(vulnerability)

vex_output["vulnerabilities"] = vuln_entry

# Generate file
with open(filename, "w") as outfile:
json.dump(vex_output, outfile, indent=" ")

def output_file(self, output_type="console"):

"""Generate a file for list of CVE"""
Expand Down
55 changes: 55 additions & 0 deletions doc/MANUAL.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,10 @@
- [-c CVSS, --cvss CVSS](#-c-cvss---cvss-cvss)
- [-S {low,medium,high,critical}, --severity {low,medium,high,critical}](#-s-lowmediumhighcritical---severity-lowmediumhighcritical)
- [--report](#--report)
- [-A [<distro_name>-<distro_version_name>], --available-fix [<distro_name>-<distro_version_name>]](#-A-distro_name-distro_version_name---available-fix-distro_name-distro_version_name)
- [-b [<distro_name>-<distro_version_name>], --backport-fix [<distro_name>-<distro_version_name>]](#-b-distro_name-distro_version_name---backport-fix-distro_name-distro_version_name)
- [--affected-versions](#--affected-versions)
- [--vex VEX_FILE](#--vex-vex_file)
- [Output verbosity](#output-verbosity)
- [Quiet Mode](#quiet-mode)
- [Logging modes](#logging-modes)
Expand Down Expand Up @@ -97,8 +100,12 @@ which is useful if you're trying the latest code from
minimum CVE severity to report (default: low)
--report Produces a report even if there are no CVE for the
respective output format
-A [<distro_name>-<distro_version_name>], --available-fix [<distro_name>-<distro_version_name>]
Lists available fixes of the package from Linux distribution
-b [<distro_name>-<distro_version_name>], --backport-fix [<distro_name>-<distro_version_name>]
Lists backported fixes if available from Linux distribution
--affected-versions Lists versions of product affected by a given CVE (to facilitate upgrades)
--vex VEX Provide vulnerability exchange (vex) filename

Merge Report:
-a INTERMEDIATE_PATH, --append INTERMEDIATE_PATH
Expand Down Expand Up @@ -309,6 +316,19 @@ You can provide either CSV or JSON file as input_file with vendor, product and v
4. **severity** - This field allows you to adjust severity score of specific product or CVE. This can be useful in the case where CVE affects a portion of the library that you aren't using currently but you don't want to ignore it completely. In that case, you can reduce severity for this CVE.
5. **cve_number** - This field give you fine grained control over output of specific CVE. You can change remarks, comments and severity for specific CVE instead of whole product.

You can also provide a Vulnerability Exchange (VEX) file which contains the reported vulnerabilities for components within a product. The supported format
is the [CycloneDX](https://cyclonedx.org/capabilities/vex/) VEX format which can be generated using the `--vex` option. A VEX file is identified with a file extension of .vex.
For the triage process, the **state** value in the analysis section of each CVE should have one of the following values:

```
"under_review" - this is the default state and should be used to indicate the vulnerability is to be reviewed
"in_triage" - this should be used to indicate that the vulnerability is being reviewed
"exploitable" - this should be used to indicate that the vulnerability is known to be exploitable
"not_affected" - this should be used to indicate that the vulnerability has been mitigated
```

The **detail** value in the analysis section can be used to provide comments related to the state

You can use `-i` or `--input-file` option to produce list of CVEs found in given vendor, product and version fields (Usage: `cve-bin-tool -i=test.csv`) or supplement extra triage data like remarks, comments etc. while scanning directory so that output will reflect this triage data and you can save time of re-triaging (Usage: `cve-bin-tool -i=test.csv /path/to/scan`).

Note that `--input-file`, unlike `cve-bin-tool directory` scan, will work on *any* product known in the National Vulnerability Database, not only those that have checkers written.
Expand Down Expand Up @@ -615,6 +635,30 @@ Note that this option is overridden by `--cvss` parameter if this is also specif

This option produces a report for all output formats even if there are 0 CVEs. By default CVE Binary tool doesn't produce an output when there are 0 CVEs.

### -A [<distro_name>-<distro_version_name>], --available-fix [<distro_name>-<distro_version_name>]

This option lists the available fixes of the package from Linux distribution if there are any.

The currently supported Linux distributions are:

```
debian-bullseye
debian-stretch
debian-buster
ubuntu-hirsute
ubuntu-groovy
ubuntu-focal
ubuntu-eoan
ubuntu-disco
ubuntu-cosmic
ubuntu-bionic
ubuntu-artful
ubuntu-zesty
ubuntu-yakkety
ubuntu-xenial
```


### -b [<distro_name>-<distro_version_name>], --backport-fix [<distro_name>-<distro_version_name>]

This option outputs the available backported fixes for the packages with CVEs if there are any.
Expand Down Expand Up @@ -644,6 +688,17 @@ ubuntu-yakkety
ubuntu-xenial
```

### --affected-versions

This options reports the versions of a product affected by a given CVE.

### --vex VEX_FILE

This option allows you to specify the filename for a Vulnerability Exchange (VEX)
file which contains all the reported vulnerabilities detected by the scan. This file is typically
updated (outside of the CVE Binary tool) to record the results of a triage activity
and can be used as a file with `--input-file` parameter.

### Output verbosity

As well as the modes above, there are two other output options to decrease or increase the number of messages printed:
Expand Down

0 comments on commit 7f327c9

Please sign in to comment.