NVD Enrichment Provider

[ericrippetoe]

Target Audience

Consumers of Vulnerabilities:

• SaaS Governance
• ACT
• CSCRM

What’s the Value

• Ability to directly pull NVD vulnerability data dependencies that have vulnerabilities registered in the NVD.
• Multiple existing enrichment providers have vulnerability APIs, but wrap the NVD data in their own proprietary schema. Having a native NVD enrichment providers will allow us to:
  • Standardize on the NVD schema.
  • Treat proprietary enrichment sources and as add-on data.
  • Ship a core feature set that that does not require organizations wishing to use Harbor, to have a commercial vulnerability data subscription. (True Open Source/Open Data).

Details

Most of the enrichment providers use NVD for vulnerabilities, so Harbor will have its own integration directly with the NVD and use in addition to, or in place of, existing vulnerability enrichments, where it makes sense.

Definition of Done

• The ability to correlate package URLs to CPE IDs, when possible.
• Daily scheduled task that refreshes/syncs a copy of the NVD vulnerability data set.
• Daily scheduled enrichment task that uses NVD as a data source for vulnerability data.
• At the data store level, the Vulnerability collection should be standardized on the NVD schema.
  • This will require updates to the daily analytics export.
  • Commercial vulnerability data should be segmented by vendor and treated as ancillary data.
  • We will need to analyze and design how and/or if we want to include commercial vulnerability data in the daily exports.

Tasks

• Develop construction provider to download NVD Vulnerabilities and create usable data set in DocumentDB

  • Create construction provider framework in CLI
  • Create NVD service in Core
    • Create functions to get NVD CVE Metadata and check the NVD CVE collection to see if the data is up to date
    • Create function to download raw NVD CVE data in archive (gz or zip) to local file system
    • Create functions to unzip the archives, parse the data and populate the CVE collection
  • Create NVD data construction Task provider to use the functionality in the NVD service to populate the CVE collection

• Develop enrichment provider to evaluate dependencies against the NVD dataset

  • Create enrichment provider framework in CLI
  • Add functionality to support dependency evaluation into NVD Core Service
    • Create functions in NVD Service to lookup CVEs by CPE and populate Vulnerability structs.
      • Add code to Analytic Service to find dependant Packages that have CPEs
      • Create functions that use the found CPEs to extract CVE data and massage into Vulnerability structs
      • Create functions to add the Vulnerability data to the Vulnerability collection in DOCDB
    • Create functions to lookup CVEs by other parameters if no CVE exists. This functionality will be best effort
      • Add code to Analytic Service to find "unknown" CPEs.
      • Create functions that extract parameters from the dependent package to attempt to find matches in the NVD data set
      • Use existing functionality to add Vulnerabilities If we can identify applicable Vulnerabilities that match the package parameters.
    • Create NVD enrichment Task provider to use functionality in the NVD Service to evaluate dependent Packages for vulnerabilities using the NVD data set

[ericrippetoe]

Related to #121

[sbolel]

⚠️ Repository Decommission Notice: This repository is scheduled to be archived as it has been decommissioned and will no longer be actively maintained. As part of the archival process, we are closing all open issues and pull requests.