A simple Java command-line utility to mirror the entire contents of VulnDB.
Clone or download
Latest commit e89410c Dec 28, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
src/main Updating Unirest Dec 28, 2018
.travis.yml Setting up Travis Oct 7, 2017
LICENSE Initial commit Oct 7, 2017
README.md Updated console output to include status Nov 30, 2017
pom.xml Updating Unirest Dec 28, 2018

README.md

Build Status

VulnDB Data Mirror

A simple Java command-line utility to mirror the entire contents of the VulnDB service from Risk Based Security.

The intended purpose of vulndb-data-mirror is to be able to replicate the VulnDB vulnerabiity data inside a company firewall so that local (faster) access to data can be achieved and reused by the OWASP Dependency-Check and OWASP Dependency-Track ecosystem.

In addition to mirroring functionality, VulnDB Data Mirror includes a parser that can automatically convert JSON data to model objects (defined as POJO's). This greatly eases the ramp-up time needed to consume the VulnDB data in a programmatic way.

The VulnDB service utilizes a paginated REST API that must be walked for each type of feed. Due to the large data-set the service provides, it may take an hour or more to mirror the contents. Because of the performance impact due to this design, a separate mirroring utility is favorable instead of native VulnDB mirroring support in Dependency-Check or Dependency-Track. VulnDB Data Mirror serves this purpose.

For best results, use vulndb-data-mirror with cron or another scheduler to keep the mirrored data fresh.

A subscription to VulnDB is required for use. Contact VulnDB for evaluation and subscription information. VulnDB Data Mirror or it's creator are not affiliated with VulnDB or Risk Based Security. This is a community-driven project that acknowledges the value of third-party vulnerability intelligence to enhance or supplement publicly disclosed information.

By using VulnDB Data Mirror, you accept that it will be used in a manner that conforms to the VulnDB terms of service.

Distribution

VulnDB Data Mirror is distributed two different ways.

Pre-compiled binaries WILL be available (once 1.0.0 is released) from GitHub. This distribution is intended to be extracted and executed in order to run and maintain a working VulnDB mirror. This is the recommended method for most users.

The standalone library is available in the Maven Central Repository. This distribution is useful for programmatic access to the mirroring or parsing functionality.

<dependency>
    <groupId>us.springett</groupId>
    <artifactId>vulndb-data-mirror</artifactId>
    <version>1.0.0-SNAPSHOT</version>
</dependency>

Usage

Windows

vulndb-data-mirror.bat --consumer-key mykey --consumer-secret mysecret --dir "c:\path\to\mirror"

Unix/Linux

vulndb-data-mirror.sh --consumer-key mykey --consumer-secret mysecret --dir "/path/to/mirror"

When running, the console output will resemble:

VulnDB API Status:
--------------------------------------------------------------------------------
Organization Name.............: Example Inc.
Name of User Requesting.......: Jane Doe
Email of User Requesting......: jane@example.com
Subscription Expiration Date..: 2018-12-31
API Calls Allowed per Month...: 25000
API Calls Made This Month.....: 1523
--------------------------------------------------------------------------------

Mirroring Vendors feed...
  Processing 18344 of 18344 results
Mirroring Products feed...
  Processing 136853 of 136853 results
Mirroring Vulnerabilities feed...
  Processing 142500 of 166721 results

Getting Help

Execute vulndb-data-mirror.bar or vulndb-data-mirror.sh (without options)

usage: vulndb-data-mirror
    --consumer-key <key>          The Consumer Key provided by VulnDB
    --consumer-secret <secret>    The Consumer Secret provided by VulnDB
    --dir <dir>                   The target directory to store contents
 -prod,--mirror-products          Mirror the products data feed
 -vend,--mirror-vendors           Mirror the vendors data feed
 -vuln,--mirror-vulnerabilities   Mirror the vulnerabilities data feed
 -stat,--status-only              Displays VulnDB API status only

VulnDB API License

The process of mirroring the contents of VulnDB takes several thousand requests. You may estimate the number of requests required by dividing 100 by the total number of results in each of the three feeds. After mirroring is complete, make a backup of the contents so that a full mirror does not have to take place again. VulnDB may be licensed based on the number of API calls made to the service. Check with the vendor for details.

Compiling

mvn clean package

Related Projects

Copyright & License

vulndb-data-mirror is Copyright (c) Steve Springett. All Rights Reserved.

Dependency-Track is Copyright (c) Steve Springett. All Rights Reserved.

Dependency-Check is Copyright (c) Jeremy Long. All Rights Reserved.

VulnDB is Copyright (c) Risk Based Security. All Rights Reserved.

Permission to modify and redistribute is granted under the terms of the Apache 2.0 license. See the [LICENSE] Apache 2.0 file for the full license.