Skip to content
Vulncode-DB project
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
app Added basic setup instructions, simplified API usage and fixed many b… Mar 27, 2019
data More changes for the Docker compose setup. Apr 4, 2019
docker
lib Added basic setup instructions, simplified API usage and fixed many b… Mar 27, 2019
static Added basic setup instructions, simplified API usage and fixed many b… Mar 27, 2019
templates Updated readme to point ot the main website, small UI fix for the edi… Mar 29, 2019
.eslintrc.json Initial commit. Mar 5, 2019
.gitignore First steps towards a proper docker compose setup. Apr 2, 2019
CONTRIBUTING.md Initial commit. Mar 5, 2019
LICENSE
README.md Updated readme to point ot the main website, small UI fix for the edi… Mar 29, 2019
appengine_config.py Initial commit. Mar 5, 2019
cfg.py
crawl_patches.py Smaller fixes and strongly improved config management. Mar 24, 2019
deploy.sh Initial commit. Mar 5, 2019
dev_requirements.txt Fixes bugs and design flaws for deployment. Mar 24, 2019
example_app.yaml
example_vcs_proxy.yaml First steps towards a proper docker compose setup. Apr 2, 2019
format.sh
gce_vcs_proxy.py
lint.sh
main.py More changes for the Docker compose setup. Apr 4, 2019
manage.sh Fixes bugs and design flaws for deployment. Mar 24, 2019
package.json Initial commit. Mar 5, 2019
pylintrc
requirements.txt Added basic setup instructions, simplified API usage and fixed many b… Mar 27, 2019
run.sh
setup.sh More changes for the Docker compose setup. Apr 4, 2019
vcs_requirements.txt Added basic setup instructions, simplified API usage and fixed many b… Mar 27, 2019

README.md

Vulncode-DB

License

Overview

The vulnerable code database (Vulncode-DB) is a database for vulnerabilities and their corresponding source code if available. The database extends the NVD / CVE data sets with user-supplied information regarding patch links, vulnerable code offsets and descriptions. Particularly, the database intends to make real-world examples of vulnerable code universally accessible and useful.

The main instance is hosted on vulncode-db.com and more context is provided at vulncode-db.com/about.

Please note:

This application is currently in an experimental alpha version mostly for demonstration purposes. The application might be unreliable, contains many bugs and is not feature complete. Please set your expectations accordingly.

Directory structure

├── app
├── cert (SSL certificates)
├── data
│   ├── forms
│   └── models
├── lib
│   └── vcs_handler
├── static
│   ├── css
│   ├── js
│   │   └── lib
│   ├── monaco
│   │   └── themes
│   └── tutorial
├── templates
│   └── editor
├── third_party (third-party content)
└── vulnerable_code (temporary directory used for caching repositories)

Setup

These are some vague instructions to setup this project on your own. A simplified docker container setup will follow.

git clone https://github.com/google/vulncode-db

# Install Google's Cloud SDK. Follow steps on:
# See steps at: https://cloud.google.com/appengine/docs/standard/python/download and https://cloud.google.com/sdk/docs/#linux
# wget https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-239.0.0-linux-x86_64.tar.gz
# tar xfvz google-cloud-sdk-239.0.0-linux-x86_64.tar.gz
# ./google-cloud-sdk/install.sh
# Also make sure to run:
gcloud components install app-engine-python
gcloud components install app-engine-python-extras

# Note: If you don't use sudo here make sure to add ~/.local/bin to your $PATH variable.
sudo pip install virtualenv
virtualenv venv
source venv/bin/activate

# Install mysqldb Python connector libraries.
sudo apt install default-libmysqlclient-dev

# Install third party dependencies.
mkdir third_party
pip install -t third_party -r requirements.txt

# Install developer dependencies.
pip install -r dev_requirements.txt

# Setup a mysql database.
sudo apt install default-mysql-server
sudo mysql_secure_installation

# Mariadb uses a plugin for socket authentication by default.
# We can disable it in the following way.
echo "use mysql; update user set plugin='' where User='root'; flush privileges;" | sudo mysql -u root
# We should now be able to use mysql without sudo. You can test it with.
# mysql -u root -p

# Create required databases.
echo "CREATE DATABASE main; CREATE DATABASE cve; CREATE DATABASE cwe;" | sudo mysql -u root -p

# Setup and configure the application in app.yaml.
mv example_app.yaml app.yaml
# Edit app.yaml and 
# 1) Add the new mysql root password.
# 2) Fill out the remaining information.

# Init alembic migrations.
./manage.sh db init

# Test the application.
./run.sh

Fetching CWE identifiers.

wget https://cwe.mitre.org/data/xml/views/2000.xml.zip
unzip 2000.xml.zip
grep -oP '(?<= ID=").*?" Name=".*?(?=")' 2000.xml | sed 's/" Name="/| /' | awk '{print "CWE-" $0}' > cwe.csv
# Import into database with.
echo "load data local infile 'cwe.csv' into table cwe.cwe_data fields terminated by '|' lines terminated by '\n' (cwe_id, cwe_name)" | sudo mysql -uroot -p 

Importing NVD/CVE data.

Currently, this is a manual process using Docker and go-cve-dictionary.

git clone https://github.com/kotakanbe/go-cve-dictionary
cd go-cve-dictionary
# Reset to specific version in July 2018.
git reset --hard c2bcc418e037d6bc2d6b47c2d782900126b4f884
# Attention: Replace static.nvd.nist.gov with nvd.nist.gov as the application doesn't auto follow updated download urls.
sed -i 's/static.nvd.nist.gov/nvd.nist.gov/' nvd/nvd.go
# Build image
docker build -t go-cve-dictionary -f Dockerfile .
# Run image 
sudo docker run --network host -it --entrypoint /bin/sh go-cve-dictionary
# Import data into host mysql via:
go-cve-dictionary fetchnvd -dbtype mysql -dbpath "[username]:[password]@tcp(127.0.0.1:3306)/cve" -years 2019

Third-party Data

This project builds upon data provided by the CVE and NVD data sets.

Common Vulnerabilities and Exposures (CVE®)

The CVE® is maintained by the Mitre Corporation. Please see the Mitre CVE®'s Terms of use:

CVE Usage: MITRE hereby grants you a perpetual, worldwide, non-exclusive,
no-charge, royalty-free, irrevocable copyright license to reproduce, prepare
derivative works of, publicly display, publicly perform, sublicense, and
distribute Common Vulnerabilities and Exposures (CVE®). Any copy you make for
such purposes is authorized provided that you reproduce MITRE's copyright
designation and this license in any such copy.

National Vulnerabilitiy Database (NVD)

The National Vulnerability Database is maintained by the U.S. government. Please see the NVD's FAQ:

All NVD data is freely available from our XML Data Feeds. There are no fees,
licensing restrictions, or even a requirement to register. All NIST
publications are available in the public domain according to Title 17 of the
United States Code. Acknowledgment of the NVD  when using our information is
appreciated. In addition, please email nvd@nist.gov to let us know how the
information is being used.

Disclaimer

This is not an officially supported Google product.

You can’t perform that action at this time.