Vulnerability Database

Persistent store for vulnerabilities data obtained from Vulcan scans.

The Vulnerability Database acts as a worker which reads from a queue containing the checks status changes and data, it then processes this data, including checks report, in order to maintain a historic representation of vulnerabilities lifecycle, affected assets, executed checks, etc.

For running the component locally, clone and run at the root of the repo the following:

go install ./...
cd db && source postgres-start.sh && cd -
cd db && source flyway-migrate.sh && cd -
vulnerability-db-consumer -c _resources/config/local.toml

How to run the Vulnerability DB in development mode

You can test the Vulnerability DB Consumer locally in your machine. The commands bellow will launch the necessary components required by the application.

# Navigate to the local_deployment folder
cd local_deployment

# Start the dependencies
docker-compose up -d

# Build and run the vulnerability-db-consumer
./start.sh

You can test that everything works by sending a message to the mocked SNS topic, using the AWS cli:

AWS_ACCESS_KEY_ID=fake AWS_SECRET_ACCESS_KEY=fake aws sns publish \
    --region local \
    --endpoint-url http://localhost:4100  \
    --topic-arn arn:aws:sns:local:012345678900:VulcanLocalhostChecks \
    --message '
        {
            "status":"FINISHED",
            "id":"old-model-happy-path-01",
            "tag":"team:test",
            "target":"api.example.com",
            "checktype_name":"vulcan-http-headers",
            "report":"http://localhost:8080/old-model-happy-path-01.json"
        }' \
    --message-attributes '{"status":{"DataType":"String","StringValue":"FINISHED"}}'

To stop the dependencies, run:

docker-compose down --remove-orphans

To purge local mocked SQS queue:

AWS_ACCESS_KEY_ID=fake AWS_SECRET_ACCESS_KEY=fake aws sqs purge-queue \
    --region local \
    --endpoint-url http://localhost:4100 \
    --queue-url http://localhost:4100/012345678900/VulcanLocalhostVulnDBChecks

Docker execute

Those are the variables you have to use:

Variable	Description	Sample
MAX_EVENT_AGE	Defines the max age for which check events are processed. Older events are discarded	365
LOG_LEVEL		error
PG_HOST	Database host	localhost
PG_NAME	Database name	vulnerabilitydb
PG_USER	Database user	vulnerabilitydb
PG_PASSWORD	Database password	vulnerabilitydb
PG_PORT	Database port	5432
PG_SSLMODE	One of these (disable,allow,prefer,require,verify-ca,verify-full)	disable
PG_CA_B64	A base64 encoded CA certificate
SQS_NUMBER_OF_PROCESSORS	Number of concurrent SQS processors	Default: 10
SQS_QUEUE_ARN	Checks queueu ARN	arn:aws:sqs:xxx:123456789012:yyy
SNS_TOPIC_ARN	ARN of topic to publish new vulnerabilities	arn:aws:sns:xxx:123456789012:yyy
RESULTS_URL	External vulcan-results URL	https://results.vulcan.com
RESULTS_INTERNAL_URL	Internal vulcan-results URL	http://vulcan-results
AWS_SQS_ENDPOINT	Endpoint for SQS creation queue (optional)	http://custom-aws-endpoint
AWS_SNS_ENDPOINT	Endpoint for SNS topic (optional)	http://custom-aws-endpoint

docker build . -t vdb

# Use the default config.toml customized with env variables.
docker run --env-file ./local.env -e AWS_SECRET_ACCESS_KEY -e AWS_ACCESS_KEY_ID  --name vdb --rm vdb

# Use custom config.toml
docker run -v `pwd`/custom.toml:/app/config.toml vdb