Vulnerability Tracker: Automated CVE Monitoring for Software Dependencies

Abstract

Modern software projects depend on hundreds of third-party packages, each introducing potential security risk. Vulnerability databases such as the NIST National Vulnerability Database (NVD) publish thousands of CVEs annually, but the burden of cross-referencing project dependencies against these databases falls on operators who rarely have the time to do so manually. This project implements a lightweight, Bash-based vulnerability scanner that extracts package dependencies from common manifest files, queries the NVD API 2.0 for known CVEs, and outputs an RSS 2.0 feed suitable for consumption by FreshRSS or any standard feed reader. The tool integrates with GitLab CI/CD for automated, scheduled scanning across entire project groups.

---

1. Problem Statement

1.1 The Dependency Problem

A typical web application in 2024 depends on 200–800 transitive packages across its stack. Each package version is a potential attack surface:

• Go projects via go.mod — 50–150 direct + indirect dependencies
• Node.js projects via package.json — 100–500+ (including node_modules depth)
• Python projects via requirements.txt — 20–80 pinned dependencies
• Docker images — base OS, runtime, and language packages

1.2 The Monitoring Gap

Most teams discover vulnerabilities reactively — after a public disclosure forces a response. Proactive monitoring requires:

1. Knowing what is deployed (dependency inventory)
2. Knowing where each dependency is used (project mapping)
3. Knowing when a CVE is published (timely notification)
4. Knowing how severe the issue is (priority triage)

Commercial dependency scanning tools (Snyk, Dependabot, GitLab SAST) solve this but introduce cost, vendor lock-in, or require specific platform integration. For self-hosted infrastructure — GitLab CE, air-gapped environments, or small teams — a lightweight alternative is valuable.

1.3 Design Goals

Goal	Implementation
Parse common manifests	go.mod, package.json, requirements.txt, Dockerfile, .gitlab-ci.yml
Query authoritative source	NIST NVD API 2.0 (free, no vendor lock-in)
Output consumable feed	RSS 2.0 (compatible with any feed reader)
Integrate with CI/CD	GitLab CI pipeline support
Zero runtime dependencies	Pure Bash + curl + jq (standard server tools)
Self-hosted friendly	No cloud accounts, no SaaS subscriptions

---

2. Architecture

2.1 Pipeline Design

The scanner operates as a three-phase pipeline:

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  Phase 1:       │    │  Phase 2:       │    │  Phase 3:       │
│  EXTRACT        │───►│  QUERY          │───►│  OUTPUT         │
│                 │    │                 │    │                 │
│  Parse manifests│    │  NVD API 2.0    │    │  RSS 2.0 feed   │
│  Deduplicate    │    │  Rate-limited   │    │  Text summary   │
│  Normalize      │    │  CPE matching   │    │  NVD deep links │
└─────────────────┘    └─────────────────┘    └─────────────────┘
       │                      │                      │
  lib/parsers.sh         lib/nvd.sh            lib/feed.sh

2.2 Dependency Representation

All parsers emit a normalized format:

ecosystem:package:version

Examples:

go:github.com/gorilla/mux:1.8.0
npm:express:4.18.2
pypi:django:4.2.7
docker:nginx:1.25-alpine
deb:openssl:3.0.12

This unified format allows a single NVD query pipeline regardless of source ecosystem.

2.3 CPE Construction

The NVD uses Common Platform Enumeration (CPE) identifiers to match software products. The scanner constructs CPE 2.3 strings from the normalized dependency format:

ecosystem:package:version  →  cpe:2.3:a:<vendor>:<product>:<version>:*:*:*:*:*:*:*

Vendor and product extraction is ecosystem-specific:

Ecosystem	Example Input	Vendor	Product
Go	github.com/gorilla/mux	gorilla	mux
npm	@babel/core	babel	core
npm	express	express	express
PyPI	django	django	django
Docker	library/nginx	nginx	nginx

2.4 NVD API Integration

Endpoint:   https://services.nvd.nist.gov/rest/json/cves/2.0
Auth:       Optional API key (recommended)
Rate:       5 req/30s (public) | 50 req/30s (with key)
Response:   JSON with CVSS v3.1/v3.0/v2.0 scoring

The scanner uses keyword-based search (broader match) rather than exact CPE matching, because NVD CPE dictionary coverage is inconsistent — particularly for Go and npm packages that lack formal vendor registration.

---

3. Module Design

3.1 Dependency Parsers (lib/parsers.sh)

Six parsers handle the most common manifest formats:

Parser	File	Method
parse_gomod	go.mod	Line-by-line with require block detection
parse_packagejson	package.json	jq extraction of dependencies + devDependencies
parse_requirements	requirements.txt	Regex extraction of package==version
parse_docker_images	Dockerfile	FROM directive extraction
parse_docker_images	docker-compose.yml	image: field extraction
parse_gitlab_ci	.gitlab-ci.yml	Docker images + pip/npm/apt install commands

Design decisions:

• go.mod: Only direct dependencies are scanned (lines marked // indirect are skipped). Indirect dependency vulnerabilities are the responsibility of the direct dependency maintainer.
• package.json: Both dependencies and devDependencies are scanned. Dev dependencies are often deployed in CI environments and Docker build stages.
• .gitlab-ci.yml: The parser extracts both Docker images and inline package installation commands (pip install, npm install, apt install). This catches dependencies that exist only in CI pipelines and not in application manifests.

3.2 NVD Client (lib/nvd.sh)

The NVD client handles:

• CPE construction — ecosystem-aware vendor/product extraction
• Keyword search — broader matching when exact CPE fails
• Result parsing — extracts CVE ID, CVSS score/severity, description, dates
• Rate limiting — automatic delay between requests (6s public, 1s with key)
• Batch scanning — processes a list of dependencies sequentially with progress reporting

3.3 Feed Generator (lib/feed.sh)

Generates valid RSS 2.0 with:

• Severity-tagged titles — 🔴 [CRITICAL] CVE-2024-3094 — xz-utils
• NVD deep links — each item links to the official NVD detail page
• CDATA descriptions — HTML-formatted details with package, score, and advisory link
• Category tags — <category>CRITICAL</category> for feed reader filtering
• Atom self-link — for feed discovery and validation

3.4 Shared Library (lib/common.sh)

Provides:

• Color-coded logging (debug, info, warn, error, pass, vuln)
• Configuration file loading from config.env
• Dependency checks (curl, jq)
• Deduplication of extracted dependencies
• Severity-based result filtering
• Timestamp helpers

---

4. Integration Points

4.1 FreshRSS

The RSS feed output is designed for FreshRSS integration. Three deployment options are documented:

1. Static file — write feed directly to FreshRSS's web directory
2. Nginx — serve feed via a reverse proxy
3. GitLab Pages — publish as a Pages artifact

See docs/freshrss-integration.md for detailed setup.

4.2 GitLab CI/CD

Three pipeline strategies are supported:

1. Scheduled pipeline — daily group-wide scan via --gitlab-group
2. MR gate — block merge requests when HIGH/CRITICAL CVEs are found
3. Multi-project pipeline — trigger scans across downstream projects

See docs/gitlab-ci-integration.md for pipeline YAML examples.

4.3 Automation

The recommended deployment uses a systemd timer running daily at 03:00 with jitter. The scanner writes its RSS feed to a location FreshRSS monitors, creating a continuous monitoring loop without manual intervention.

systemd timer (03:00 daily)
    → scan.sh --gitlab-group mygroup --feed /path/to/feed.xml
        → FreshRSS detects new items
            → Operator reviews CVEs in browser

---

5. Usage

5.1 Scan a Local Repository

./scan.sh /path/to/your/project

5.2 Scan a Single Manifest

./scan.sh --file /path/to/go.mod

5.3 Generate RSS Feed with Severity Filter

NVD_API_KEY=your-key ./scan.sh /path/to/repos \
    --feed output/feed.xml \
    --severity HIGH

5.4 Scan a GitLab Group

GITLAB_TOKEN=your-token NVD_API_KEY=your-key \
    ./scan.sh --gitlab-group mygroup --feed output/feed.xml

5.5 Example Output

See docs/example-output.md for sample scanner output, summary reports, and RSS feed items.

---

6. Observations

6.1 NVD CPE Coverage Is Incomplete

The NVD's CPE dictionary has strong coverage for established products (OpenSSL, nginx, Linux kernel) but poor coverage for language-ecosystem packages (Go modules, npm packages). Many Go and npm libraries lack a registered CPE, which means exact CPE matching misses them entirely. The scanner compensates by using keyword-based search, which trades precision for recall — it may return false positives that require human triage.

6.2 Rate Limiting Constrains Scale

Without an API key, the NVD allows 5 requests per 30 seconds. Scanning 100 dependencies takes approximately 10 minutes. With an API key (free), the same scan takes approximately 2 minutes. For large organizations scanning thousands of packages, consider caching results and implementing delta scanning (only query packages whose versions changed since the last scan).

6.3 RSS Is the Right Delivery Mechanism

RSS is a 25-year-old standard that is supported by every feed reader, email client, and automation platform. Using RSS for vulnerability notifications means:

• No vendor-specific webhook format
• No API subscription for consumers
• Works with FreshRSS, Miniflux, Thunderbird, or any RSS-capable tool
• Can be aggregated, filtered, and forwarded without custom code

6.4 CI Integration Changes Behavior

When vulnerability scanning is a CI gate — merge requests fail if HIGH/CRITICAL CVEs are present — teams stop introducing vulnerable dependencies. The scanner becomes preventive, not just detective.

---

7. Conclusions

1. Dependency scanning is a solvable problem — extracting packages from manifests and querying a public database requires 500 lines of Bash, not a SaaS subscription. The NVD API is free, authoritative, and maintained by NIST.

2. RSS remains the optimal notification format — it's push-capable, filterable, self-hosted, and vendor-neutral. FreshRSS turns vulnerability feeds into a centralized, browsable dashboard.

3. GitLab CI integration makes scanning continuous — scheduled pipelines and MR gates ensure that vulnerability status is always current and actionable, not a quarterly review exercise.

4. False positives are acceptable at scale — keyword-based NVD matching produces some noise, but missing a real CRITICAL CVE is worse than reviewing a false positive. The severity filter (--severity HIGH) reduces noise for operators who only want actionable findings.

5. The tool is deliberately minimal — curl, jq, and bash are available on every Linux server. No Docker, no Python, no Node.js runtime. A security tool that requires a complex runtime defeats the purpose of keeping infrastructure simple.

---

Appendix: Repository Structure

vuln-tracker/
├── README.md                       ← This document
├── scan.sh                         # Entry point — 3-phase pipeline orchestrator
├── config.env.example              # Configuration template
├── lib/
│   ├── common.sh                   # Logging, config, dedup, severity filtering
│   ├── parsers.sh                  # Manifest parsers (go.mod, package.json, etc.)
│   ├── nvd.sh                      # NVD API 2.0 client (CPE, keyword, batch)
│   └── feed.sh                     # RSS 2.0 generator + text summary
└── docs/
    ├── freshrss-integration.md     # FreshRSS setup guide
    ├── gitlab-ci-integration.md    # GitLab CI pipeline examples
    └── example-output.md           # Sample scanner output

Requirements

• Bash 4.0+
• curl (NVD API requests)
• jq (JSON parsing — apt install jq)
• git (optional — for --gitlab-group cloning)
• python3 (optional — for URL encoding fallback)
• NVD API Key (optional but recommended — request here)

License

MIT