Skip to content

RogoLabs/CVE-Splunk-TA

BranchesTags

Repository files navigation

TA-cvelist - CVE Intelligence for Splunk

Splunk Technology Add-on for ingesting CVE V5 records from the official CVE Program repository.

Overview

This add-on ingests CVE (Common Vulnerabilities and Exposures) V5 records from the CVEProject/cvelistV5 GitHub repository into Splunk. It is a clean-room rewrite focused on pure CVE data without external EPSS/KEV enrichment.

Key Features

  • Efficient Bulk Download: Uses GitHub Release ZIP files instead of per-file API calls
  • Incremental Updates: Baseline load followed by hourly delta processing
  • Full CVE V5 Schema Support: Extracts cveMetadata, CNA containers, and ADP enrichment
  • CVSS Score Extraction: Parses CVSS v2.0, v3.0, v3.1, and v4.0 scores
  • CISA-ADP Integration: Includes CISA Authorized Data Publisher enrichment (SSVC, inline KEV)
  • CPE and Version Range Extraction: Structured affected product data
  • Secure Credential Storage: GitHub token stored via Splunk's encrypted storage
  • Resource Management: Memory limits (512MB) and timeout guards (1 hour)
  • Resumable Baseline: Checkpoint-based resume capability for large initial loads
  • CIM Vulnerabilities Mapping: Compatible with Splunk Common Information Model
  • Cloud Compatible: Meets Splunk Cloud and AppInspect vetting requirements

Requirements

  • Splunk Enterprise 10.0+ or Splunk Cloud
  • Python 3.11+ (bundled with Splunk 10)
  • Network access to GitHub API (api.github.com)

Installation

From Package

  1. Download the latest .tar.gz package from Releases
  2. Go to Apps > Manage Apps > Install app from file
  3. Upload the .tar.gz package
  4. Restart Splunk if prompted

Manual Installation

  1. Extract the add-on to $SPLUNK_HOME/etc/apps/
  2. Restart Splunk

Configuration

The add-on starts ingesting CVE data immediately after installation using a default input. No setup is required for basic operation.

GitHub Token (Optional but Recommended)

The add-on works without authentication, but GitHub's unauthenticated API rate limit is 60 requests/hour — shared across all unauthenticated traffic from your IP. If you run other tools that call the GitHub API (CLI, CI/CD, IDE plugins, etc.), you may exhaust this limit and see "rate limit exceeded" errors.

A GitHub Personal Access Token increases this to 5,000 requests/hour:

  1. Create a token at Settings > Developer settings > Personal access tokens > Tokens (classic)
  2. Select scope: public_repo (read-only access to public repositories)
  3. In Splunk, store the token via the REST API: 
    curl -k -u admin:<password> \
  https://localhost:8089/servicesNS/nobody/TA-cvelist/storage/passwords \
  -d name=github_api_token -d realm=TA-cvelist -d password=<your_token>

Note: Most users with light GitHub API usage will not hit the rate limit. The initial baseline download requires only a few API calls. If you see "GitHub API rate limit exceeded" errors in the add-on logs, configuring a token will resolve the issue.

Customizing the Data Input

The default input writes to index=main with hourly polling. To customize:

# local/inputs.conf
[cvelist://default]
index = cve_data
include_adp = true
include_rejected = false
batch_size = 500
interval = 3600

You can also create additional inputs via Settings > Data Inputs > CVE List.

Changing the Index

All dashboards and saved searches use the cvelist_index macro (defaults to index=main). To use a different index:

# local/macros.conf
[cvelist_index]
definition = index=cve_data

Sourcetypes

Sourcetype Description
cvelist:record CVE vulnerability records (primary data)
cvelist:error Error events during processing
cvelist:audit Audit and operational events

Extracted Fields

Core CVE Fields

  • cve_id - CVE identifier (e.g., CVE-2024-1234)
  • state - Record state (PUBLISHED, REJECTED)
  • date_published - Initial publication date
  • date_updated - Last modification date
  • date_reserved - Date CVE ID was reserved
  • assigner - Assigning CNA short name
  • assigner_org_id - CNA organization UUID
  • title - Vulnerability title (from descriptions)
  • description - Vulnerability description

Affected Products

  • affected_vendor - Affected vendor names (multi-value)
  • affected_product - Affected product names (multi-value)
  • affected_versions - Version strings (multi-value)
  • cpe - Common Platform Enumeration strings (multi-value)
  • cwe_id - CWE identifiers (multi-value)

CVSS Scores

  • cvss_v40_score, cvss_v40_severity, cvss_v40_vector - CVSS v4.0
  • cvss_v31_score, cvss_v31_severity, cvss_v31_vector - CVSS v3.1
  • cvss_v30_score, cvss_v30_severity, cvss_v30_vector - CVSS v3.0
  • cvss_v20_score, cvss_v20_vector - CVSS v2.0
  • cvss_score - Best available CVSS score (v4.0 > v3.1 > v3.0 > v2.0)
  • cvss_severity - Best available severity rating

ADP Enrichment

  • has_cisa_adp - Boolean: CISA-ADP data present
  • has_cve_program_container - Boolean: CVE Program Container present
  • cisa_kev_due_date - CISA KEV due date (if present in ADP)
  • cisa_kev_required_action - CISA KEV required action
  • ssvc_exploitation, ssvc_automatable, ssvc_technical_impact - CISA SSVC decision tree values (multi-value)

References

  • reference - Reference URLs (multi-value)

CIM Vulnerabilities Data Model

The following fields map to the Splunk CIM Vulnerabilities data model:

  • vulnerability_id - CVE identifier
  • cvss - Best available CVSS score
  • severity_id - CVSS score (numeric severity)
  • vendor_product - Affected vendor
  • dest - CVE identifier (destination)
  • signature - CVE identifier
  • signature_id - CVE identifier
  • category - CWE classification

Example Searches

High Severity CVEs (Last 7 Days)

`cvelist_index` sourcetype="cvelist:record"
| where cvss_v31_score >= 9.0 OR cvss_v40_score >= 9.0
| eval severity=coalesce(cvss_v40_severity, cvss_v31_severity, "Unknown")
| table cve_id, title, severity, affected_vendor, affected_product

CVEs by Vendor

`cvelist_index` sourcetype="cvelist:record"
| mvexpand affected_vendor
| stats count by affected_vendor
| sort -count
| head 20

CVEs with CISA-ADP Enrichment

`cvelist_index` sourcetype="cvelist:record" has_cisa_adp=true
| table cve_id, title, cisa_kev_due_date, ssvc_exploitation

CISA KEV Tracking (Inline from ADP)

`cvelist_index` sourcetype="cvelist:record" cisa_kev_due_date=*
| table cve_id, title, cisa_kev_due_date, cisa_kev_required_action, cvss_score
| sort cisa_kev_due_date

New CVEs by Day

`cvelist_index` sourcetype="cvelist:record"
| eval _time=strptime(date_published, "%Y-%m-%dT%H:%M:%S.%3NZ")
| timechart span=1d count

CVEs with SSVC Exploitation Active

`cvelist_index` sourcetype="cvelist:record" ssvc_exploitation=active
| table cve_id, title, ssvc_exploitation, ssvc_automatable, ssvc_technical_impact

Dashboards

CVE Explorer

Search and filter the full CVE database. Includes:

  • Publication sparkline (last 30 days)
  • Filterable table with vendor/severity/CWE dropdowns
  • CISA-ADP filter toggle
  • Date-sorted results

Operational Health

Technical diagnostics dashboard:

  • Last successful run timestamp
  • Error count (24 hours)
  • Total events ingested
  • Lookup table sizes
  • Recent audit events
  • Daily volume by publication date
  • Recent errors/warnings from splunkd

Troubleshooting

Check Input Status

index=_internal sourcetype=splunkd component=ModularInputs "cvelist"

Check Add-on Logs

index=_internal source="*TA-cvelist.log*"

Verify Checkpoint

| inputlookup ta_cvelist_checkpoints

Check CVE Lookup

| inputlookup cve_lookup
| head 100

Common Issues

Issue Solution
Rate limit errors Configure GitHub Personal Access Token
No events ingested Check network connectivity to api.github.com
Incomplete initial load Allow sufficient time; baseline contains 300K+ CVEs
Memory errors Resource guards will terminate and resume on next run
Timeout during baseline Baseline is resumable via checkpoint

Data Volume Estimates

  • Initial Load: ~300,000+ CVE records (~2-3 GB indexed)
  • Daily Updates: ~50-200 new/updated CVEs (~10-50 MB/day)
  • Hourly Deltas: ~5-20 CVEs per delta release

Architecture

GitHub CVEProject/cvelistV5
    │
    ├── Releases
    │   ├── cve_X.X.X (baseline) → all_CVEs.zip → nested per-year ZIPs
    │   └── cve_X.X.X_YYYYMMDD_HHMM (delta) → deltaCves.zip
    │
    ▼
TA-cvelist Modular Input (cvelist.py)
    │
    ├── GitHub Client: Download ZIP (streaming)
    ├── CVE Processor: Parse CVE JSON, extract CVSS/SSVC/CPE
    ├── Checkpoint: KV Store resume tracking
    ├── Resource Guards: 512MB memory limit, 1hr timeout
    └── Rate Limiter: Exponential backoff for API calls
    │
    ▼
Splunk Index                          KV Store Collections
    ├── sourcetype=cvelist:record         ├── ta_cvelist_checkpoints
    ├── sourcetype=cvelist:error          └── cve_lookup (from saved search)
    └── sourcetype=cvelist:audit

Development

Running Tests

# Unit tests
PYTHONPATH=TA-cvelist/bin:TA-cvelist/bin/lib pytest tests/unit/ -v

# Integration tests (requires Docker)
docker compose -f docker-compose.test.yml up -d
pytest tests/integration/ -v

Building Package

COPYFILE_DISABLE=1 tar -czf TA-cvelist-1.0.0.tar.gz \
  --exclude='.*' --exclude='__pycache__' --exclude='*.pyc' \
  --exclude='local' --exclude='metadata/local.meta' \
  TA-cvelist/

AppInspect

splunk-appinspect inspect TA-cvelist-1.0.0.tar.gz --mode precert

Support

License

Apache License 2.0 - See LICENSE

About

Splunk CVE Data TA

Resources

Readme

License

Apache-2.0 license
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages