Real-Time Network Vulnerability Scanner

A production-grade network vulnerability scanner with real-time monitoring capabilities, enhanced with CVE/NVD database integration, EPSS exploit prediction scoring, and advanced risk assessment.

⚠️ Legal and Ethical Notice

IMPORTANT: This tool is designed for authorized security testing only. Users are responsible for:

• Obtaining proper authorization before scanning any networks
• Complying with local laws and regulations
• Using the tool only on networks you own or have explicit permission to test
• Not using this tool for malicious purposes

Unauthorized network scanning may be illegal in your jurisdiction.

Architecture

The scanner follows a microservices architecture:

• Scanner Controller: Schedules and manages scan jobs
• Scanner Workers: Execute port scans, service fingerprinting, and vulnerability checks
• Nmap NSE Integration: 600+ Nmap Scripting Engine scripts for comprehensive vulnerability detection
• Enhanced Service Detection: Deep probing, protocol state analysis, and honeypot detection
• CVE/NVD Integration: Real-time vulnerability database with EPSS exploit prediction
• Advanced Vulnerability Scanner: Configuration-specific checks and cryptographic analysis
• Enhanced Risk Engine: Multi-factor risk assessment with external threat intelligence
• Aggregator: Processes scan results and calculates comprehensive risk scores
• Web API: FastAPI service with REST and WebSocket endpoints
• Dashboard: Real-time web interface for monitoring scan results
• Message Queue: Redis for job queuing and event streaming
• Database: PostgreSQL for persistent storage, Redis for caching

Quick Start

1. Prerequisites:

   • Docker and Docker Compose
   • Python 3.11+ (for development)

2. Run the scanner:

   # Clone and navigate to project
   cd network-vulnerability-scanner
   
   # Start all services
   docker-compose up -d
   
   # Wait for services to be ready (30-60 seconds)
   docker-compose logs -f
   
   # Access dashboard at http://localhost:8080
   # API documentation at http://localhost:8000/docs

3. Run demo scan:

   # Basic demo
   chmod +x demo_scan.sh
   ./demo_scan.sh
   
   # Enhanced vulnerability detection demo
   python demo_enhanced_vulnerability_detection.py scanme.nmap.org
   
   # Advanced Nmap NSE scanning demo (NEW!)
   python demo_nmap_advanced.py scanme.nmap.org
   python demo_nmap_advanced.py 192.168.1.100 comprehensive

Configuration

Edit config/scanner.yaml to customize:

• Scan profiles: fast, safe, deep
• Target networks: CIDR ranges and IP lists
• Auto-discovery: automatically find network hosts
• Rate limiting: concurrent scans and delays
• Vulnerability checks: enable/disable specific tests
• Enhanced service detection: deep probing and honeypot detection
• External databases: CVE/NVD integration and EPSS scoring
• Risk assessment: advanced multi-factor risk calculation

Example configuration:

scanner:
  profile: "safe"  # fast, safe, deep
  max_concurrent_hosts: 50
  max_concurrent_ports: 100
  scan_timeout: 30
  rate_limit_delay: 0.1

targets:
  auto_discovery:
    enabled: true                    # Enable automatic network discovery
    methods: ["ping", "arp"]        # Discovery methods
    auto_detect_networks: true      # Auto-detect local networks
    include_gateways: false         # Include gateway devices
    enhanced_discovery:
      enabled: true                 # Enable enhanced network discovery
      aggressive_arp: true          # Aggressive ARP with MAC mapping
      traceroute_analysis: true     # Network topology mapping
      passive_monitoring: false     # DHCP/DNS monitoring (requires privileges)
  networks:
    - "192.168.1.0/24"
    - "10.0.0.0/8"
  hosts:
    - "scanme.nmap.org"

vulnerability_checks:
  enabled: true
  intrusive: false  # Set to true for more aggressive scanning

enhanced_service_detection:
  enabled: true
  deep_probing: true
  protocol_state_analysis: true
  honeypot_detection: true

Enhanced Service Detection

The scanner includes advanced service detection capabilities that go beyond basic banner grabbing:

Deep Service Probing (Fingerprinting)

• Protocol-Specific Probes: Sends targeted requests to elicit detailed responses
• HTTP Probing: HEAD /, GET /, OPTIONS / requests for accurate web server detection
• Database Probing: MySQL handshake analysis, PostgreSQL SSL requests
• SSH Probing: Version negotiation and capability detection
• Multi-Probe Analysis: Combines results from multiple probes for higher accuracy

Protocol State Analysis

• FTP State Traversal: SYST, HELP commands to identify server type and capabilities
• SMTP Interaction: EHLO/HELO commands to discover mail server features
• SSH Negotiation: Protocol version exchange and feature detection
• Stateful Logic: Analyzes command responses and error codes for service confirmation

Honeypot/Deception Detection

• Signature Detection: Identifies known honeypot software (Kippo, Dionaea, Honeyd)
• Response Analysis: Detects generic/template responses indicating fake services
• SSL Certificate Analysis: Identifies suspicious certificates (localhost, example.com)
• Timing Analysis: Detects unusual response patterns
• Confidence Scoring: Provides reliability metrics for service identification

Enhanced Features

• Version Extraction: Detailed version and product information
• Confidence Metrics: Reliability scores for service identification
• SSL/TLS Analysis: Enhanced certificate and cipher suite detection
• Banner Analysis: Advanced pattern matching for service fingerprinting

Demo Enhanced Scanning

# Run enhanced service detection demo
python demo_enhanced_scanning.py scanme.nmap.org 22,80,443

# Compare basic vs enhanced detection
python demo_enhanced_scanning.py 127.0.0.1 21,22,25,80

Enhanced Vulnerability Detection

The scanner now includes comprehensive vulnerability detection with external database integration and advanced risk assessment:

CVE/NVD Database Integration

• Real-time Updates: Automatically syncs with National Vulnerability Database (NVD)
• CVE Matching: Maps detected services/versions to known CVEs
• CVSS Scoring: Official CVSS v3.1 scores for accurate risk assessment
• Comprehensive Coverage: Access to 200,000+ vulnerability records

EPSS Exploit Prediction Scoring

• Exploit Likelihood: Uses EPSS (Exploit Prediction Scoring System) data
• Prioritization: Identifies vulnerabilities most likely to be exploited
• Real-world Context: Based on actual exploit activity in the wild
• Risk Prioritization: Combines CVSS severity with exploit probability

Configuration-Specific Vulnerability Checks

• Default Credentials: Tests for common username/password combinations
• Weak Configurations: Identifies insecure service configurations
• Anonymous Access: Checks for unauthorized access (FTP, SMB, etc.)
• Directory Traversal: Tests for path traversal vulnerabilities
• Admin Interfaces: Detects exposed management interfaces

Advanced Cryptographic Analysis

• Weak Protocols: Detects SSLv2, SSLv3, TLSv1.0/1.1 support
• Cipher Suite Analysis: Identifies weak encryption algorithms (RC4, DES, 3DES)
• Certificate Validation: Checks for expired, self-signed, or weak certificates
• Key Strength Analysis: Validates RSA/ECDSA key sizes
• Known Vulnerabilities: Tests for Heartbleed, POODLE, CRIME, etc.

Multi-Factor Risk Assessment

The enhanced risk engine considers multiple factors:

1. Vulnerability Severity (35% weight)

   • CVSS scores and severity levels
   • Number and type of vulnerabilities

2. Exploit Availability (25% weight)

   • EPSS scores for exploit prediction
   • Known exploit code availability

3. Service Exposure (15% weight)

   • Network accessibility (internet vs internal)
   • Service criticality (database, admin, etc.)

4. Configuration Weakness (10% weight)

   • Default credentials and misconfigurations
   • Insecure protocol usage

5. Asset Criticality (10% weight)

   • Service importance to business operations
   • Asset type and environment

6. Temporal Factors (5% weight)

   • Vulnerability age and patch availability
   • Time since disclosure

Enhanced Features

• Intelligent Deduplication: Removes duplicate vulnerability findings
• Confidence Scoring: Reliability metrics for each finding
• Contextual Recommendations: Tailored remediation advice
• Priority Scoring: Combines risk and exploit likelihood
• Comprehensive Reporting: Detailed vulnerability analysis

🔍 Advanced Nmap NSE Integration (NEW!)

The scanner now leverages Nmap's powerful NSE (Nmap Scripting Engine) with 600+ specialized vulnerability detection scripts:

NSE-Based Vulnerability Scanning

• 600+ NSE Scripts: Comprehensive vulnerability coverage across all major services
• Automated CVE Extraction: Automatically extracts CVE IDs from NSE output
• Service-Specific Scripts: Intelligent script selection based on detected services
• Multi-Category Support: Safe, intrusive, discovery, and vulnerability-focused scripts

Service-Specific NSE Scripts

Service	Example NSE Scripts
HTTP/HTTPS	http-vuln-*, http-sql-injection, http-shellshock, http-xss
SSL/TLS	ssl-heartbleed, ssl-poodle, ssl-ccs-injection, sslv2-drown
SSH	ssh-auth-methods, ssh-brute, ssh-hostkey, sshv1
SMB	smb-vuln-*, smb-os-discovery, smb-enum-shares
FTP	ftp-anon, ftp-vsftpd-backdoor, ftp-vuln-cve2010-4221
MySQL	mysql-audit, mysql-empty-password, mysql-vuln-*
SMTP	smtp-vuln-*, smtp-open-relay, smtp-commands
DNS	dns-zone-transfer, dns-recursion, dns-cache-snoop

Scan Profiles

Profile	Port Range	Scan Type	NSE Category	Duration
Quick	1-1000	TCP Connect	Safe	~30-60s
Standard	1-10000	SYN Scan	Vuln	~5-10m
Comprehensive	1-65535	SYN Scan	Vuln	~20-60m
Stealth	1-1000	Slow SYN	Safe	~10-20m

OS Detection & Fingerprinting

• Advanced OS Detection: Uses Nmap's extensive fingerprinting database
• EOL Detection: Identifies End-of-Life operating systems (Windows XP, Vista, 7, etc.)
• Accuracy Metrics: Provides confidence percentage for OS matches
• Multiple Matches: Shows alternative OS possibilities

Key Features

• ✅ Automated CVE Correlation: Extracts CVE IDs and queries vulnerability databases
• ✅ Severity Analysis: Intelligent severity detection from NSE output
• ✅ Evidence Collection: Captures detailed evidence for each finding
• ✅ Deduplication: Removes duplicate findings across multiple scan types
• ✅ Integration: Seamlessly works with existing vulnerability scanners

Usage Example

from src.scanners.nmap_vulnerability_scanner import NmapVulnerabilityScanner

# Run comprehensive NSE scan
result = await nmap_scanner.run_comprehensive_nmap_scan(
    target="192.168.1.100",
    scan_profile="comprehensive"
)

# Access results
ports = result["port_scan"]["ports"]
services = result["service_detection"]["services"]
vulnerabilities = result["vulnerabilities"]
os_info = result["os_detection"]

Demo Script

# Run comprehensive Nmap demo
python demo_nmap_advanced.py scanme.nmap.org

# Use different scan profiles
python demo_nmap_advanced.py 192.168.1.100 comprehensive
python demo_nmap_advanced.py example.com stealth

For detailed documentation, see NMAP_ENHANCEMENTS.md

Enhanced Auto-Discovery Reliability

The scanner includes advanced auto-discovery capabilities for improved network accuracy, especially in complex environments:

Layer 2 Discovery Enhancement

• Aggressive ARP Discovery: Enhanced ARP probing with MAC address mapping
• ARP Table Refresh: Clears and refreshes ARP cache for accurate results
• Vendor Identification: Maps MAC addresses to device vendors
• Network Interface Analysis: Comprehensive local network detection

Traceroute Analysis

• Network Topology Mapping: Identifies intermediate routing devices
• Hop Distance Calculation: Maps network topology structure
• Router Identification: Distinguishes routers from end hosts
• Gateway Discovery: Automatically identifies network gateways

DHCP/DNS Inspection

• Passive DHCP Monitoring: Inspects DHCP lease files for active hosts
• DNS Record Analysis: Parses DNS records for hostname mapping
• Static Host Discovery: Reads /etc/hosts for static entries
• Lease Information: Extracts MAC addresses and hostnames from DHCP

Demo Enhanced Discovery

# Run enhanced discovery demo
python demo_enhanced_discovery.py

# Test auto-discovery functionality
python test_auto_discovery.py

# Compare standard vs enhanced discovery
python demo_enhanced_discovery.py --compare

Enhanced Discovery Configuration

targets:
  auto_discovery:
    enhanced_discovery:
      enabled: true                 # Enable enhanced network discovery
      aggressive_arp: true          # Aggressive ARP with MAC mapping
      traceroute_analysis: true     # Network topology mapping
      passive_monitoring: false     # DHCP/DNS monitoring (requires privileges)
      max_hops: 10                  # Maximum hops for traceroute
      arp_refresh: true             # Clear ARP table before discovery
      vendor_identification: true   # Identify device vendors
      topology_mapping: true        # Build network topology map

Demo Enhanced Vulnerability Detection

# Run comprehensive vulnerability scan with external database integration
python demo_enhanced_vulnerability_detection.py scanme.nmap.org

# Scan with specific context
python demo_enhanced_vulnerability_detection.py database-server.example.com

# View detailed cryptographic analysis
python demo_enhanced_vulnerability_detection.py secure-site.example.com

Configuration Example

vulnerability_checks:
  enabled: true
  intrusive: false  # Enable for deep profile only
  
  # External database integration
  external_databases:
    nvd_enabled: true
    nvd_api_key: ""  # Optional for higher rate limits
    epss_enabled: true
    update_interval: 86400  # 24 hours
  
  # Enhanced checks
  checks:
    weak_encryption: true
    certificate_analysis: true
    protocol_analysis: true
    configuration_checks: true
    default_credentials: false  # Only in deep profile

# Enhanced risk scoring
risk_scoring:
  enabled: true
  weights:
    vulnerability_severity: 0.35
    exploit_availability: 0.25
    service_exposure: 0.15
    configuration_weakness: 0.10
    asset_criticality: 0.10
    temporal_factors: 0.05

Auto-Discovery Features

The scanner now includes automatic network discovery capabilities:

Automatic Host Detection

• Ping Sweep: Discovers active hosts using ICMP ping
• ARP Discovery: Finds hosts from ARP table entries
• Port Scanning: Detects hosts with open ports

Smart Network Detection

• Automatically detects local network interfaces
• Identifies network ranges (CIDR notation)
• Excludes loopback and link-local addresses
• Optional gateway inclusion/exclusion

Dashboard Integration

• Auto-Discovery Mode: Select in scan creation dialog
• Preview Hosts: See discoverable hosts before scanning
• Real-time Results: Live updates as hosts are found

Usage Examples

Via Dashboard:

1. Click "New Scan" → Select "Auto-Discovery"
2. Click "Preview discoverable hosts" to see available targets
3. Start scan to automatically scan all discovered hosts

Via API:

# Create auto-discovery scan
curl -X POST "http://localhost:8000/api/v1/scans/auto-discovery" \
  -H "Content-Type: application/json" \
  -d '{"name": "Auto Network Scan", "profile": "safe"}'

# Preview discoverable hosts  
curl "http://localhost:8000/api/v1/network/discovery"

Configuration Options:

targets:
  auto_discovery:
    enabled: true                    # Enable/disable auto-discovery
    methods: ["ping", "arp"]        # Discovery methods to use
    auto_detect_networks: true      # Auto-detect local networks
    include_gateways: false         # Include gateway devices
    max_network_size: 1024          # Max hosts per network
    discovery_timeout: 2.0          # Discovery timeout in seconds

API Usage

REST Endpoints

# Start a manual scan
curl -X POST "http://localhost:8000/api/v1/scans" \
  -H "Content-Type: application/json" \
  -d '{"targets": ["192.168.1.0/24"], "profile": "safe"}'

# Start an auto-discovery scan (automatically finds network hosts)
curl -X POST "http://localhost:8000/api/v1/scans/auto-discovery" \
  -H "Content-Type: application/json" \
  -d '{"name": "Auto Network Scan", "profile": "safe"}'

# Get network discovery information
curl "http://localhost:8000/api/v1/network/discovery"

# Get scan results
curl "http://localhost:8000/api/v1/scans/{scan_id}/results"

# Get host details
curl "http://localhost:8000/api/v1/hosts/192.168.1.10"

WebSocket Events

Connect to ws://localhost:8000/ws to receive real-time events:

{
  "type": "scan_result",
  "host": "192.168.1.10",
  "ports": [
    {
      "port": 22,
      "state": "open",
      "service": "ssh",
      "banner": "OpenSSH 8.9p1",
      "vulnerabilities": []
    }
  ],
  "risk_score": 3.4,
  "timestamp": "2025-01-01T12:00:00Z"
}

Development

Local Setup

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Run tests
pytest tests/

# Run individual components
python src/controller.py
python src/worker.py
python src/aggregator.py
python src/api.py

Adding New Vulnerability Checks

1. Create a new check in src/scanners/vulnerability_checks.py
2. Register it in the VulnerabilityScanner class
3. Add test cases in tests/test_vulnerability_checks.py

Example:

async def check_weak_ssh_config(self, host: str, port: int, banner: str) -> List[Vulnerability]:
    if "ssh" in banner.lower() and "openssh" in banner.lower():
        # Check for weak configurations
        return [Vulnerability(
            name="Weak SSH Configuration",
            severity="medium",
            description="SSH service detected with potential weak configuration"
        )]
    return []

Monitoring and Observability

• Health checks: http://localhost:8000/health
• Metrics: Prometheus metrics at http://localhost:8000/metrics
• Logs: Structured JSON logs via Docker Compose logs

Security Considerations

• All scans are non-intrusive by default
• Rate limiting prevents network flooding
• No exploit payloads are executed
• Configurable scan intensity levels
• Audit logging for all scan activities

Troubleshooting

Common Issues

1. Permission denied for packet capture:

   # Run with elevated privileges or disable packet capture
   sudo docker-compose up

2. High CPU usage:

   • Reduce max_concurrent_hosts in configuration
   • Use "fast" profile for lighter scanning

3. Database connection errors:

   • Ensure PostgreSQL container is healthy
   • Check database credentials in docker-compose.yml

Debug Mode

Enable debug logging:

export LOG_LEVEL=DEBUG
docker-compose up

Contributing

1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Ensure all tests pass
5. Submit a pull request

License

MIT License - see LICENSE file for details.