NetSentinel AI

NetSentinel AI is an AI-assisted network monitoring and incident detection system developed for Cisco Catalyst switches.

The objective of the project is to automate network monitoring by collecting switch logs over SSH, parsing operational data, detecting known and unknown network faults using both rule-based logic and machine learning, storing incidents in a database, and notifying administrators through a Streamlit dashboard, HTML email alerts, and audible notifications.

Network incident detection and monitoring for Cisco Catalyst C9200L (IOS XE 17.15) over SSH. Collects, parses, stores, detects, and visualizes.

Technologies Used

Programming Language

• Python

Libraries

• Netmiko
• Streamlit
• Pandas
• NumPy
• Scikit-learn
• SQLite3
• Regex (re)
• smtplib

Machine Learning

• Isolation Forest

Networking

• SSH
• Cisco IOS XE

Database

• SQLite

Architecture

      Cisco Switch

           │

           ▼

     SSH Collector

           │

           ▼

        Parser

           │

   ┌───────┴────────┐

   ▼                ▼

Rule Engine Isolation Forest

   └───────┬────────┘

           ▼

       SQLite DB

    ┌─────────────┐

    ▼             ▼

Dashboard Email Alerts

config.py     switch inventory, commands, thresholds (edit this to scale)
collector.py  Netmiko SSH -> raw CLI text (no parsing)
parser.py     raw text -> structured dicts (pure, testable)
rules.py      structured data + history -> incidents (delta-aware)
database.py   SQLite schema + typed access layer
monitor.py    orchestrator: collect -> parse -> store -> detect -> store
dashboard.py  Streamlit read-only view

Data flows one way: monitor.py writes the DB on a schedule; dashboard.py reads it. They run as separate processes.

Setup

pip install -r requirements.txt

# credentials via env (do NOT hard-code in config.py for production)
export NS_SW1_HOST=10.0.0.1
export NS_SW1_USER=netsentinel
export NS_SW1_PASS='...'
export NS_SW1_SECRET='...'   # if enable secret required

Run

# Offline: run against a captured console session (no switch, no netmiko)
python monitor.py --sample sample_C_SERVERROOM_13.txt

python monitor.py --once     # single live poll cycle
python monitor.py            # live loop on POLL_INTERVAL_SECONDS
streamlit run dashboard.py   # dashboard (separate terminal)

In production run monitor.py under systemd or as a cron --once job.

Validated against real C9200L output

sample_C_SERVERROOM_13.txt is a real console capture. Running it produces:

• 28 ports, 13 connected
• Gi1/0/20: 45,223 FCS + 45,296 receive errors -> 2 Critical (failing optic/cable)
• 6 ports with output discards > 1000 -> Warning
• CPU 1% (read from the history graph)
• Fans OK, PSU 1A Good, PSU bay 1B / PS2 NOT PRESENT -> correctly ignored, not alerted
• All temperatures under 70 C -> no false alerts

The parsers handle real-world quirks the first cut missed: the three stacked sub-tables in counters errors, trunk VLANs, names like >>> C_AP_CS, the Sensor List / FAN table / PSU table layout of show environment all, and unpopulated PSU/fan bays (treated as inventory, not incidents).

Adding switches

Append a SwitchConfig to SWITCHES in config.py. Everything else (collection, storage, detection, dashboard selector) is already multi-switch.

Alerting

When an incident is detected, notifier.py alerts the user by channel, chosen per severity in config.NOTIFY (default: Critical = email + beep, Warning = email). Two behaviours keep it useful rather than noisy:

• New faults only. An incident already present in the previous poll is suppressed, so a fault that persists for hours alerts once, not every cycle.
• Email dry-run by default. Without SMTP configured, the intended email is logged ([EMAIL DRY-RUN] ...) so the system is demonstrable with no mail server. To send real mail, set NS_EMAIL_ENABLED=true and the NS_SMTP_* / NS_EMAIL_* environment variables.

The beep uses winsound on Windows and the terminal bell elsewhere; disable with NOTIFY_BEEP_ENABLED = False.

Dashboard

The Streamlit dashboard provides:

• Total monitored ports
• Connected ports
• Critical alerts
• Warning alerts
• Environment status
• Incident history
• Multi-switch support
• Color-coded incident visualization

Detection approach (rule-based, by design)

Detection is a deterministic rule engine, not a machine-learning model. For threshold-based fault detection this is the right call: it is fully explainable (every alert states which counter crossed which threshold), needs no labelled training data, and never produces false positives from model drift. An ML/anomaly-detection layer is a possible future addition for faults with no fixed threshold, not a replacement for these rules.

Detection approach (rules + ML, layered)

Detection has two layers that complement each other:

1. Deterministic rules (rules.py) apply fixed thresholds to known fault conditions. Fully explainable, no training data, no false positives from model drift. This is the right tool for conditions you can name in advance.

2. ML anomaly layer (ml_detector.py) uses an IsolationForest to flag ports that are statistically unlike their peers, with NO threshold given. This is the 'unknown pattern' capability: a port can be flagged for looking abnormal even when it has not crossed any hard limit.

How they work together:

• The ML layer ranks every interface by an anomaly score (0..1).
• Only ports the rules did NOT already catch, and whose score clears ML_MIN_SCORE, become ML incidents (category Anomaly). No double-alerting.
• ML_MIN_SCORE is the noise/sensitivity lever: higher = fewer, high-confidence anomalies; lower = more sub-threshold early warnings (e.g. a port at 696 discards climbing toward the 1000 rule).

Honest limitation: ML anomaly detection finds statistical outliers, which are not always operational problems (a port with 25 discards is unusual but harmless). That is exactly why ML supplements the rules rather than replacing them, and why the score gate exists. On a single snapshot it works cross-sectionally (port vs peer ports); its full value (temporal baselines, cross-switch patterns) appears once history and more switches accumulate.

Validated on the real sample: with no thresholds, the model ranks Gi1/0/20 #1 (score 1.0, driven by rcv_err/fcs_err) and Gi1/0/3 #2, agreeing with the rules.

Detection rules

#	Condition (per poll interval when USE_DELTA=True)	Category	Severity
1	FCS errors > 100	Network	Critical
2	Receive errors > 100	Network	Critical
3	Output discards > 1000	Network	Warning
4	CPU (1-min avg) > 90%	Performance	Critical
5	CPU (1-min avg) > 70%	Performance	Warning
6	Fan status not OK	Hardware	Critical
7	Power supply not GOOD	Power	Critical
8	Temperature > 70 C	Hardware	Critical

Project Screenshots

Dashboard Overview

Displays the overall health of the monitored Cisco switch including:

• Total Ports
• Connected Ports
• Critical Alerts
• Warning Alerts
• Environment Status

---

Incident Table

Shows all detected incidents with:

• Timestamp
• Severity
• Interface
• Description
• Recommendation

---

Incident Details

Clicking an incident expands it to show complete diagnostic information.

---

HTML Email Alert

When a new incident is detected, NetSentinel AI automatically sends an HTML email summarizing all detected Critical and Warning incidents.

Design decisions you should know

1. Counter rules use deltas, not lifetime totals (USE_DELTA=True). Absolute thresholds on monotonic counters detect uptime, not faults. The first poll for an interface has no prior, so it falls back to the raw value; from the second poll on, thresholds apply to the per-interval increase. Set USE_DELTA=False in config.py for the naive cumulative behaviour.

2. CPU uses show processes cpu | include utilization, not show processes cpu history. The history command returns an ASCII graph that is fragile to parse; the utilization line gives the same 5s/1m/5m percentages cleanly. The 1-minute average is used for alerting.

3. parse_environment() is the platform-specific weak point. Fan/power/temp output format varies by model and image. Validate it against your real show environment all and adjust the regex. (The bundled self-test shows a tabular fan layout that the generic parser does not catch - that is the kind of thing to tune.)

Where this should go next (beyond v1)

SSH polling is a v1 ceiling. For real-time detection at scale, subscribe to IOS XE model-driven telemetry (gNMI/NETCONF) and syslog, and use CLI only for on-demand enrichment. Swap collector.py for a telemetry backend; the parser, rules, database, and dashboard stay as-is. Consider Genie/pyATS or ntc-templates to replace the hand-rolled parsers, and move from SQLite to Postgres/TimescaleDB when you outgrow a single file.