CIC‐IDS2017 Dataset Inspection Report (Tuesday Working Hours)
File inspected: Tuesday-WorkingHours.pcap_ISCX.csv
This file contains normal traffic(BENIGN) along with FTP-Patator attacks and SSH-Patator attacks
| Property | Value |
|---|---|
| Rows | 445,909 |
| Columns | 79 |
Each row represents a network flow.
A network flow is a summarized network conversation between two devices. Instead of storing every individual packet separately, the dataset stores statistical information about the connection such as packet counts, bytes transferred, connection duration, timing information, packet size statistics, traffic rates
The raw CSV file contains hidden leading spaces in many column names.
Examples:
" Label"
" Flow Duration"
" Total Fwd Packets"This comes from the original formatting of the CICIDS2017 dataset. Hidden spaces can cause problems later during feature selection, preprocessing, training, inference.
All column names will be normalized using:
df.columns = df.columns.str.strip()After normalization:
" Label" -> "Label"
" Flow Duration" -> "Flow Duration"The dataset is heavily imbalanced.
| Label | Count | Percentage |
|---|---|---|
| BENIGN | 432,074 | 96.90% |
| FTP-Patator | 7,938 | 1.78% |
| SSH-Patator | 5,897 | 1.32% |
This reflects realistic network behavior where malicious traffic represents only a small percentage of total activity. The imbalance is important because machine learning models can become biased toward predicting the majority class (BENIGN) if evaluation metrics are not chosen carefully.
Missing value analysis showed:
| Column | Missing Values |
|---|---|
| Flow Bytes/s | 201 |
All rows containing missing values belong to the BENIGN class.
A missing value or Not a Number(NaN) means the dataset generator failed to produce or store a valid value for that feature.
Infinite values were detected in the following columns:
| Column | Infinite Values |
|---|---|
| Flow Bytes/s | 63 |
| Flow Packets/s | 264 |
To understand why this happens, it is necessary to understand what these features represent.
Flow Bytes/s means How many bytes moved every second during the connection.
Flow Packets/s means How many packets moved every second during the connection.
These values are approximately calculated as:
Flow Bytes/s = Total Bytes / Flow Duration
Flow Packets/s = Total Packets / Flow DurationSome flows in the dataset contain:
| Feature | Value |
|---|---|
| Flow Duration | 0 |
This means the connection started and ended so quickly that the measured duration became zero. However, some of these flows still contain packet activity, transferred bytes, real network behavior
Example:
| Total Bytes | Flow Duration |
|---|---|
| 100 | 0 |
The dataset generator attempts:
100 / 0Division by zero is mathematically impossible. Instead of crashing, the calculation produces infinity
This means the infinity values are not random corruption. They are mathematical side effects caused by divide-by-zero calculations.
Rows containing infinity values were inspected manually.
Observations:
- some rows still contain valid packet counts
- some rows transferred actual bytes
- some flows completed extremely quickly
- not all rows are empty or corrupted
Example observations included:
- backward packets existing even when forward packets are zero
- bytes transferred despite extremely short durations
This indicates that some zero-duration flows may still represent meaningful network behavior.
Very short-lived flows can occur during rapid connection attempts, reconnaissance traffic, scanning activity, brute-force authentication attempts, incomplete TCP handshakes. This means zero-duration flow != useless flow. A mathematically invalid throughput calculation does not automatically mean the underlying network event is invalid.
The inspection revealed an important distinction between:
| Type | Meaning |
|---|---|
| Missing telemetry | value was never recorded |
| Mathematical artifact | divide-by-zero calculation |
| Real network behavior | actual packet activity |
The infinity values mostly belong to the second category. The network event itself may still be real even if one derived metric becomes mathematically invalid.
The inspection process showed that unusual values in cybersecurity datasets are not always simple data corruption. Some invalid values originate from timing limitations, flow generation methods, mathematical edge cases, extremely short network behavior
Understanding why these values exist is important before designing preprocessing or training pipelines.
