This Jupyter Notebook provides a comprehensive analysis and modeling of network traffic data. The main objectives are to explore the data, handle preprocessing, and apply machine learning models to predict network actions.
To replicate this analysis, ensure your Python environment is set up correctly. Create a requirements.txt file with the following content and install the packages using pip:
You can install all required packages with the command:
pip install -r requirements.txt
pandas,numpy: For data manipulation and numerical calculations.matplotlib,seaborn: For plotting and visualizations.scipy: Used in statistical tests and transformations.sklearn: For preprocessing data, model building, and evaluation.
The data is loaded from a CSV file. Initial examination includes checking the structure, basic statistics, and null values of the dataset.
Visual explorations are performed using histograms, count plots, and heatmaps to understand distributions and relationships in the data. Insights regarding port usage, action distributions, and correlations between features are drawn.
Data cleaning steps include handling outliers through methods such as winsorization, and encoding categorical variables. Numerical features are scaled using standard scaling techniques.
Two machine learning models are trained:
- Random Forest: A robust model suitable for handling non-linear relationships and interactions.
- Support Vector Machine (SVM): Chosen for its effectiveness in high-dimensional spaces.
Models are evaluated using confusion matrices, and performance metrics like precision, recall, and F1-score are calculated. A comparison between the models' performance is also provided.
The notebook concludes with insights gathered from model performances and suggestions for future improvements in model accuracy and processing.