Skip to content

Detection of network traffic anomalies using unsupervised machine learning

Notifications You must be signed in to change notification settings

kaiyoo/ML-Anomaly-Detection

Repository files navigation

[1] Overview

This project was done in the subject, COMP90073 (Security Analytics) taken in Semester2, 2020 in the University of Melbourne.

  1. https://cloudstor.aarnet.edu.au/plus/s/Hvu7YyCDDG7ByWb
  2. https://cloudstor.aarnet.edu.au/plus/s/38CH3I8HbuYkh3r

[2] Features

More details in the anomaly_detection_reports.pdf

  • Feature1: Numeric value (existing + newly generated) + Standardscaler + PCA

  • Feature2: Feature1 + One-hot encoded categorical feature

  • Feature3: Scale (Cumulative features grouped by stream_id + time-based feature) + PCA

[3] Model

  1. Iforest
  2. OneclassSVM

[4] Hyperparameter tuning (2 examples among 6)

  • Criteria of setting a threshold: Accuracy > 0.88 and Max(TPR-FPR)
  1. OCSVM + feature3

alt text

  1. Iforest + feature 3

alt text

[5] Clustering visualisation and Evaluation (2 examples among 6)

  1. OCSVM + feature3

SCORES:

alt text

CLUSTERING:

alt text

  1. Iforest + feature3

SCORES:

alt text

CLUSTERING:

alt text

[6] Interpretation of the result

alt text alt text

  • Attack Timeline alt text

[7] Generating Adversarial samples (FGSM)

  • FGSM generates adversarial samples with the error rate of almost 100%.

alt text