Adversarial Attack & Defense Evaluation System for CNN Models (CIFAR-10)

This project implements an Adversarial Attack and Defense Evaluation System for Convolutional Neural Network (CNN) models trained on the CIFAR-10 dataset. It allows for training various CNN architectures, generating adversarial examples using common attack methods, evaluating model performance under attack, and exploring basic defense mechanisms.

Features

1. Model Training: Train popular CNN models (ResNet18, VGG16, MobileNet) on the CIFAR-10 dataset.
2. Adversarial Attack Generation: Generate adversarial examples using Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) attacks.
3. Performance Evaluation: Evaluate the accuracy drop of models when subjected to adversarial attacks.
4. Defense Mechanisms: Implement and evaluate simple defense strategies (e.g., Adversarial Training, Preprocessing).
5. Streamlit Demo: An interactive web application to demonstrate adversarial attacks on uploaded images.

Project Structure

adversarial_project/
├── data/
│   └── (auto-downloaded CIFAR10 dataset)
├── models/
│   ├── resnet.py
│   ├── vgg.py
│   └── mobilenet.py
├── attacks/
│   ├── fgsm.py
│   └── pgd.py
├── defenses/
│   ├── adversarial_training.py
│   └── preprocessing.py
├── utils/
│   ├── metrics.py
│   └── dataset.py
├── train.py
├── evaluate.py
├── config.py
├── main.py
├── app.py
└── requirements.txt

Setup and Installation

1. Clone the repository (if applicable):

   git clone <your-repo-url>
   cd adversarial_project

2. Create a virtual environment (recommended):

   python -m venv venv
   source venv/bin/activate  # On Windows use `venv\Scripts\activate`

3. Install dependencies:

   pip install -r requirements.txt

Usage

1. Configure the Project

Edit config.py to set parameters such as DEVICE (cuda/cpu), BATCH_SIZE, EPOCHS, LR, EPSILON for FGSM, PGD_STEPS, PGD_ALPHA, and MODEL_NAME (resnet, vgg, or mobilenet).

2. Train and Evaluate Models

Run main.py to train a model, evaluate its clean accuracy, and then evaluate its performance under FGSM and PGD attacks. This script will also save the trained model weights to model.pth, which is required for the Streamlit application.

python main.py

3. Run the Streamlit Application

After running main.py and saving model.pth, you can launch the interactive Streamlit demo:

streamlit run app.py

Open the URL provided by Streamlit in your web browser to interact with the application. You can upload an image, select an attack type, and adjust the epsilon value to see the adversarial examples and their impact on model predictions.

Files Overview

• config.py: Global configuration settings for training, attacks, and model selection.
• main.py: Orchestrates model training, evaluation, and saves the trained model.
• train.py: Contains the training loop for the models.
• evaluate.py: Handles the evaluation of models against adversarial attacks.
• app.py: The Streamlit web application for interactive demonstrations.
• models/: Directory containing definitions for different CNN architectures.
• attacks/: Directory containing implementations of adversarial attack methods.
• defenses/: Directory for adversarial defense mechanisms.
• utils/: Utility functions, including dataset loading and metric calculation.
• requirements.txt: Lists all Python dependencies.