imageAttackDetectionAndDenoising

A diffusion-based denoising approach to mitigate online adversarial image attacks and an FFT-based detector

How to use this code

This code has been tested on Ubuntu 18.04 LTS and ROS melodic with Python 3.6.9 and Python 3.10.0 and GCC 8.4.0.

Open up 3 terminals

Create a catkin workspace and download the repositories as follows

# in terminal 1

mkdir -p ~/defender_ws/src && cd ~/defender_ws/src

git clone https://github.com/r-bahrami/iros_image_attack.git
git clone https://github.com/gaudelbijay/attack-defender.git
git clone https://github.com/ros-perception/vision_opencv -b melodic

cd ..

The attack-defender model requires Python 3.10.0. As such, set up a virtual environment for installing the requirements in requirements.txt.

# in terminal 2

python3.10 -m venv diffusion_env
source diffusion_env/bin/activate
cd <path to defender>/src/attack-defender
pip install -r requirements.txt

cd src/diffusion_model/ && mkdir results

Download the trained parameters of diffusion-based denoiser, named model-100.pt from this link and copy into the results folder.

Build and train the RL-based attacker model

# in terminal 1
# cd ~/defender_ws/

catkin_make -DPYTHON_EXECUTABLE=/usr/bin/python3

source  devel/setup.bash

Set up the AirSim and simulation environment settings (Environment 2) as described in iros_image_attack package.

For integration of the attack-defender, in model_yolo.py, uncomment the line 71 and comment line 70. This enforces the simulation to use the denoised images, and thus the RL-baed attacker is trained while the attack-defender is in the loop.

Run the ./Blocks.sh environment in terminal 3.

Run the attack-defender,

# in terminal 2
# source diffusion_env/bin/activate

cd ~/defender_ws/src/attack-defender/src/diffusion_model

python node_denoising.py

Then train the attacker using

# in terminal 1
roslaunch iros_image_attack train.launch

After 10 episodes, the code will stop and the trained model will be saved. Afterward, different experiments can be run as described below.

runtime

# in terminal 2
# source diffusion_env/bin/activate
# cd ~/defender_ws/src/attack-defender/src/diffusion_model

python node_denoising.py

# in terminal 1
roslaunch iros_image_attack run.launch

TO-DO

• compare the code with this commit to figure out the encoding bug.

Integration with attack models.

• The reinforcement-based attacker of the RAL-paper based on the forked iros_image_attack repository.

To integrate attack-defender model with the RL-based attacker in iros_image_attack, we only added one line in the node_yolo.py module to subscribe to the denoised images. Also, we enforced the image encoding encoding="bgr8" for the published attacked images in the node_image_attacker.py module.

For integration to take place, in model_yolo.py, uncomment the line 71 and comment line 70. This enables the simulation to run using the denoised images.

• integrate the FFT code for online attack detection.
• update the FFT code for online attack detection in 2D space of Height x Width where we have a better parameterized threshold for detection.
• add the training details

Acknowledgement

We used denoising-diffusion-pytorch repo for the implementation of the training of our diffusion model of the attack-defender.

We used iros_image_attack repo for the reinforcement-based attacker.