Generative Adversarial Super-Resolution at the Edge with Knowledge Distillation
LPIPS results (lower is better) on Set5 vs. framerate (80 × 60 input) of different visual-oriented SISR methods for upsampling. Real-time (RT) and over-real-time (ORT) framerates are marked as references. Our models, marked with ★, reach real-time performance with a competitive perceptual similarity index on the CPU. Edge TPU models can further increase inference speed far beyond real-time, still outperforming the bicubic baseline.
Single-Image Super-Resolution can support robotic tasks in environments where a reliable visual stream is required to monitor the mission, handle teleoperation or study relevant visual details. In this work, we propose an efficient Generative Adversarial Network model for real-time Super-Resolution, called EdgeSRGAN. We adopt a tailored architecture of the original SRGAN and model quantization to boost the execution on CPU and Edge TPU devices, achieving up to 200 fps inference. We further optimize our model by distilling its knowledge to a smaller version of the network and obtain remarkable improvements compared to the standard training approach. Our experiments show that our fast and lightweight model preserves considerably satisfying image quality compared to heavier state-of-the-art models. Finally, we conduct experiments on image transmission with bandwidth degradation to highlight the advantages of the proposed system for mobile robotic applications.
EdgeSRGAN distillation process.
This repository allows to train and test EdgeSRGAN on different Single Image Super-Resolution datasets using adversarial training combined with feature-wise Knowledge Distillation.
We suggest to use a virtual environment (conda, venv, ...)
git clone git@github.com:PIC4SeR/EdgeSRGAN.git
pip install -r requirements.txt
To set all the training/testing parameters modify the configuration file:
nano config.yaml
Run the following command to train EdgeSRGAN on DIV2K:
python main.py
Run the following command to test EdgeSRGAN on all the included SISR test datasets:
sh test.sh
Pretrained weights are available at this link.
edgesrgan.h5: float32 TF weightsedgesrgan.tflite: float16 TFLite model for inference on CPUedgesrgan_int8.tflite: int8 TFLite model for inference on Coral EdgeTPU
Visual comparison of bicubic image SR (x4) methods on random samples from the considered datasets. EdgeSRGAN achieves results that are comparable to state-of-the-art solutions with ~10% of the weights.
This repository is intended for research scopes. If you use it for your research, please cite our paper using the following BibTeX:
@article{angarano2023generative,
title = {Generative Adversarial Super-Resolution at the edge with knowledge distillation},
journal = {Engineering Applications of Artificial Intelligence},
volume = {123},
pages = {106407},
year = {2023},
issn = {0952-1976},
author = {Simone Angarano and Francesco Salvetti and Mauro Martini and Marcello Chiaberge}}
We would like to thank the Interdepartmental Center for Service Robotics PIC4SeR, Politecnico di Torino.