Skip to content

A Fast Deep Learning Model to Upsample Low Resolution Videos to High Resolution at 30fps


Notifications You must be signed in to change notification settings


Repository files navigation


The goal of this repository is to enable real time super resolution for upsampling low resolution videos. Currently, the design follows the SR-GAN architecture. For speed, the upsampling is done through pixel shuffle.

The training setup looks like the following diagram:

Speed Benchmarks

The following runtimes/fps are obtained by averaging runtimes over 800 frames. Measured on MPS (MacBook M1 Pro GPU).

Input Image Size Output Size Time (s) FPS
90x160 360x640 (360p) 0.01 82
180x320 720x1080 (720p) 0.04 27

We see it's possible to upsample to 720p at around 30fps.


This was tested on Python 3.10. To install the required packages, use the provided Pipfile:

pip install pipenv --upgrade
pipenv install --system --deploy

Pre-trained Model

A pretrained generator model on the DIV2k dataset is provided in the 'models' directory. It uses 8 residual blocks, with 64 filters in every layer of the generator.

To try out the provided pretrained model on your own images, run the following:

python --image_dir 'path/to/your/image/directory' --output_dir 'path/to/save/super/resolution/images'


To train, simply edit the config file in the folder configs/config.yaml with your settings, and then launch the training with:


You can also change the config parameters from the command line. The following will run training with a batch_size of 32, a generator with 12 residual blocks, and a path to the image directory /path/to/image/dataset.

python data.image_dir="/path/to/image/dataset" training.batch_size=32 generator.n_layers=12

This is powered by hydra, which means all the parameters in the config are editable via the CLI.

Model checkpoints and training summaries are saved in tensorboard. To monitor training progress, open up tensorboard by pointing it to the outputs directory that will be created when you start training.


Following are some results from the provided trained model. Left shows the low res image, after 4x bicubic upsampling. Middle is the output of the model. Right is the actual high resolution image.

The following shows images upsampled 4x by bicubic interpolation, the pretrained model from this repository and the original high resolution image as a comparison


If you have ideas on improving model performance, adding metrics, or any other changes, please make a pull request or open an issue. I'd be happy to accept any contributions.