Skip to content
Unblurs images. Maybe.
Python Other
  1. Python 98.7%
  2. Other 1.3%
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data
.gitignore
LICENSE
README.md
conv.py
cost.py
kern.py
plot.gnuplot
plot.sh
resize.py
tfconv_test.py
util.py
view.py

README.md

convolver

Unblurs images. Maybe.

./conv.py data/blur1.2.jpg data/k1.2.npy

This will display:

unblurred image

Starting from the input image data/blur1.2.jpg:

blurry image

And convolving it with the deblur kernel:

kernel

The convolution operation is:

output_img = tf.nn.conv2d(input_img, kernel)

(the other 1000+ lines of Python are basically overhead)

conv.py calculates output_img.

kern.py calculates kernel.

Introduction

My hope is that these tools are easy enough to install and operate, so that they can be used by people who don't necessarily have a Python background, to process images to improve them or just for fun.

Convolver has two main components:

conv.py convolves an image with a kernel. Depending on the choice of kernel, this can be a blur, an unblur(!), an edge detector, etc.

kern.py finds the convolution kernel that converts one image into another.

Given a blurry and a sharp version of the same image, kern.py will try to find either the Point Spread Function of the blur, or (depending on argument order) the inverse of the Point Spread Function which unblurs the image. It does this using gradient descent.

Installing this software

Optionally, do this in a Docker container (this is how I tested it):

docker run -it debian:buster-slim bash
apt update

Optionally, set up a virtualenv:

apt install virtualenv
virtualenv -p python3 my_env
source my_env/bin/activate

Install dependencies:

apt install python3 python3-pip python3-dev build-essential pkg-config libglib2.0-dev libgirepository1.0-dev libcairo2-dev libgtk-3-dev
pip3 install tensorflow pillow pycairo pygobject

Install convolver:

git clone https://github.com/emikulic/convolver.git
cd convolver

Run the test like at the top of this page:

./conv.py data/blur1.2.jpg data/k1.2.npy

User's Guide

conv.py

This convolves an image with a kernel and displays the result. Hit ESC to close the viewer window. Example:

./conv.py input.jpg kernel.npy

To save to a file instead of viewing the output:

./conv.py input.jpg kernel.npy -out output.png

By default, no gamma correction is applied. If your image has gamma 2.2 and you want to correct for it:

./conv.py input.jpg kernel.npy -gamma 2.2

kern.py

This finds a convolution kernel given two images, using gradient descent. Example:

./kern.py blurry.jpg sharp.jpg kernel-1

The above will try to find a kernel such that:

tf.nn.conv2d(blurry, kernel) == sharp

i.e. It will find a kernel that unblurs blurry.jpg so that it looks like sharp.jpg.

You could also pass the inputs in reverse and recover the blur kernel, but that's not as fun.

The above command will display an image of the current progress, and produce a directory kernel-1/ which contains snapshots of the kernel (as .npy files) and snapshots of the displayed image.

The directory can be fed into conv.py as a kernel.

The optimizer can be stopped by hitting ESC (or using the -max_steps argument as documented below).

Running kern.py with an existing kernel directory will resume optimizing it.

The kern.py script has a lot of commandline arguments to tune the process:

  • -n sets the size of the kernel. You probably want this to be an odd number. Larger kernels are slower and cause more artifacts at the edge of the output, but are otherwise more accurate.

  • -sym False will produce a non-symmetric kernel. If the blur was symmetric, then you can use a symmetric deblur kernel, which is the default.

  • -gamma can be used to set the input image's gamma. By default, there's no gamma correction.

  • -reg_cost is used to regularize the kernel, i.e. put pressure on the optimizer to minimize the values in the kernel. The sum of absolute kernel values is multiplied by reg_cost and added to the overall cost.

  • -border is set by default to -n divided by two. It's the number of pixels discarded from each edge when evaluating the output.

  • -learn_rate is the learning rate for the optimizer. If the optimizer is not converging, change the learning rate. If that doesn't work, change the learning rate more.

  • -epsilon is a Greek letter.

  • -max_steps - exit after this many steps. The default is to run forever. This option is useful if you're doing a parameter sweep and you want to e.g. make a shell script that trains ten kernels with different settings with a thousand steps each.

  • -log_every - log progress after this many steps of the optimizer.

  • -save_every - save a snapshot to the kernel directory after this many steps of the optimizer.

  • -crop_{x,y,w,h} - when optimizing, only consider the window starting at (x,y) with size (w,h). By default, the entire input image is used.

  • -fps - how many times per second to update the progress image. Set to zero to disable the viewer and run headless.

Utilities - User's Guide

cost.py

Can be used to assess multiple kernels over the same image pair. Takes arguments in the same order as kern.py. Example:

./cost.py blurry.jpg sharp.jpg kernel-*

plot.sh

Uses gnuplot to plot the cost (total-cost and diff-cost as two lines) against the step number, based on a kernel's log.txt file. Example:

./plot.sh kernel-1

resize.py

Resizes a kernel, either to shrink (crop) it, or to expand it, in which case the new pixels are randomized. In either direction, it's a good idea to resume training the resized kernel in order to minimize error. Example:

# Train small kernel.
./kern.py blurry.jpg sharp.jpg -n 7 kernel-1

# Expand it.
./resize.py kernel-1 kernel-2 11

# Train the bigger kernel.
./kern.py blurry.jpg sharp.jpg kernel-2

# Compare results.
./cost.py blurry.jpg sharp.jpg kernel-*

view.py

Displays one or more kernels. Zero is black, negative values are blue. Example:

./view.py kernel-1

To write to a file instead of displaying:

./view.py kernel-* -out kernels.png
You can’t perform that action at this time.