restoration.wiener doc example gives poor result

[elena-pascal]

Description

I had a look at the Example code for Wiener-Hunt deconvolution and the result I get looks blurrier than the image it starts with. Is this the expected behavior? I played around with the balance parameter but it didn't help. I am trying to understand if this is the deconvolution that would be applicable to my problem.

Way to reproduce

from skimage import color, data, restoration
img = color.rgb2gray(data.astronaut())
from scipy.signal import convolve2d
psf = np.ones((5, 5)) / 25
conv = convolve2d(img, psf, 'same')
rng = np.random.default_rng()
noised = conv + 0.1 * conv.std() * rng.standard_normal(conv.shape)
deconvolved_img = restoration.wiener(noised, psf, 1100)


fig, ax = plt.subplots(nrows=1, ncols=4, figsize=(8, 6))
plt.gray()


ax[0].imshow(img)
ax[0].set_title('Original')
ax[0].axis('off')

ax[1].imshow(conv)
ax[1].set_title('Convoluted')
ax[1].axis('off')

ax[2].imshow(noised)
ax[2].set_title('Noised')
ax[2].axis('off')

ax[3].imshow(deconvolved_img)
ax[3].set_title('Restoration using Wiener')
ax[3].axis('off')

fig.subplots_adjust(wspace=0.02, hspace=0.2,
                    top=0.9, bottom=0.05, left=0, right=1)
plt.show()

Version information

3.9.7 (default, Sep 16 2021, 13:09:58) 
[GCC 7.5.0]
Linux-3.10.0-1160.36.2.el7.x86_64-x86_64-with-glibc2.17
scikit-image version: 0.18.1
numpy version: 1.21.2

[grlee77]

Thanks for reporting this @elena-pascal. I can reproduce the same issue on the current development branch as well.

I think the problem is in the value of 1100 in this line:

deconvolved_img = restoration.wiener(noised, psf, 1100)

I think at some point this function must have changed how the data was rescaled internally and the value of 1100 for balance here is no longer reasonable. If this is changed to a much smaller value of around 1.0 it looks much less blurred. Will have to investigate a bit to see how far back this change occurred and what the equivalent value is for the current implementation.

[elena-pascal]

Oh I see. The test is also written for a balance value of 0.05 at that indeed looks right. It makes sense why I didn't have any luck playing around with that parameter, I didn't go low enough. How are the optimal values for balance determined btw?
I'm happy to try and update the docs with the 0.05 value and perhaps output image if the consistency search agrees with this value.

[mkcor]

Dear @elena-pascal (and @grlee77 😉),

Looking at the history, I don't think that balance=1100 in deconvolved_img = restoration.wiener(img, psf, 1100) was ever suitable with img deriving from color.rgb2gray(data.astronaut()). Maybe this value once worked with data.lena() (see 1b217f9#diff-87be2b5ca31afe4ca1d683cb77680447282dc3a265e6620d4661669a9b546b4a)? I wouldn't even think so, since converting the image with color.rgb2gray would bring its values in the 0 to 1 range.

I guess this was simply never caught because there is neither a visual check (as in plotting, like you've done) nor a quantitative test (as in test_wiener, that you've linked to).

Indeed, the only recent changes that this function and its docstring have undergone recently (a few months ago vs. 7 or 8 years ago) are:

• the update of numpy.random.x and
• the addition of single-precision float type support.

Neither change can account for a value of balance=1100 which would work before and not anymore.

So, I would say, please go ahead and update the docstring! To determine an optimal balance value, maybe you could run the newly implemented blur metric on the output, and pick the best? This should also match what you're perceiving with your eyes... 👀 🙂

Thanks again for reporting!

PS: This link https://pro.orieux.fr/files/papers/OGR-JOSA10.pdf (provided in the references) is now broken. 🙁

[elena-pascal]

Some time passed. Sorry to reopen this, I finally got around to do the recommended changes to the documentation.

Thank you @mkcor for detailed response. I did have a look at the blur metric tool and it does indeed quantify what the eye can see. Unfortunately, for looking at the balance value in deconvolution is not entirely useful, unless I didn't understand how to use it, since with smaller and smaller balance values the blur metric gets infinitely smaller and smaller. Which makes sense. At some point the deconvolution edge effects are becoming worse than the deblur we get and that is not accounted in the blur metric. That is to say I ended up eyeballing a value of 0.1 for the balance in the example since it looked the best compromise to me.

[mkcor]

Dear @elena-pascal,

Thank you so much for working on this! Eyeballing a balance value of 0.1 looks reasonable; for future reference:

import matplotlib.pyplot as plt
import numpy as np
from scipy.signal import convolve2d

from skimage import color, data, measure, restoration


img = color.rgb2gray(data.astronaut())

psf = np.ones((5, 5)) / 25
conv = convolve2d(img, psf, 'same')
rng = np.random.default_rng()
noised = conv + 0.1 * conv.std() * rng.standard_normal(conv.shape)

# Try different balance values for image restoration using Wiener-Hunt deconvolution
balance_values = [1100, 1, 0.1, 0.01, 0.001]
restored_images = [restoration.wiener(noised, psf, balance=b) for b in balance_values]

fig, ax = plt.subplots(nrows=2, ncols=3, figsize=(8, 8))
plt.gray()

ax[0, 0].imshow(img)
ax[0, 0].set_title('Original')
ax[0, 0].axis('off')

ax[0, 1].imshow(restored_images[0])
ax[0, 1].set_title('Restoration w/ balance = 1100')
ax[0, 1].axis('off')

ax[0, 2].imshow(restored_images[1])
ax[0, 2].set_title('Restoration w/ balance = 1')
ax[0, 2].axis('off')

ax[1, 0].imshow(restored_images[2])
ax[1, 0].set_title('Restoration w/ balance = 0.1')
ax[1, 0].axis('off')

ax[1, 1].imshow(restored_images[3])
ax[1, 1].set_title('Restoration w/ balance = 0.01')
ax[1, 1].axis('off')

ax[1, 2].imshow(restored_images[4])
ax[1, 2].set_title('Restoration w/ balance = 0.001')
ax[1, 2].axis('off')

fig.subplots_adjust(wspace=0.02, hspace=0.01,
                    top=0.9, bottom=0.05, left=0, right=1)
plt.show()

The blur metric measures the blur effect (or blur annoyance) which is associated with a loss of details (i.e., loss of high-frequency content). The lower the balance value, the more high-frequency content, which looks 'sharper' to the blur metric... It makes sense, indeed, but clearly we are not moving towards better and better restoration quality as the blur metric simply tends to 0.

[measure.blur_effect(im) for im in restored_images]

outputs this sequence of decreasing values for the blur metric:

[0.9271564699373505,
 0.5667472924590226,
 0.4533045608494091,
 0.32612928916395323,
 0.2085498573057157]

So my suggestion about "determining an optimal balance value" doesn't work in an absolute sense... Not giving up on using the blur metric as "a means to compare the quality of restoration methods or scaling methods," I checked its value for the original image:

measure.blur_effect(img)

which turns out to be

0.39957512013087865

so that's how 'blurry' it is to begin with. The restored image with balance = 0.1 has (among the five balance_values under consideration) the blur metric value (i.e., 0.4533...) closest to that of the original image. So maybe that's a way of finding the 'optimum' in this case?

Let me poke @ladretp who authored this blur metric and might have some feedback for us...

I'll review your PR now. Thanks again!

[elena-pascal]

Thank you for documenting what I was trying to explain. I do like your idea of how to use the blur metric as a way to find an optimal value that is not necessary the smallest value.

[ladretp]

Hello Le 27/02/2022 à 13:14, Marianne Corvellec a écrit :

The blur metric measures the blur effect (or blur annoyance) which is associated with a loss of details (i.e., loss of high-frequency content). The lower the balance value, the more high-frequency content, which looks 'sharper' to the blur metric... It makes sense, indeed, but clearly we are not moving towards better and better restoration quality as the blur metric simply tends to 0.

Yes it's clear that blur metric cannot be the only criterion for the quality of an image restoration, as bounce effects are often much more annoying than blur from a perceptual point of view. I don't think that this measure can be used to find the right parameters for a restoration algorithm, or possibly combined with other criteria. Patricia

[mkcor]

Dear @ladretp,

Thank you for your precious feedback!

Sure, in real-world problems, we don't have the ground truth...

[Ir1d]

It is unchanged on this page: https://scikit-image.org/docs/stable/api/skimage.restoration.html#wiener