Add image enhancement measure (EME) #3776

iliailmer · 2019-03-01T01:48:31Z

Description

This PR contains implementation of an image quality measure called EME, based on this paper.

It is based no the human visual system, as mentioned in the paper, and it can be used as a metric for enhancement algorithms to see if they work or not.

Checklist

Docstrings for all functions
Gallery example in ./doc/examples (new features only)
Benchmark in ./benchmarks, if your changes aren't covered by an
existing benchmark
Unit tests
Clean style in the spirit of PEP8

For reviewers

Check that the PR title is short, concise, and will make sense 1 year
later.
Check that new functions are imported in corresponding __init__.py.
Check that new features, API changes, and deprecations are mentioned in
doc/release/release_dev.rst.
Consider backporting the PR with @meeseeksdev backport to v0.14.x

pep8speaks · 2019-03-01T01:48:33Z

Hello @iliailmer! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2019-11-11 18:26:51 UTC

grlee77 · 2019-03-01T14:43:57Z

Hi @iliailmer,

Thank you for the contribution! I have not yet done a detailed review of the publication, but it has a substantial number of citations and the implementation looks simple and should be easy to maintain going forward. I think this is a reasonable candidate for inclusion in scikit-image.

I can do a more thorough review over the coming week, but a few initial comments:

1.) The function should not be stored in the top-level skimage directory. A more appropriate location would be in the metrics submodule, perhaps as an addition to the current simple_metrics.py

2.) Use a more descriptive name for the function than quality. Perhaps enhancement, enhancement_measure or something similarly descriptive. For example the existing functions compare_psnr and compare_ssim are measures of image similarity often referred to as image quality metrics. Having PSNR or SSIM in the function name gives some immediate indication of what "quality" is being measured.

3.) It looks like this can be trivially extended to the n-dimensional case rather than just 2d and 3d. We can help with this if you have questions on how to approach it. A convention in the library is to include a multichannel argument so that the user can indicate whether the final axis of an array is a "spatial" axis or whether it represents channels (e.g R, G, B channels for a color image, for example).

…re/simple_metrics.py, added footprint parameter for n-dimensional filtering

iliailmer · 2019-03-01T18:05:27Z

Hi @grlee77 , thanks for your comments!

Regarding the multidimensional extension, I realize now that the max/min filters used here are, by default, usable with any number of dimensions, so one does not need to check the length of the array. In this case the multidimensional parameter may not be required, and the filter shape (footprint) will be inferred automatically.

skimage/image_quality_measure.py

lagru · 2019-03-06T20:16:48Z

Looks promising! I'll have a more thorough look later.

lagru

Thanks for addressing @grlee77's feedback. Good work so far! With the recent changes this should work for the n-dimensional case, perhaps we could mention that in the docstring. I think you forgot to remove the now obsolete file image_quality_measure. Could you fix that?

Otherwise I don't yet fully understand your implementation when looking at the linked paper (see comment). Do you or someone else know a source for the clock image used in the paper? It would be useful to compare results of this function to the paper.

skimage/measure/simple_metrics.py

skimage/image_quality_measure.py

lagru · 2019-03-17T11:11:56Z

@iliailmer

I think you forgot to remove the now obsolete file image_quality_measure. Could you fix that?

I'm getting confused now. The diff on GitHub shows that this PR still includes 2 redundant files with nearly the same code. Could you ensure that the redundant files are removed? We only want to keep the changes in skimage/measure/simple_metrics.py as suggested by @grlee77.

skimage/measure/simple_metrics.py

…ties

lagru

@iliailmer thanks for addressing my feedback. The solution with eps as an optional parameter is much better now but I'm not sure whether this is the right way to solve this. As I understand the paper the formula is meant to quantify a contrast improvement somehow. While rescaling the image won't affect the Weber contrast I'm unsure about this measurement.

Is there a reason to rescale between [0, 1] in particular? If we were to chose a value above zero as the lower bound we could omit the eps parameter altogether.

skimage/measure/simple_metrics.py

emmanuelle · 2019-04-03T10:01:27Z

Hi @iliailmer thanks for your PR. It looks good, but we're missing unit tests (write unit tests on small images, 2D and 3D, with int and float types for example) as well as a gallery example.

iliailmer · 2019-04-05T18:59:09Z

@emmanuelle Thank you, I will get to it as soon as I can. Should the test images come from the data module of scikit-image?

emmanuelle · 2019-04-05T21:17:46Z

@iliailmer yes, either images from skimage.data (you can downscale them for faster execution, or choose a ROI, since we want tests to execute fast enough) or synthetic data which you create yourself. You can take a look at the different tests folders in each submodule of scikit-image, to get an idea of how tests are written.

iliailmer · 2019-11-08T19:49:35Z

I keep getting
Warning, treated as error: C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\site-packages\skimage\measure\simple_metrics.py:docstring of skimage.measure.compare_mse:23:Explicit markup ends without a blank line; unexpected unindent.
mingw32-make[1]: *** [Makefile:53: html] Error 2
mingw32-make[1]: Leaving directory 'd:/a/1/s/doc'
mingw32-make: *** [Makefile:27: html] Error 2

Tried to fix docstring couple of times already, not sure where the unindent is happening

rfezzani · 2019-11-09T08:33:57Z

Tried to fix docstring couple of times already, not sure where the unindent is happening

Have you tried the solution proposed here?

iliailmer · 2019-11-10T00:50:08Z

@rfezzani Thank you, it worked.

sciunto · 2019-11-10T07:48:56Z

skimage/measure/simple_metrics.py

+            "A new measure of image enhancement.",
+            IASTED International Conference on Signal Processing
+            & Communication, Citeseer, 2000,
+            :DOI:10.1.1.35.4021


Suggested change

:DOI:10.1.1.35.4021

:DOI:`10.1.1.35.4021`

The DOI is incorrect. Also, backquotes must be set as suggested.

Hi @sciunto, thanks for pointing that our!
I was having troubles with the DOI though.

The only source that lists the DOI is this link, and I doubt that there is a better version or at least i couldn't find one. The doi.org says this is only a prefix, but google search for doi=10.1.1.35.4021 returns the link above no problem. The original journal is not available (see here) so I am not sure what is the best way to prcoceed here.

I think doi.org must be the reference. If no working doi can be found, it is better to do not display a doi. Eventually, a link to the pdf file can be added instead.

I removed it for now.

doc/examples/filters/plot_enhancement_measure.py

iliailmer · 2020-02-07T01:27:00Z

Thanks for your contribution! I've added a few small suggestions.

Hi @stefanv, could you please take a look and check if anything is amiss? All the checks are passed so far. Thanks!

lagru · 2022-12-21T11:36:09Z

Hey @iliailmer, sorry for the slow review process. I still want to get this into scikit-image! There are some merge conflicts now, do you want to resolve these or shall I? Afterwards I can do another review and hopefully approve to get this going again. 🙏

lagru · 2023-01-06T13:35:19Z

Hey, I took the liberty to merge this which was a little complicated due to us moving simple_metrics from skimage/measure to skimage/metrics on main. I hope I got every relevant change pertaining to adding the new function enhancement_measure.

simple_metrics was moved from skimage/measure to skimage/metrics making it quite complicated to preserve the changes during the merge. This commit re-adds the original changes authored by iliailmer <i.ilmer@icloud.com> back. Co-authored-by: iliailmer <i.ilmer@icloud.com>

Type hints are not (yet) used within scikit-image. Within the docstring I tried to describe the measure in more precise terms. However, the paper seems a bit vague on the interpretation of this measure.

lagru

I took the liberty and tried to make the docstring use a more precise description of the EME. However, I'm still a bit vague on how to EME should be interpreted precisely and what kind of "enhancement" it measures. Does it purely measure contrast enhancement? To me that seems to me where we can gain the most value yet.

Otherwise I suggested some smaller but easier changes that should probably be addressed.

lagru · 2023-01-07T10:10:04Z

skimage/metrics/simple_metrics.py

+    >>> before = enhancement_measure(img)
+    >>> after = enhancement_measure(equalize_hist(img))
+    >>> before < after
+    True


Other measure functions in this module are applied on and compare two images. As far as I understand, the EME of an image is not useful on its own but rather it's difference to the EME of an improved image. It might be worth thinking about making the function take two images and return a bool or difference between the EME of both images. That way we also prevent users from assigning to much meaning to the absolute value of the measure.

lagru · 2023-01-07T10:15:44Z

doc/examples/filters/plot_enhancement_measure.py

+estimate of image quality, useful when comparing image enhancement algorithms.
+
+"""
+import numpy as np


Suggested change

import numpy as np

Not used.

lagru · 2023-01-07T10:17:06Z

doc/examples/filters/plot_enhancement_measure.py

+# We will create a function that takes an original image as `before`
+# and applies a `transform` to it. We will use a gaussian filter to


Nitpick: the function doesn't actually apply a transform to it. It just calls enhancement_measure on both and plots the results. Rewording this might be a lot clearer to new users.

lagru · 2023-01-07T10:36:00Z

skimage/metrics/simple_metrics.py

+    ----------
+    image : array
+        Input image of which the quality should be assessed.
+        Can be either 3-channel RGB or 1-channel grayscale.


We should probably support only grayscale images as input to make sure that the user determines how the total luminance of a colored image is calculated. Currently, the way that maximum_filter and minimum_filter is combining color channels is not something that is covered by the paper.

lagru · 2023-01-07T10:43:00Z

skimage/metrics/simple_metrics.py

@@ -259,3 +263,51 @@ def normalized_mutual_information(image0, image1, *, bins=100):
    H01 = entropy(np.reshape(hist, -1))

    return (H0 + H1) / H01
+
+
+def enhancement_measure(image, size=3, eps=1e-6):


Is there some reasoning between choosing eps=1e-6 as the default? Is something else more reasonable?

lagru · 2023-01-07T10:44:20Z

doc/examples/filters/plot_enhancement_measure.py

+
+
+def compare_side_by_side(before, after):
+    fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(16, 14))


Suggested change

fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(16, 14))

fig, ax = plt.subplots(nrows=1, ncols=2)

I think removing the forced figure size leads to a nicer layout. Otherwise the images seem very small.

lagru · 2023-01-07T10:47:10Z

skimage/metrics/tests/test_simple_metrics.py

+    assert_less(enhancement_measure(orig, size=5),
+                enhancement_measure(enhanced, size=5))
+    assert_less(enhancement_measure(orig, size=11),
+                enhancement_measure(enhanced, size=11))


We should probably check the actual values of the EMEs and that they are within an expected range. That way our tests would inform us if some change leads to significantly different EMEs.

lagru · 2024-01-11T14:36:55Z

Hey @iliailmer, would you be able to look into the feedback? This seems like something that would be cool to have in scikit-image. It's also totally fine if you don't have the bandwidth in which case maybe someone else could take over. :)

iliailmer added 3 commits February 27, 2019 13:32

Proposition to add the following image quality assessment function

8bec6b0

Proposition to add the following image quality assessment function

d2b7135

modified examples

130d912

iliailmer added 2 commits February 28, 2019 20:52

pep8 corrections

9bfe66e

pep8 corrections

a798d36

changed name to 'enhancement_measure', placed the funciton into measu…

4c3fa15

…re/simple_metrics.py, added footprint parameter for n-dimensional filtering

some pep8 corrections

6f02d9b

sciunto added type: new feature 🧐 Needs review labels Mar 5, 2019

sciunto added this to the 0.16 milestone Mar 5, 2019

lagru reviewed Mar 6, 2019

View reviewed changes

skimage/image_quality_measure.py Outdated Show resolved Hide resolved

lagru reviewed Mar 7, 2019

View reviewed changes

skimage/measure/simple_metrics.py Outdated Show resolved Hide resolved

skimage/measure/simple_metrics.py Outdated Show resolved Hide resolved

skimage/measure/simple_metrics.py Outdated Show resolved Hide resolved

skimage/image_quality_measure.py Outdated Show resolved Hide resolved

iliailmer added 2 commits March 8, 2019 21:14

change to examples

f9c5b69

added name to __all__, fixed examples"

d266205

lagru reviewed Mar 17, 2019

View reviewed changes

skimage/measure/simple_metrics.py Outdated Show resolved Hide resolved

iliailmer added 4 commits March 27, 2019 12:20

removed unnecessary files

651d332

filter parameter is used instead of

268d0e2

added epsilon instead of +1 to avoid division by zero.

ac8e9ab

added intensity rescaling to account for images with negative intensi…

4fd9f8d

…ties

lagru reviewed Apr 1, 2019

View reviewed changes

doctest fixes

86e0bed

added unit tests and fixed some numerial stability issues

f41fa48

docstring fix: blank line

fd61c0c

docstring fix

3d7dd31

sciunto reviewed Nov 10, 2019

View reviewed changes

doc/examples/filters/plot_enhancement_measure.py Outdated Show resolved Hide resolved

iliailmer added 2 commits November 10, 2019 08:39

doi backquotes

208c641

removed doi

de9345a

Base automatically changed from master to main February 18, 2021 18:23

mkcor mentioned this pull request Feb 22, 2021

2021's calendar of community management #5169

Closed

grlee77 added 🙏 Feature request and removed type: new feature labels Feb 22, 2022

mkcor mentioned this pull request Feb 28, 2022

2022's calendar of community management #6165

Closed

jarrodmillman modified the milestones: 0.17, 0.20 Jun 4, 2022

lagru added 👶 type: New feature and removed 🙏 Feature request labels Dec 21, 2022

lagru changed the title ~~Image quality measure~~ Add image enhancement measure (EME) Dec 21, 2022

Merge branch 'main' into pr3776-image-quality

ed4f7cf

lagru force-pushed the image-quality branch from ad5fa65 to 73a95e9 Compare January 6, 2023 17:21

lagru added 2 commits January 7, 2023 11:26

Remove type hints and improve docstring

0872877

Type hints are not (yet) used within scikit-image. Within the docstring I tried to describe the measure in more precise terms. However, the paper seems a bit vague on the interpretation of this measure.

Add a link to the reference paper

8fcdc8c

lagru reviewed Jan 7, 2023

View reviewed changes

stefanv removed this from the 0.20 milestone Jan 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add image enhancement measure (EME) #3776

Add image enhancement measure (EME) #3776

iliailmer commented Mar 1, 2019 •

edited

pep8speaks commented Mar 1, 2019 •

edited

grlee77 commented Mar 1, 2019

iliailmer commented Mar 1, 2019

lagru commented Mar 6, 2019

lagru left a comment

lagru commented Mar 17, 2019

lagru left a comment

emmanuelle commented Apr 3, 2019

iliailmer commented Apr 5, 2019

emmanuelle commented Apr 5, 2019

iliailmer commented Nov 8, 2019

rfezzani commented Nov 9, 2019

iliailmer commented Nov 10, 2019

sciunto Nov 10, 2019

sciunto Nov 10, 2019

iliailmer Nov 10, 2019

sciunto Nov 11, 2019

iliailmer Nov 11, 2019

iliailmer commented Feb 7, 2020

lagru commented Dec 21, 2022

lagru commented Jan 6, 2023

lagru left a comment

lagru Jan 7, 2023

lagru Jan 7, 2023

lagru Jan 7, 2023

lagru Jan 7, 2023

lagru Jan 7, 2023

lagru Jan 7, 2023

lagru Jan 7, 2023

lagru commented Jan 11, 2024

		# We will create a function that takes an original image as `before`
		# and applies a `transform` to it. We will use a gaussian filter to



		def compare_side_by_side(before, after):
		fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(16, 14))

	fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(16, 14))
	fig, ax = plt.subplots(nrows=1, ncols=2)

Add image enhancement measure (EME) #3776

Are you sure you want to change the base?

Add image enhancement measure (EME) #3776

Conversation

iliailmer commented Mar 1, 2019 • edited

Description

Checklist

For reviewers

pep8speaks commented Mar 1, 2019 • edited

Comment last updated at 2019-11-11 18:26:51 UTC

grlee77 commented Mar 1, 2019

iliailmer commented Mar 1, 2019

lagru commented Mar 6, 2019

lagru left a comment

Choose a reason for hiding this comment

lagru commented Mar 17, 2019

lagru left a comment

Choose a reason for hiding this comment

emmanuelle commented Apr 3, 2019

iliailmer commented Apr 5, 2019

emmanuelle commented Apr 5, 2019

iliailmer commented Nov 8, 2019

rfezzani commented Nov 9, 2019

iliailmer commented Nov 10, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iliailmer commented Feb 7, 2020

lagru commented Dec 21, 2022

lagru commented Jan 6, 2023

lagru left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lagru commented Jan 11, 2024

iliailmer commented Mar 1, 2019 •

edited

pep8speaks commented Mar 1, 2019 •

edited