Fast Marching Method for Image Inpainting

[chintak]

Tested for gray scale images. Cython implementation in _inpaint.pyx and inpaint wrapper function and initialise function implemented in python in inpaint.py.

demo_inpaint.py provides an interactive way for image inpainting. Unit tests are provided in test_inpaint.py.

TODO:

1. Add support for 3 or 4 channel image - Wait
2. Add Error checks for the dimensions - Done

[stefanv]

This should not be a separate module. Probably should go under filter?

[tonysyu]

This function as a whole is pretty difficult to understand and the variable names aren't helpful. That said, the papers I've looked at aren't terribly enlightening. Telea's 2002 paper might be a better reference here and maybe Sethian's 1996 paper (specifically section 4) would be useful, but I can't honestly say that I really follow it.

[chintak]

I agree, it is difficult. And I had read the two papers during literature survey but there always seems to be a gap in explaining how they reach the code implementation from the equation as stated using forward and backward differences. However, we know what we really need to compute.

Given the distance/time values of 2 adjacent neighbours, how do we compute the distance/time it takes to reach the pixel in question ? (Distance and time are interchangeable since the speed is taken to be 1 in a direction normal to the curve at all points.) Let's break it down a bit:

In _fast_marching_method I always pass the adjacent neighbours, in the sense:

0 A 0
B ? D
0 C 0

I'll pass A-B or A-D or B-C or C-D not A-C or B-D. So that's where the 2 comes into the picture in r computation. Essentially, since the distance between them is sqrt(2), that is the max difference between there distance/time values.

So, in what case will it be lesser ? I can imagine a curve inclined at -45 deg hitting B and C. In this case, intuitively, the values should be the same. Assume the values for both is 1. So my r would simply be sqrt(2). Now, since my s value according to L210 would be less than 1. I go to L214. And s = (u1 + u2 + r) / 2. This would be essentially be 1 / sqrt(2) times greater than u1 or u2 i.e. 1.

Now if we imagine a unit isosceles right angle triangle as:

B -- ?
 \   |
   \ |
     C

And the curve parallel to side BC, then yes the distance value given as 1 + 1/sqrt(2) for ? given 1 for B and C, does make sense.

Does this train of thought make sense ? @tonysyu @stefanv

[chintak]

Now, I like to believe that for curve inclined at various different angles, the essence of r would be to compute the perpendicular distance to this curve using the difference in u values (which depends on the direction of the curve), since the curve always travels normal to itself and with unit speed, the perpendicular distance from the curve to ? should give me the answer. I just add an offset in time to keep the u values always progressing ahead in time, which is what s does here.

[stefanv]

This type of explanation should be summarized succinctly in the docstring
so that future maintainers can follow along.

[chintak]

Alright. Can I give a link to the above comment ?

[stefanv]

No, it would be better to summarise it neatly and include it into the code.
Also, just read over the code a few times and think whether there's a way
to better express it. If there isn't, that's okay, but then we put in the
due consideration.

[tonysyu]

I'll give my +1 for merging, but I'd like for another dev to look over it before actually merging into master. The code is still a bit difficult to understand in parts, but that may just be a symptom of the algorithm.

[stefanv]

If the code is hard to read to a seasoned developer, then we need to work
on making it easier to understand. Steps we can take: think of ways to
simplify the logic (I'm often surprised at how often this is possible with
a bit of thinking), additional comments to describe intuitively what
variables represent, or, like you said, references to external sources.

[chintak]

Now, using the same train of thought, imagine a horizontal curve hitting B. Then the time/distance taken to reach ? will be 1 + time till B.

[tonysyu]

I think it would be useful to explicitly mention that this diverges from Telea's implementation, which is what's referenced.

[stefanv]

Agreed. /ping @chintak

[chintak]

Noted.

[tonysyu]

Other than a couple of minor comments, I'm fine with merging. But as before, I'd like someone else to have a read through, since I've looked at the code so many times that I'm probably too familiar with the implementation to be unbiased.

[stefanv]

No explanation of the meaning of "u" (this is the Eikonal function? Err, no, I guess not. Gradient?)

[coveralls]

Changes Unknown when pulling e3625e5 on chintak:inpaint into * on scikit-image:master*.

[alexdesiqueira]

Closed until further notice. Feel free to reopen if you would like to continue working on this; please check also #1053.