region_adjacency_graph(), merge_hierarchical() and selective search

[michaelborck]

Overview

For my use case, I needed to sample an image to provide a list of regions that may contain an object. One strategy is to use an over-segmented image, hierarchical merging and a similarity measure to produce a list of proposals. I required the ability to generate a RAG with node descriptions and edge weights that differed from the default rag_mean_color(). To achieve this I created the region_adjacency_graph() method and modified merge_heirachical(). Provided two examples. The first example is a simplified version of my use case, Selected Search. Picture of output below. The second example is how to construct a RAG using region_adjacency_graph and a user specified callback. The second example example duplicates the functionality of "RAG Thresholding" from the scikit-image gallery.

Replaces PR #1723

Description of Changes

Added method region_adjacency_graph() which allows a user defined function to describe the graph's nodes, edges and calculate the edge weights.

Modified __init__.py to include region_adjacency_graph()

Modified rag_mean_color() to call region_adjacency_graph(). Provides the same functionality as the original by using a callback defined locally within the method (i.e. not file/module scope). The API signature of rag_mean_colour() is unchanged.

Added flag to merge_hierarchical() to indicate the each step of the hierarchical merge is to be returned. merge_hierarchical() now returns a list of intermediate merge steps. The list contains only regions merged, as opposed to a previous verison which contain complete lable maps.

Added a example application which uses the list of merge steps output from the modified hierarchical_merge() to propose regions that may contain an object. The goal of the example application is a list of regions to be used by a machine learning process. To visualise some of these regions the example code plots the last 10 bounding boxes detected. Below is an exampleof the image produced by plot_selective_search.py

[vighneshbirodkar]

Why does this method have a generic name ? It is doing something very specific, ie, defining a color similarity based RAG.

[michaelborck]

The scope of the function is local to rag_mean_color(). Are you suggesting to call it define_color_based_similarity()?

[vighneshbirodkar]

Oh I see, but nested functions are generally bad for performance. Cause even though the function is doing something specific, it has to be instantiated every time you made a call to the parent function. What I would do is put this outside with a name like define_mean_color_rag or something similar.

[michaelborck]

Ok. Can move to file/module level and give it a more descriptive name.

[michaelborck]

Stylistic question. Since define_mean_color_rag is not part of the public interface, essentially a "private" function used by rag_mean_color, should it be called _define_mean_color_rag, that is, have a leading underscore?

[jni]

should it be called _define_mean_color_rag, that is, have a leading underscore?

yes. =)

[vighneshbirodkar]

@michaelborck Thank you for your efforts. It is really exciting for us to see someone using and further enhancing our contributions. Could you attach the output image from the example so that all of us can see ?

[michaelborck]

@vighneshbirodkar As requested, output image from plot_selective_search.py added to opening description of this PR.

[michaelborck]

In this example I use the rag_mean_color() and the callbacks from the RAG Merging example in the gallery. I did this thinking familiarity with the other RAG examples in the gallery and to keep the example code short. Instead, would it be better to use region_adjacency_graph(), so will need another callback? Similar to the original paper on Selective Search (referenced in the example) should I use other similarity measures, say based on colour, texture, fill and size? These will make the example longer, but potentially more useful to others, and will also need some supporting functions like histogram_intersection to calculate the weights. Thoughts?

[sciunto]

@jni @vighneshbirodkar @michaelborck What's the status of this PR?

[jdavidavendano]

Hey, @michaelborck @vighneshbirodkar @sciunto @jni @soupault If you change in rag.py in the line 324

from:

graph = region_adjacency_graph(labels, image=image,
                                   connectivity=connectivity,
                                   describe_func=_define_color_mean_rag,    # <----
                                   extra_arguments=[],
                                   extra_keywords=extra_keywords)

to:

graph = region_adjacency_graph(labels, image=image,
                                   connectivity=connectivity,
                                   describe_func=_define_mean_color_rag,    # <----
                                   extra_arguments=[],
                                   extra_keywords=extra_keywords)

(check the describe_func argument)

And in graph_merge.py in the line 146

from:

return label_map[labels], trace

to:

if merge_trace:
    return label_map[labels], trace
return label_map[labels]

Conflicts should disappear.