Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(Keras images) Add an optional image argument, and other improvements #329

Merged
merged 21 commits into from Aug 10, 2019
Merged
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
c4eb039
(keras) Make image argument required
teabolt Aug 5, 2019
1111105
Update README.md to include keras
teabolt Aug 6, 2019
7d6d982
Merge branch 'master' of https://github.com/teabolt/eli5 into keras-g…
teabolt Aug 6, 2019
14452c3
Remove mentions of target_names (not implemented)
teabolt Aug 7, 2019
9e85021
Add dispatch function and image implementation
teabolt Aug 7, 2019
9a0cd53
Update dispatcher and image function docstrings
teabolt Aug 7, 2019
6b002a6
Automatically check if model/input is image-based. Convert input to a…
teabolt Aug 7, 2019
7d82c30
Mock keras.preprocessing.image in docs conf (CI fix)
teabolt Aug 7, 2019
d1af643
Update tests, docs, tutorial with image argument changes
teabolt Aug 7, 2019
6aec486
Blank line between header and list in docstring (CI fix)
teabolt Aug 8, 2019
aaa83a7
Test keras not supported function
teabolt Aug 8, 2019
47b03c1
Docstring typo
teabolt Aug 8, 2019
df25038
Clarify "not supported" error message.
teabolt Aug 8, 2019
6234aaa
Remove TODO: explain Grad-CAM in docstring. (Will be explained in ker…
teabolt Aug 8, 2019
13e1847
Move image extraction call from dispatcher to image function
teabolt Aug 8, 2019
3e875bb
Move Keras to second place in supported package list
teabolt Aug 8, 2019
4192939
Remove warnings for 'maybe image' dispatch and conversion to RGBA
teabolt Aug 8, 2019
2bb7ba5
'not supported' error typo
teabolt Aug 8, 2019
991159b
Test 'maybe image' check with both input and model
teabolt Aug 9, 2019
10921be
Add Grad-CAM image to README
teabolt Aug 9, 2019
0cf31fe
Remove line breaking backslash from docstring
teabolt Aug 10, 2019
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

Automatically check if model/input is image-based. Convert input to a…

…n image
  • Loading branch information
teabolt committed Aug 7, 2019
commit 6b002a6900bf5de57807fe33d9e83ce9c0a6291a
@@ -9,6 +9,14 @@
import keras.backend as K # type: ignore
from keras.models import Model # type: ignore
from keras.layers import Layer # type: ignore
from keras.layers import ( # type: ignore
Conv2D,
MaxPooling2D,
AveragePooling2D,
GlobalMaxPooling2D,
GlobalAveragePooling2D,
)
from keras.preprocessing.image import array_to_img # type: ignore

from eli5.base import Explanation, TargetExplanation
from eli5.explain import explain_prediction
@@ -107,6 +115,14 @@ def explain_prediction_keras(model, # type: Model
targets=targets,
layer=layer,
)
elif _maybe_image(model, doc):
This conversation was marked as resolved by lopuhin

This comment has been minimized.

Copy link
@lopuhin

lopuhin Aug 8, 2019

Contributor

Not sure if it makes sense to warn in this case - I think if we say that image is optional, then we shouldn't raise warnings if user didn't pass it. If we remove the warning, then it's possible to join this with above "if" branch.

image = _extract_image(doc)
return explain_prediction_keras_image(model,
doc,
image=image,
targets=targets,
layer=layer,
)
else:
return explain_prediction_keras_not_supported(model, doc)

@@ -197,6 +213,45 @@ def explain_prediction_keras_image(model,
)


def _maybe_image(model, doc):
return _maybe_image_input(doc) and _maybe_image_model(model)


def _maybe_image_input(doc):
rank = len(doc.shape)
# image with channels or without (spatial only)
return rank == 4 or rank == 3


def _maybe_image_model(model):
# FIXME: replace try-except with something else
try:
# search for the first occurrence of an "image" layer
_search_layer_backwards(model, _is_possible_image_model_layer)
This conversation was marked as resolved by lopuhin

This comment has been minimized.

Copy link
@lopuhin

lopuhin Aug 8, 2019

Contributor

Nice heuristics 👍

return True
except ValueError:
return False


image_model_layers = (Conv2D,
MaxPooling2D,
AveragePooling2D,
GlobalMaxPooling2D,
GlobalAveragePooling2D,
)


def _is_possible_image_model_layer(model, layer):
return isinstance(layer, image_model_layers)


def _extract_image(doc):
im_arr, = doc # rank 4 batch -> rank 3 single image
image = array_to_img(im_arr)
image = image.convert(mode='RGBA')
return image


def _validate_doc(model, doc):
# type: (Model, np.ndarray) -> None
"""
@@ -226,7 +281,7 @@ def _get_activation_layer(model, layer):
"""
Get an instance of the desired activation layer in ``model``,
as specified by ``layer``.
"""
"""
if layer is None:
# Automatically get the layer if not provided
activation_layer = _search_layer_backwards(model, _is_suitable_activation_layer)
@@ -251,7 +306,7 @@ def _get_activation_layer(model, layer):


def _search_layer_backwards(model, condition):
# type: (Model, Callable[[Model, int], bool]) -> Layer
# type: (Model, Callable[[Model, Layer], bool]) -> Layer
"""
Search for a layer in ``model``, backwards (starting from the output layer),
checking if the layer is suitable with the callable ``condition``,
@@ -271,7 +326,7 @@ def _is_suitable_activation_layer(model, layer):
Check whether the layer ``layer`` matches what is required
by ``model`` to do Grad-CAM on ``layer``.
Returns a boolean.
Matching Criteria:
* Rank of the layer's output tensor.
"""
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.