Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update saliency methods to use a framework-agnostic "core" subpackage, and moves all TF1 functionality to an optional "tf1" subpackage. #49

Merged
merged 132 commits into from
Mar 15, 2021

Conversation

bwedin
Copy link
Collaborator

@bwedin bwedin commented Jan 29, 2021

Changes in this PR:

  • Creates two subpackages, "core" and "tf1"
  • "core" is a framework-agnostic package that re-implements all existing methods (with the exception of GuidedBackprop). Each method calls a model through a user-supplied "call_model_function", which receives the inputs and any auxiliary model arguments, and returns relevant outputs (e.g. gradients for a single output neuron with respect to the input) that are used to calculate feature attributions.
  • "tf1" is functionally equivalent to the prior version, so any code that uses saliency can simply use saliency.tf1 instead.
  • To avoid unnecessary dependencies, tensorflow is no longer required as part of install, but is instead listed as an extra in saliency[full] and saliency[tf1].
  • Adds notebooks Examples_core.ipynb and Examples_tf1.ipynb, demonstrating how to use these subpackages with a model trained on ImageNet. Examples_tf1 uses a TF1 TFHub model, while Examples_core uses a VGG16 model from tf.keras.applications.
  • Gradient path methods (IG, BlurIG, XRAI) now have a batch_size parameter, which calls the model with multiple steps in a single batch. Default batch_size is 1, which mirrors previous behavior (i.e. each step is a separate model call).
  • Adds new tests for methods that previously had none (Occlusion, Gradients), as well as additional tests for error handling.

Bugfixes:

  • Fixed an off-by-one error in Occlusion
  • Added "scikit-image" as a package dependency

Tests for this change:

  • Running all tests on a local virtual environment
  • Running example notebooks and verifying the resulting masks match previous notebook outputs.

@bwedin bwedin closed this Jan 29, 2021
@bwedin bwedin deleted the call_model branch January 29, 2021 14:31
@bwedin bwedin restored the call_model branch January 29, 2021 14:31
@bwedin bwedin reopened this Jan 29, 2021
Copy link
Collaborator

@gkapish gkapish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

saliency/base.py Outdated Show resolved Hide resolved
saliency/base.py Outdated Show resolved Hide resolved
saliency/base.py Outdated Show resolved Hide resolved
saliency/base.py Outdated Show resolved Hide resolved
saliency/blur_ig.py Outdated Show resolved Hide resolved
saliency/base.py Outdated Show resolved Hide resolved
saliency/blur_ig_test.py Outdated Show resolved Hide resolved
saliency/integrated_gradients.py Outdated Show resolved Hide resolved
saliency/core/base.py Outdated Show resolved Hide resolved
saliency/core/gradients.py Outdated Show resolved Hide resolved
saliency/core/gradients.py Outdated Show resolved Hide resolved
saliency/tf1/base.py Show resolved Hide resolved
Copy link
Collaborator

@tolga-b tolga-b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you Ben, it looks great! I left some minor comments.
Could you also please update the PR title and description to list major updates that are happening. The title could be about making the library framework agnostic, and you could list the changes in the features or bugs this fixes the description as well as how you tested (e.g. ran notebooks visually compared and ran method tests). Thank you

saliency/__init__.py Outdated Show resolved Hide resolved
saliency/tf1/occlusion.py Outdated Show resolved Hide resolved
saliency/tf1/integrated_gradients.py Outdated Show resolved Hide resolved
saliency/core/base.py Outdated Show resolved Hide resolved
saliency/core/grad_cam.py Outdated Show resolved Hide resolved
saliency/core/xrai.py Outdated Show resolved Hide resolved
@tolga-b
Copy link
Collaborator

tolga-b commented Mar 10, 2021

Looks good, could you update the PR description and title please?

@bwedin bwedin changed the title Add CallModelSaliency, and implement for GradientSaliency, IG, BlurIG, XRAI, and GradCAM.Call model Update saliency methods to use a framework-agnostic "core" subpackage, and moves all TF1 functionality to an optional "tf1" subpackage. Mar 10, 2021
@tolga-b
Copy link
Collaborator

tolga-b commented Mar 10, 2021

This also resolves #16. We may want to think about adding more notebook examples for other frameworks in future.

Copy link
Collaborator

@tolga-b tolga-b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add to the description that you no longer use tf.lite models and use tf hub for tf1 and keras for the core notebooks, respectively.

Copy link
Collaborator

@gkapish gkapish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants