Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for AIX360 explanations #1094

Merged
merged 50 commits into from Sep 29, 2020
Merged

Add support for AIX360 explanations #1094

merged 50 commits into from Sep 29, 2020

Conversation

drewbutlerbb4
Copy link
Contributor

What this PR does / why we need it:
Adds a new Explainer framework called Artificial Intelligence eXplainability 360 (AIX360) from the Trusted AI project. This PR contains the first explainer using a package called LIME (local interpretable model-agnostic explanations). A sample has been created which gives explanations for a random forest model's predictions on the MNIST dataset.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1093

Special notes for your reviewer:

  1. Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Release note:


@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@kubeflow-bot
Copy link

This change is Reviewable

python/aixexplainer/aixserver/__main__.py Outdated Show resolved Hide resolved
python/aixexplainer/setup.py Outdated Show resolved Hide resolved
python/aixexplainer/aixserver/__init__.py Outdated Show resolved Hide resolved
docs/samples/explanation/aix/mnist/rfserver/rf.Dockerfile Outdated Show resolved Hide resolved
@yuzisun
Copy link
Member

yuzisun commented Sep 19, 2020

Thanks for the awesome contribution!

Can we add a e2e test like alibi explainer here ? https://github.com/kubeflow/kfserving/blob/master/test/e2e/explainer/test_tabular_explainer.py

@Tomcli
Copy link
Member

Tomcli commented Sep 22, 2020

/retest

},
"aix": {
"image" : "aipipeline/aix-explainer",
"defaultImageVersion": "0.4.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the pointer.

def __init__(self, name: str, predictor_host: str, segm_alg: str, num_samples: str,
top_labels: str, min_weight: str, positive_only: str, explainer_type: str):
super().__init__(name)
print("INIT")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use proper logger?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replaced print with logger.info

KFServing.wait_isvc_ready(service_name, namespace=KFSERVING_TEST_NAMESPACE, timeout_seconds=720)
except RuntimeError as e:
logging.info(KFServing.api_instance.get_namespaced_custom_object("serving.knative.dev", "v1",
KFSERVING_TEST_NAMESPACE, "services", service_name + "-predictor"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh the service name is actually, service_name + "-predictor-default", I was trying to remove -default but figured it breaks the backwards compatibility.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, we added back -default

@Tomcli
Copy link
Member

Tomcli commented Sep 25, 2020

@yuzisun @animeshsingh
Sorry for the late update. Our AIX360 API and server now should work on both the alpha and beta API with the latest v1beta1 controller and the new changes from the kfserving python server.

@yuzisun
Copy link
Member

yuzisun commented Sep 25, 2020

@yuzisun @animeshsingh
Sorry for the late update. Our AIX360 API and server now should work on both the alpha and beta API with the latest v1beta1 controller and the new changes from the kfserving python server.

That's awesome!

Copy link
Member

@animeshsingh animeshsingh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @drewbutlerbb4 - few nits, else looks good to go

docs/samples/explanation/aix/mnist/README.md Outdated Show resolved Hide resolved
docs/samples/explanation/aix/mnist/README.md Outdated Show resolved Hide resolved
Co-authored-by: Animesh Singh <singhan@us.ibm.com>
@drewbutlerbb4
Copy link
Contributor Author

/retest

@animeshsingh
Copy link
Member

/lgtm
/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: animeshsingh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Proposing a new explainer framework
6 participants