docs/source/user_guide/model_understanding.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Understanding\n",
    "\n",
    "Simply examining a model's performance metrics is not enough to select a model and promote it for use in a production setting. While developing an ML algorithm, it is important to understand how the model behaves on the data, to examine the key factors influencing its predictions and to consider where it may be deficient. Determination of what \"success\" may mean for an ML project depends first and foremost on the user's domain expertise.\n",
    "\n",
    "EvalML includes a variety of tools for understanding models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's train a pipeline on some data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import evalml\n",
    "\n",
    "class RFBinaryClassificationPipeline(evalml.pipelines.BinaryClassificationPipeline):\n",
    "    component_graph = ['Simple Imputer', 'Random Forest Classifier']\n",
    "\n",
    "X, y = evalml.demos.load_breast_cancer()\n",
    "\n",
    "pipeline = RFBinaryClassificationPipeline({})\n",
    "pipeline.fit(X, y)\n",
    "print(pipeline.score(X, y, objectives=['log_loss_binary']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Importance\n",
    "\n",
    "We can get the importance associated with each feature of the resulting pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline.feature_importance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also create a bar plot of the feature importances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline.graph_feature_importance()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Permutation Importance\n",
    "\n",
    "We can also compute and plot [the permutation importance](https://scikit-learn.org/stable/modules/permutation_importance.html) of the pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "evalml.pipelines.calculate_permutation_importance(pipeline, X, y, 'log_loss_binary')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "evalml.pipelines.graph_permutation_importance(pipeline, X, y, 'log_loss_binary')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Confusion Matrix\n",
    "\n",
    "For binary or multiclass classification, we can view a [confusion matrix](https://en.wikipedia.org/wiki/Confusion_matrix) of the classifier's predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = pipeline.predict(X)\n",
    "evalml.pipelines.graph_utils.graph_confusion_matrix(y, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Precision-Recall Curve\n",
    "\n",
    "For binary classification, we can view the precision-recall curve of the pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the predicted probabilities associated with the \"true\" label\n",
    "y_encoded = y.map({'malignant': 0, 'benign': 1})\n",
    "y_pred_proba = pipeline.predict_proba(X)[\"benign\"]\n",
    "evalml.pipelines.graph_utils.graph_precision_recall_curve(y_encoded, y_pred_proba)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ROC Curve\n",
    "\n",
    "For binary and multiclass classification, we can view the [Receiver Operating Characteristic (ROC) curve](https://en.wikipedia.org/wiki/Receiver_operating_characteristic) of the pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the predicted probabilities associated with the \"benign\" label\n",
    "y_pred_proba = pipeline.predict_proba(X)[\"benign\"]\n",
    "evalml.pipelines.graph_utils.graph_roc_curve(y_encoded, y_pred_proba)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explaining Individual Predictions\n",
    "\n",
    "We can explain why the model made an individual prediction with the [explain_prediction](../generated/evalml.pipelines.prediction_explanations.explain_prediction.ipynb) function. This will use the [Shapley Additive Explanations (SHAP)](https://github.com/slundberg/shap) algorithms to identify the top features that explain the predicted value. \n",
    "\n",
    "This function can explain both classification and regression models - all you need to do is provide the pipeline, the input features (must correspond to one row of the input data) and the training data. The function will return a table that you can print summarizing the top 3 most positive and negative contributing features to the predicted value.\n",
    "\n",
    "In the example below, we explain the prediction for the third data point in the data set. We see that the `worst concave points` feature increased the estimated probability that the tumor is malignant by 20% while the `worst radius` feature decreased the probability the tumor is malignant by 5%.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from evalml.pipelines.prediction_explanations import explain_prediction\n",
    "\n",
    "table = explain_prediction(pipeline=pipeline, input_features=X.iloc[3:4],\n",
    "                           training_data=X, include_shap_values=True)\n",
    "print(table)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The interpretation of the table is the same for regression problems - but the SHAP value now corresponds to the change in the estimated value of the dependent variable rather than a change in probability. For multiclass classification problems, a table will be output for each possible class.\n",
    "\n",
    "This functionality is currently **not supported** for **XGBoost** models or **CatBoost multiclass** classifiers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explaining Multiple Predictions\n",
    "\n",
    "When debugging machine learning models, it is often useful to analyze the best and worst predictions the model made. The [explain_predictions_best_worst](../generated/evalml.pipelines.prediction_explanations.explain_predictions_best_worst.ipynb) function can help us with this.\n",
    "\n",
    "This function will display the output of [explain_prediction](../generated/evalml.pipelines.prediction_explanations.explain_prediction.ipynb) for the best 2 and worst 2 predictions. By default, the best and worst predictions are determined by the absolute error for regression problems and [cross entropy](https://en.wikipedia.org/wiki/Cross_entropy) for classification problems.\n",
    "\n",
    "We can specify our own ranking function by passing in a function to the `metric` parameter. This function will be called on `y_true` and `y_pred`. By convention, lower scores are better.\n",
    "\n",
    "At the top of each table, we can see the predicted probabilities, target value, and error on that prediction. For a regression problem, we would see the predicted value instead of predicted probabilities.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from evalml.pipelines.prediction_explanations import explain_predictions_best_worst\n",
    "\n",
    "report = explain_predictions_best_worst(pipeline=pipeline, input_features=X, y_true=y,\n",
    "                                        include_shap_values=True, num_to_explain=2)\n",
    "\n",
    "print(report)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use a custom metric ([hinge loss](https://en.wikipedia.org/wiki/Hinge_loss)) for selecting the best and worst predictions. See this example:\n",
    "\n",
    "```python\n",
    "import numpy as np\n",
    "\n",
    "def hinge_loss(y_true, y_pred_proba):\n",
    "    \n",
    "    probabilities = np.clip(y_pred_proba.iloc[:, 1], 0.001, 0.999)\n",
    "    y_true[y_true == 0] = -1\n",
    "    \n",
    "    return np.clip(1 - y_true * np.log(probabilities / (1 - probabilities)), a_min=0, a_max=None)\n",
    "\n",
    "report = explain_predictions_best_worst(pipeline=pipeline, input_features=X, y_true=y,\n",
    "                                        include_shap_values=True, num_to_explain=5, metric=hinge_loss)\n",
    "\n",
    "print(report)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also manually explain predictions on any subset of the training data with the [explain_predictions](../generated/evalml.pipelines.prediction_explanations.explain_predictions.ipynb) function. Below, we explain the predictions on the first, fifth, and tenth row of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from evalml.pipelines.prediction_explanations import explain_predictions\n",
    "\n",
    "report = explain_predictions(pipeline=pipeline, input_features=X.iloc[[0, 4, 9]], include_shap_values=True)\n",
    "print(report)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}