IBM Cloud/WML/notebooks/unstructured_image/keras/Watson OpenScale Explanation for Image Multiclass.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"id":"aec66e06-1d53-4c76-a7c4-35f47da35814"},"source":["<img src=\"https://github.com/pmservice/ai-openscale-tutorials/raw/master/notebooks/images/banner.png\" align=\"left\" alt=\"banner\">"]},{"cell_type":"markdown","metadata":{"id":"1bb1a475-8789-43eb-82e5-e21b48912813"},"source":["# Tutorial on generating an explanation for an image-based model on Watson OpenScale"]},{"cell_type":"markdown","metadata":{"id":"c1b52f0c-ea07-475a-aa31-2bcfe305af0f"},"source":["This notebook includes steps for creating an image-based watson-machine-learning model, creating a subscription, configuring explainability, and finally generating an explanation for a transaction."]},{"cell_type":"markdown","metadata":{"id":"b6eb375a-83d6-4e61-b3f6-eb5c6700d7a4"},"source":["### Contents\n","- [1. Setup](#setup)\n","- [2. Creating and deploying an image-based model](#deployment)\n","- [3. Subscriptions](#subscription)\n","- [4. Explainability](#explainability)"]},{"cell_type":"markdown","metadata":{"id":"fe26966e-84e2-4806-8f40-2c3f60dea661"},"source":["**Note**: This notebook is using runtime `IBM Runtime 23.1 on Python 3.10` if you are using IBM Watson Studio or `Python 3.10.x` if using any other tool."]},{"cell_type":"markdown","metadata":{"id":"0a867133-1758-446a-9514-cbfd6975fb93"},"source":["<a id=\"setup\"></a>\n","## 1. Setup\n","\n","### 1.1 Install Watson OpenScale and WML packages"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"0722fd4b-415e-478f-a3bf-845fbb8f858f"},"outputs":[],"source":["!pip install --upgrade ibm-watson-openscale --no-cache | tail -n 1"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"0d04d00d-3db9-4f3e-9181-95093fd951b3"},"outputs":[],"source":["!pip install --upgrade ibm-watson-machine-learning --no-cache | tail -n 1"]},{"cell_type":"markdown","metadata":{"id":"a750ce91-b4d9-4aed-b486-7ac1bd4e856f"},"source":["Note: Restart the kernel to assure the new libraries are being used."]},{"cell_type":"markdown","metadata":{"id":"2c6acc0a-169f-4146-8535-de225e0f9259"},"source":["### 1.2 Configure credentials"]},{"cell_type":"markdown","metadata":{"id":"37c17136-3581-4ea6-ba8e-3f37c9114f19"},"source":["Your Cloud API key can be generated by going to the [**Users** section of the Cloud console](https://cloud.ibm.com/iam#/users). From that page, click your name, scroll down to the **API Keys** section, and click **Create an IBM Cloud API key**. Give your key a name and click **Create**, then copy the created key and paste it below.\n","\n","**NOTE:** You can also get OpenScale `API_KEY` using IBM CLOUD CLI.\n","\n","How to install IBM Cloud (bluemix) console: [instruction](https://console.bluemix.net/docs/cli/reference/ibmcloud/download_cli.html#install_use)\n","\n","How to get api key using console:\n","```\n","bx login --sso\n","bx iam api-key-create 'my_key'\n","```"]},{"cell_type":"code","execution_count":1,"metadata":{"id":"681253ce-e532-47cc-8fc1-a8127ee65fef"},"outputs":[],"source":["CLOUD_API_KEY = \"***\"\n","IAM_URL=\"https://iam.ng.bluemix.net/oidc/token\""]},{"cell_type":"code","execution_count":2,"metadata":{"id":"91f00563-a88e-4d84-809b-ad6d111adefc"},"outputs":[],"source":["WML_CREDENTIALS = {\n","                   \"url\": \"https://us-south.ml.cloud.ibm.com\",\n","                   \"apikey\": CLOUD_API_KEY\n","}"]},{"cell_type":"markdown","metadata":{"id":"37ed9f68-2497-4018-9f85-9bc6e91e8c31"},"source":["<a id=\"deployment\"></a>\n","## 2. Creating and deploying an image-based model"]},{"cell_type":"markdown","metadata":{"id":"5fdc0c1a-4232-41ae-bacc-289d22edaa2d"},"source":["The dataset used is MNIST dataset of handwritten digits. It consists of 60,000 28x28 grayscale images of the 10 digits, along with a test set of 10,000 images. More information about the dataset can be found here: https://keras.io/datasets/#mnist-database-of-handwritten-digits\n","\n","Note: Keras and TensorFlow versions supported by WML are: Keras 2.12.0 with TensorFlow 2.12.0 backend and Keras 2.2.4 with TensorFlow 1.14 backend. The latter combination is used in this notebook."]},{"cell_type":"markdown","metadata":{"id":"c7dca138-8491-4413-96d7-557b82088c2c"},"source":["### 2.1 Creating a model"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"d4672bcb-6449-44f4-b984-5b608464f82c","scrolled":true},"outputs":[],"source":["# note: Install these package if you are not running this notebook in IBM Watson Studio\n","# !pip install keras==2.12.0\n","# !pip install tensorflow==2.12.0"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"5f0b21e2-789f-4479-a573-cb30c64369d1"},"outputs":[],"source":["!pip install keras_sequential_ascii"]},{"cell_type":"code","execution_count":5,"metadata":{"id":"01461a40-feea-470b-9c4e-a85639a11aa8"},"outputs":[],"source":["#import keras\n","from tensorflow.keras.datasets import mnist\n","from tensorflow.keras.models import Sequential\n","from tensorflow.keras.layers import Dense, Dropout, Flatten\n","from tensorflow.keras.layers import Conv2D, MaxPooling2D\n","from keras_sequential_ascii import sequential_model_to_ascii_printout\n","from tensorflow.keras import backend as keras_backend\n","#print(tensorflow.__version__)"]},{"cell_type":"code","execution_count":6,"metadata":{"id":"0ed17110-4ca3-4fdf-a0cb-5a4f05b48ac5"},"outputs":[],"source":["batch_size = 128\n","num_classes = 10\n","epochs = 5"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"72fd4fa7-6467-4fe3-8d3a-52d8f0ac1c82","scrolled":true},"outputs":[],"source":["# input image dimensions\n","img_rows, img_cols = 28, 28\n","\n","# the data, split between train and test sets\n","(x_train, y_train), (x_test, y_test) = mnist.load_data()\n","\n","if keras_backend.image_data_format() == 'channels_first':\n","    x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)\n","    x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)\n","    input_shape = (1, img_rows, img_cols)\n","else:\n","    x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)\n","    x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)\n","    input_shape = (img_rows, img_cols, 1)\n","\n","x_train = x_train.astype('float32')\n","x_test = x_test.astype('float32')\n","x_train /= 255\n","x_test /= 255\n","\n","print('x_train shape:', x_train.shape)\n","print(x_train.shape[0], 'train samples')\n","print(x_test.shape[0], 'test samples')"]},{"cell_type":"code","execution_count":8,"metadata":{"id":"a8d09a7f-fb3d-4543-a555-5211b1daea75"},"outputs":[],"source":["# convert class vectors to binary class matrices\n","from tensorflow.keras.utils import to_categorical\n","y_train = to_categorical(y_train, num_classes)\n","y_test = to_categorical(y_test, num_classes)"]},{"cell_type":"code","execution_count":9,"metadata":{"id":"1ba4149d-ac17-422d-a69c-a76d480a5ff7"},"outputs":[],"source":["# Define Model\n","from tensorflow.keras.optimizers import Adadelta\n","from tensorflow.keras.losses import categorical_crossentropy\n","def base_model():\n","    model = Sequential()\n","    model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=input_shape))\n","    model.add(Conv2D(64, (3, 3), activation='relu'))\n","    model.add(MaxPooling2D(pool_size=(2, 2)))\n","    model.add(Dropout(0.25))\n","    model.add(Flatten())\n","    model.add(Dense(128, activation='relu'))\n","    model.add(Dropout(0.5))\n","    model.add(Dense(num_classes, activation='softmax'))\n","\n","    model.compile(loss=categorical_crossentropy,\n","                  optimizer=Adadelta(),\n","                  metrics=['accuracy'])\n","    return model"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"60f858e5-2d90-4157-a458-3396e2423846"},"outputs":[],"source":["cnn_n = base_model()\n","cnn_n.summary()"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"312496f5-6a9c-4fd8-b0cc-0cfc67329304"},"outputs":[],"source":["# Vizualizing model structure\n","sequential_model_to_ascii_printout(cnn_n)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"d2d8cb78-f6e6-47a6-b462-23c708337e15"},"outputs":[],"source":["# Fit model\n","print(y_train.shape)\n","cnn = cnn_n.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_data=(x_test, y_test))"]},{"cell_type":"code","execution_count":15,"metadata":{"id":"4b411a54-4b7b-4575-9c1c-0ad66c53be04"},"outputs":[{"name":"stdout","output_type":"stream","text":["[1.8136699199676514, 0.7049000263214111]\n","Accuracy: 70.49%\n"]}],"source":["scores = cnn_n.evaluate(x_test, y_test, verbose=0)\n","print(scores)\n","print(\"Accuracy: %.2f%%\" % (scores[1]*100))"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"fcc74508-15a9-4b61-8f10-02039b08e583"},"outputs":[],"source":["cnn_n.save(\"mnist_cnn.h5\")\n","!rm mnist_cnn.tar*\n","!tar -czvf mnist_cnn.tar.gz mnist_cnn.h5"]},{"cell_type":"markdown","metadata":{"id":"082e7530-df61-4d13-9669-4973f31b5bab"},"source":["### 2.2 Storing the model"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"7528af93-b995-4465-b3f0-cf038c834628"},"outputs":[],"source":["import json\n","from ibm_watson_machine_learning import APIClient\n","\n","wml_client = APIClient(WML_CREDENTIALS)\n","wml_client.version"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"5d4e7970-1409-4fb1-864e-663adf148acc"},"outputs":[],"source":["wml_client.spaces.list(limit=10)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"1c12fc42-85a0-4263-9ca0-329180633eb7"},"outputs":[],"source":["WML_SPACE_ID='***' # use space id here\n","wml_client.set.default_space(WML_SPACE_ID)"]},{"cell_type":"code","execution_count":20,"metadata":{"id":"934507b3-90f1-4abd-923c-c28e819482c0"},"outputs":[],"source":["MODEL_NAME = \"MNIST Model\""]},{"cell_type":"code","execution_count":null,"metadata":{"id":"be578733-3430-4e62-9242-a8bdd76407e0"},"outputs":[],"source":["software_spec_uid = wml_client.software_specifications.get_uid_by_name(\"tensorflow_rt23.1-py3.10\")\n","print(\"Software Specification ID: {}\".format(software_spec_uid))\n","model_props = {\n","        wml_client.repository.ModelMetaNames.NAME:\"{}\".format(MODEL_NAME),\n","        wml_client.repository.ModelMetaNames.TYPE: 'tensorflow_2.12',\n","        wml_client.repository.ModelMetaNames.SOFTWARE_SPEC_UID: software_spec_uid,\n","    }"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"ed752091-84e9-446d-83b7-c7f72020f0f0"},"outputs":[],"source":["print(\"Storing model ...\")\n","published_model_details = wml_client.repository.store_model(\n","    model='mnist_cnn.tar.gz', \n","    meta_props=model_props, \n",")\n","\n","model_uid = wml_client.repository.get_model_id(published_model_details)\n","print(\"Done\")\n","print(\"Model ID: {}\".format(model_uid))"]},{"cell_type":"markdown","metadata":{"id":"7ca23201-93b3-4555-811b-fb2a9ac4a699"},"source":["### 2.3 Deploying the model"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"e384dfd7-03f8-4cb6-898e-427cd5572b23"},"outputs":[],"source":["deployment_details = wml_client.deployments.create(\n","    model_uid, \n","    meta_props={\n","        wml_client.deployments.ConfigurationMetaNames.NAME: \"{}\".format(MODEL_NAME + \" deployment\"),\n","        wml_client.deployments.ConfigurationMetaNames.ONLINE: {}\n","    }\n",")\n","scoring_url = wml_client.deployments.get_scoring_href(deployment_details)\n","deployment_uid=wml_client.deployments.get_id(deployment_details)\n","\n","print(\"Scoring URL:\" + scoring_url)\n","print(\"Model id: {}\".format(model_uid))\n","print(\"Deployment id: {}\".format(deployment_uid))"]},{"cell_type":"markdown","metadata":{"id":"02f02eae-c8ca-4c7e-9d35-9bc36635aa14"},"source":["## 3. Subscriptions <a id=\"subscriptions\"></a>"]},{"cell_type":"markdown","metadata":{"id":"7bdae0f0-b11c-4e10-b3cc-8a965b01d1d3"},"source":["### 3.1 Configuring OpenScale"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"4ab3ab8d26aa4d9284a475a2dcfb48ab"},"outputs":[],"source":["from ibm_cloud_sdk_core.authenticators import IAMAuthenticator,BearerTokenAuthenticator\n","\n","from ibm_watson_openscale import *\n","from ibm_watson_openscale.supporting_classes.enums import *\n","from ibm_watson_openscale.supporting_classes import *\n","\n","\n","authenticator = IAMAuthenticator(apikey=CLOUD_API_KEY)\n","wos_client = APIClient(authenticator=authenticator)\n","wos_client.version"]},{"cell_type":"code","execution_count":26,"metadata":{"id":"7a346239-99cb-4d6f-a8ca-e169981f71a3"},"outputs":[],"source":["#DB_CREDENTIALS= {\"hostname\":\"\",\"username\":\"\",\"password\":\"\",\"database\":\"\",\"port\":\"\",\"ssl\":True,\"sslmode\":\"\",\"certificate_base64\":\"\"}\n","DB_CREDENTIALS = None\n","KEEP_MY_INTERNAL_POSTGRES = True"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"d8512729-1243-4670-b55c-bdc3abb601c2"},"outputs":[],"source":["data_marts = wos_client.data_marts.list().result.data_marts\n","if len(data_marts) == 0:\n","    if DB_CREDENTIALS is not None:\n","        if SCHEMA_NAME is None: \n","            print(\"Please specify the SCHEMA_NAME and rerun the cell\")\n","\n","        print('Setting up external datamart')\n","        added_data_mart_result = wos_client.data_marts.add(\n","                background_mode=False,\n","                name=\"WOS Data Mart\",\n","                description=\"Data Mart created by WOS tutorial notebook\",\n","                database_configuration=DatabaseConfigurationRequest(\n","                  database_type=DatabaseType.POSTGRESQL,\n","                    credentials=PrimaryStorageCredentialsLong(\n","                        hostname=DB_CREDENTIALS['hostname'],\n","                        username=DB_CREDENTIALS['username'],\n","                        password=DB_CREDENTIALS['password'],\n","                        db=DB_CREDENTIALS['database'],\n","                        port=DB_CREDENTIALS['port'],\n","                        ssl=True,\n","                        sslmode=DB_CREDENTIALS['sslmode'],\n","                        certificate_base64=DB_CREDENTIALS['certificate_base64']\n","                    ),\n","                    location=LocationSchemaName(\n","                        schema_name= SCHEMA_NAME\n","                    )\n","                )\n","             ).result\n","    else:\n","        print('Setting up internal datamart')\n","        added_data_mart_result = wos_client.data_marts.add(\n","                background_mode=False,\n","                name=\"WOS Data Mart\",\n","                description=\"Data Mart created by WOS tutorial notebook\", \n","                internal_database = True).result\n","        \n","    data_mart_id = added_data_mart_result.metadata.id\n","    \n","else:\n","    data_mart_id=data_marts[0].metadata.id\n","    print('Using existing datamart {}'.format(data_mart_id))"]},{"cell_type":"code","execution_count":31,"metadata":{"id":"68faac56-5c6f-4eba-959c-da15f8e566ed"},"outputs":[],"source":["SERVICE_PROVIDER_NAME = \"Image Multiclass Watson Machine Learning V2_test\"\n","SERVICE_PROVIDER_DESCRIPTION = \"Added by tutorial WOS notebook.\""]},{"cell_type":"code","execution_count":32,"metadata":{"id":"22ba1c76-c402-4005-9ecd-4ed96b9c870c"},"outputs":[],"source":["service_providers = wos_client.service_providers.list().result.service_providers\n","for service_provider in service_providers:\n","    service_instance_name = service_provider.entity.name\n","    if service_instance_name == SERVICE_PROVIDER_NAME:\n","        service_provider_id = service_provider.metadata.id\n","        wos_client.service_providers.delete(service_provider_id)\n","        print(\"Deleted existing service_provider for WML instance: {}\".format(service_provider_id))"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"3aedf507-91e5-4837-a87d-a13aeb7e5b78"},"outputs":[],"source":["added_service_provider_result = wos_client.service_providers.add(\n","        name=SERVICE_PROVIDER_NAME,\n","        description=SERVICE_PROVIDER_DESCRIPTION,\n","        service_type=ServiceTypes.WATSON_MACHINE_LEARNING,\n","        deployment_space_id = WML_SPACE_ID,\n","        operational_space_id = \"production\",\n","        credentials=WMLCredentialsCloud(\n","            apikey=CLOUD_API_KEY,      ## use `apikey=IAM_TOKEN` if using IAM_TOKEN to initiate client\n","            url=WML_CREDENTIALS[\"url\"],\n","            instance_id=None\n","        ),\n","        background_mode=False\n","    ).result\n","service_provider_id = added_service_provider_result.metadata.id"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"37a98327-b305-4943-aba8-c079823fc4b1"},"outputs":[],"source":["asset_deployment_details = wos_client.service_providers.list_assets(data_mart_id=data_mart_id, service_provider_id=service_provider_id, deployment_id = deployment_uid, deployment_space_id = WML_SPACE_ID).result['resources'][0]\n","asset_deployment_details"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"e2ee6333-b839-491b-8907-a50cd2cd5631"},"outputs":[],"source":["model_asset_details_from_deployment=wos_client.service_providers.get_deployment_asset(data_mart_id=data_mart_id,service_provider_id=service_provider_id,deployment_id=deployment_uid,deployment_space_id=WML_SPACE_ID)\n","model_asset_details_from_deployment"]},{"cell_type":"markdown","metadata":{"id":"92272413-aeed-4858-b06f-4a63153c5ff6"},"source":["### 3.2 Subscribe the asset"]},{"cell_type":"code","execution_count":36,"metadata":{"id":"bed89bfb-8b13-4eb7-9032-5f51bf676fc9"},"outputs":[],"source":["subscriptions = wos_client.subscriptions.list().result.subscriptions\n","for subscription in subscriptions:\n","    sub_model_id = subscription.entity.asset.asset_id\n","    if sub_model_id == model_uid:\n","        wos_client.subscriptions.delete(subscription.metadata.id)\n","        print('Deleted existing subscription for model', model_uid)"]},{"cell_type":"code","execution_count":37,"metadata":{"id":"9392e589-f30d-42e3-baaf-d312ebbf89d8"},"outputs":[],"source":["from ibm_watson_openscale.base_classes.watson_open_scale_v2 import ScoringEndpointRequest"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"bc303a8f-9469-467c-9767-40f8944b859d"},"outputs":[],"source":["subscription_details = wos_client.subscriptions.add(\n","        data_mart_id=data_mart_id,\n","        service_provider_id=service_provider_id,\n","        asset=Asset(\n","            asset_id=model_asset_details_from_deployment[\"entity\"][\"asset\"][\"asset_id\"],\n","            name=model_asset_details_from_deployment[\"entity\"][\"asset\"][\"name\"],\n","            url=model_asset_details_from_deployment[\"entity\"][\"asset\"][\"url\"],\n","            asset_type=AssetTypes.MODEL,\n","            input_data_type=InputDataType.UNSTRUCTURED_IMAGE,\n","            problem_type=ProblemType.MULTICLASS_CLASSIFICATION\n","        ),\n","        deployment=AssetDeploymentRequest(\n","            deployment_id=asset_deployment_details['metadata']['guid'],\n","            name=asset_deployment_details['entity']['name'],\n","            deployment_type= DeploymentTypes.ONLINE,\n","            scoring_endpoint=ScoringEndpointRequest(url=scoring_url) # scoring model without shadow deployment\n","        ),\n","        asset_properties=AssetPropertiesRequest(\n","            probability_fields=['probability']\n","            )\n","    ).result\n","subscription_id = subscription_details.metadata.id\n","subscription_id"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"1e3e5d8e-2f19-4b9a-82bb-b10c768a5179"},"outputs":[],"source":["import time\n","\n","time.sleep(5)\n","payload_data_set_id = None\n","payload_data_set_id = wos_client.data_sets.list(type=DataSetTypes.PAYLOAD_LOGGING, \n","                                                target_target_id=subscription_id, \n","                                                target_target_type=TargetTypes.SUBSCRIPTION).result.data_sets[0].metadata.id\n","if payload_data_set_id is None:\n","    print(\"Payload data set not found. Please check subscription status.\")\n","else:\n","    print(\"Payload data set id: \", payload_data_set_id)"]},{"cell_type":"markdown","metadata":{"id":"70c02843-861a-400b-9d71-437fdb82acc3"},"source":["### 3.3 Score the model and get transaction-id"]},{"cell_type":"code","execution_count":41,"metadata":{"id":"0d570aa9-8291-47af-853d-cf18466567ca"},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAaEAAAGdCAYAAAC7EMwUAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAZ30lEQVR4nO3df2hV9/3H8df11/UHN3cEm9ybGrNQlA0jMn8sKlajm1kDk1lXsJVtuj/ErlEXUme1DgwdM9WhjGHrWL/FVaardFMnU2bTaWLF2alEGpyTFOPMMJfMoLkxtVfUz/cP8dLbpNFzvTfv3OT5gAvm3vP2fjw99enJvffE55xzAgDAwCDrBQAABi4iBAAwQ4QAAGaIEADADBECAJghQgAAM0QIAGCGCAEAzAyxXsAX3bt3T1evXlUgEJDP57NeDgDAI+ecOjo6lJeXp0GDej7X6XMRunr1qvLz862XAQB4TM3NzRozZkyP2/S5b8cFAgHrJQAAUuBR/j5PW4TefPNNFRYWavjw4ZoyZYo+/PDDR5rjW3AA0D88yt/naYnQ3r17VVFRoQ0bNqi+vl5PP/20ysrKdOXKlXQ8HQAgQ/nScRXt4uJiTZ48WTt27Ijf9/Wvf10LFy5UdXV1j7PRaFTBYDDVSwIA9LL29nZlZWX1uE3Kz4Ru376ts2fPqrS0NOH+0tJSnTx5ssv2sVhM0Wg04QYAGBhSHqFr167p7t27ys3NTbg/NzdXkUiky/bV1dUKBoPxG++MA4CBI21vTPjiC1LOuW5fpFq/fr3a29vjt+bm5nQtCQDQx6T8c0KjR4/W4MGDu5z1tLa2djk7kiS/3y+/35/qZQAAMkDKz4SGDRumKVOmqKamJuH+mpoazZw5M9VPBwDIYGm5YkJlZaV++MMfaurUqZoxY4Z+97vf6cqVK3rxxRfT8XQAgAyVlggtXrxYbW1teu2119TS0qKioiIdPnxYBQUF6Xg6AECGSsvnhB4HnxMCgP7B5HNCAAA8KiIEADBDhAAAZogQAMAMEQIAmCFCAAAzRAgAYIYIAQDMECEAgBkiBAAwQ4QAAGaIEADADBECAJghQgAAM0QIAGCGCAEAzBAhAIAZIgQAMEOEAABmiBAAwAwRAgCYIUIAADNECABghggBAMwQIQCAGSIEADBDhAAAZogQAMAMEQIAmCFCAAAzRAgAYIYIAQDMECEAgBkiBAAwQ4QAAGaIEADADBECAJghQgAAM0QIAGCGCAEAzBAhAIAZIgQAMEOEAABmiBAAwAwRAgCYIUIAADNECABghggBAMwQIQCAGSIEADBDhAAAZogQAMAMEQIAmCFCAAAzRAgAYCblEaqqqpLP50u4hUKhVD8NAKAfGJKO33TChAn64IMP4l8PHjw4HU8DAMhwaYnQkCFDOPsBADxUWl4TamxsVF5engoLC/X888/r0qVLX7ptLBZTNBpNuAEABoaUR6i4uFi7du3SkSNH9NZbbykSiWjmzJlqa2vrdvvq6moFg8H4LT8/P9VLAgD0UT7nnEvnE3R2duqpp57S2rVrVVlZ2eXxWCymWCwW/zoajRIiAOgH2tvblZWV1eM2aXlN6PNGjRqliRMnqrGxsdvH/X6//H5/upcBAOiD0v45oVgspgsXLigcDqf7qQAAGSblEVqzZo3q6urU1NSkjz76SM8995yi0aiWLl2a6qcCAGS4lH877r///a9eeOEFXbt2TU888YSmT5+uU6dOqaCgINVPBQDIcGl/Y4JX0WhUwWDQehlIk2HDhnme+fnPf+55ZvXq1Z5nJD30RdTuRCIRzzMnTpzwPLNu3TrPMz19PAJIt0d5YwLXjgMAmCFCAAAzRAgAYIYIAQDMECEAgBkiBAAwQ4QAAGaIEADADBECAJghQgAAM0QIAGCGCAEAzHABUyRt5MiRnmd++tOfep755S9/6XnmwoULnmck6ciRI55nJk+e7Hlm2rRpnmd8Pp/nmXnz5nmekaRTp04lNQd8HhcwBQD0aUQIAGCGCAEAzBAhAIAZIgQAMEOEAABmiBAAwAwRAgCYIUIAADNECABghggBAMwQIQCAGSIEADAzxHoByFyrV6/2PLNp0ybPM/v27fM884Mf/MDzjCTdunUrqTmvvv3tb3ueSWY//P3vf/c8I0lTpkzxPPPvf/87qefCwMaZEADADBECAJghQgAAM0QIAGCGCAEAzBAhAIAZIgQAMEOEAABmiBAAwAwRAgCYIUIAADNECABgxuecc9aL+LxoNKpgMGi9jAHlK1/5SlJzFy9e9Dxz8+ZNzzMTJkzwPPPZZ595nunr5s+f73nmz3/+c1LPdfXqVc8zO3fu9DyzefNmzzPIHO3t7crKyupxG86EAABmiBAAwAwRAgCYIUIAADNECABghggBAMwQIQCAGSIEADBDhAAAZogQAMAMEQIAmCFCAAAzXMAUKikpSWru6NGjnmcqKio8z/zmN7/xPIP73n777aTmfvzjH6d4Jd0bNIh/B/dnXMAUANCnESEAgBnPETp+/LgWLFigvLw8+Xw+HThwIOFx55yqqqqUl5enESNGqKSkROfPn0/VegEA/YjnCHV2dmrSpEnavn17t49v2bJF27Zt0/bt23X69GmFQiHNnz9fHR0dj71YAED/MsTrQFlZmcrKyrp9zDmnX//619qwYYMWLVokSXrnnXeUm5urPXv2aMWKFY+3WgBAv5LS14SampoUiURUWloav8/v92vOnDk6efJktzOxWEzRaDThBgAYGFIaoUgkIknKzc1NuD83Nzf+2BdVV1crGAzGb/n5+alcEgCgD0vLu+N8Pl/C1865Lvc9sH79erW3t8dvzc3N6VgSAKAP8vyaUE9CoZCk+2dE4XA4fn9ra2uXs6MH/H6//H5/KpcBAMgQKT0TKiwsVCgUUk1NTfy+27dvq66uTjNnzkzlUwEA+gHPZ0I3b97UJ598Ev+6qalJ586dU3Z2tsaOHauKigpt2rRJ48aN07hx47Rp0yaNHDlSS5YsSenCAQCZz3OEzpw5o7lz58a/rqyslCQtXbpUv//977V27VrdunVLL730kq5fv67i4mK9//77CgQCqVs1AKBf8ByhkpIS9XTNU5/Pp6qqKlVVVT3OutCLiouLk5q7c+eO55n33nsvqedCcnrzjT7//Oc/e+250H9w7TgAgBkiBAAwQ4QAAGaIEADADBECAJghQgAAM0QIAGCGCAEAzBAhAIAZIgQAMEOEAABmiBAAwAwRAgCYSelPVkVmeu6555Ka++tf/+p5pqWlJannQt939epV6yUgA3EmBAAwQ4QAAGaIEADADBECAJghQgAAM0QIAGCGCAEAzBAhAIAZIgQAMEOEAABmiBAAwAwRAgCY4QKm0LVr15KaCwaDvTLT3t7ueQa9r7a21noJyECcCQEAzBAhAIAZIgQAMEOEAABmiBAAwAwRAgCYIUIAADNECABghggBAMwQIQCAGSIEADBDhAAAZriAKfSLX/wiqblkLlj5wQcfeJ7ZtGmT55n9+/d7nsHjSeZCsyNGjPA887Of/czzzGuvveZ5Br2DMyEAgBkiBAAwQ4QAAGaIEADADBECAJghQgAAM0QIAGCGCAEAzBAhAIAZIgQAMEOEAABmiBAAwAwXMIVOnjyZ1Nz69es9z/zqV7/yPPN///d/nmecc55nJOnAgQNJzUH6xje+4Xlm3bp1nmeSuQgu+i7OhAAAZogQAMCM5wgdP35cCxYsUF5ennw+X5dvXyxbtkw+ny/hNn369FStFwDQj3iOUGdnpyZNmqTt27d/6TbPPPOMWlpa4rfDhw8/1iIBAP2T5zcmlJWVqaysrMdt/H6/QqFQ0osCAAwMaXlNqLa2Vjk5ORo/fryWL1+u1tbWL902FospGo0m3AAAA0PKI1RWVqbdu3fr6NGj2rp1q06fPq158+YpFot1u311dbWCwWD8lp+fn+olAQD6qJR/Tmjx4sXxXxcVFWnq1KkqKCjQoUOHtGjRoi7br1+/XpWVlfGvo9EoIQKAASLtH1YNh8MqKChQY2Njt4/7/X75/f50LwMA0Ael/XNCbW1tam5uVjgcTvdTAQAyjOczoZs3b+qTTz6Jf93U1KRz584pOztb2dnZqqqq0ve//32Fw2FdvnxZr776qkaPHq1nn302pQsHAGQ+zxE6c+aM5s6dG//6wes5S5cu1Y4dO9TQ0KBdu3bpxo0bCofDmjt3rvbu3atAIJC6VQMA+gWfS/ZKj2kSjUYVDAatl4E0Wb16teeZNWvWeJ4ZM2aM5xlJ2rZtm+eZzZs3e565ffu255mOjg7PM+fOnfM8I91/U1FvaGho8DzzrW99y/PMtWvXPM/g8bW3tysrK6vHbbh2HADADBECAJghQgAAM0QIAGCGCAEAzBAhAIAZIgQAMEOEAABmiBAAwAwRAgCYIUIAADNECABghggBAMxwFW30eclcEXvJkiVJPderr77qeeZhVwnuzt27dz3PnDlzxvNMcXGx55lkffTRR55nFixY4HmGK2JnDq6iDQDo04gQAMAMEQIAmCFCAAAzRAgAYIYIAQDMECEAgBkiBAAwQ4QAAGaIEADADBECAJghQgAAM1zAFPicJ5980vPMj370I88zfr/f80wy/6uuWLHC84wkhUIhzzPZ2dmeZ27cuOF5BpmDC5gCAPo0IgQAMEOEAABmiBAAwAwRAgCYIUIAADNECABghggBAMwQIQCAGSIEADBDhAAAZogQAMAMFzAF+rH6+vqk5goKCjzPjB071vPMzZs3Pc8gc3ABUwBAn0aEAABmiBAAwAwRAgCYIUIAADNECABghggBAMwQIQCAGSIEADBDhAAAZogQAMAMEQIAmBlivQAAfc+5c+c8z3AxUiSDMyEAgBkiBAAw4ylC1dXVmjZtmgKBgHJycrRw4UJdvHgxYRvnnKqqqpSXl6cRI0aopKRE58+fT+miAQD9g6cI1dXVqby8XKdOnVJNTY3u3Lmj0tJSdXZ2xrfZsmWLtm3bpu3bt+v06dMKhUKaP3++Ojo6Ur54AEBme6yfrPq///1POTk5qqur0+zZs+WcU15enioqKvTKK69IkmKxmHJzc7V582atWLHiob8nP1kVSJ1kf7Lq9evXPc/MmzcvqedC/5X2n6za3t4uScrOzpYkNTU1KRKJqLS0NL6N3+/XnDlzdPLkyW5/j1gspmg0mnADAAwMSUfIOafKykrNmjVLRUVFkqRIJCJJys3NTdg2Nzc3/tgXVVdXKxgMxm/5+fnJLgkAkGGSjtDKlSv18ccf649//GOXx3w+X8LXzrku9z2wfv16tbe3x2/Nzc3JLgkAkGGS+rDqqlWrdPDgQR0/flxjxoyJ3x8KhSTdPyMKh8Px+1tbW7ucHT3g9/vl9/uTWQYAIMN5OhNyzmnlypXat2+fjh49qsLCwoTHCwsLFQqFVFNTE7/v9u3bqqur08yZM1OzYgBAv+HpTKi8vFx79uzRX/7yFwUCgfjrPMFgUCNGjJDP51NFRYU2bdqkcePGady4cdq0aZNGjhypJUuWpOUPAADIXJ4itGPHDklSSUlJwv07d+7UsmXLJElr167VrVu39NJLL+n69esqLi7W+++/r0AgkJIFAwD6j8f6nFA68DkhoHuzZ8/2PFNbW5vUc61atcrzzBtvvJHUc6H/SvvnhAAAeBxECABghggBAMwQIQCAGSIEADBDhAAAZogQAMAMEQIAmCFCAAAzRAgAYIYIAQDMECEAgBkiBAAwk9RPVgXQ+5K5snWyDh061GvPhYGNMyEAgBkiBAAwQ4QAAGaIEADADBECAJghQgAAM0QIAGCGCAEAzBAhAIAZIgQAMEOEAABmiBAAwAwXMAUMDB8+3PPMlClTPM9EIhHPM5J048aNpOYArzgTAgCYIUIAADNECABghggBAMwQIQCAGSIEADBDhAAAZogQAMAMEQIAmCFCAAAzRAgAYIYIAQDMcAFTwMDUqVM9z3z1q1/1PPPKK694npG4gCl6D2dCAAAzRAgAYIYIAQDMECEAgBkiBAAwQ4QAAGaIEADADBECAJghQgAAM0QIAGCGCAEAzBAhAIAZLmAKGBg1apTnmc7OTs8z7733nucZoDdxJgQAMEOEAABmPEWourpa06ZNUyAQUE5OjhYuXKiLFy8mbLNs2TL5fL6E2/Tp01O6aABA/+ApQnV1dSovL9epU6dUU1OjO3fuqLS0tMv3qp955hm1tLTEb4cPH07pogEA/YOnNyb87W9/S/h6586dysnJ0dmzZzV79uz4/X6/X6FQKDUrBAD0W4/1mlB7e7skKTs7O+H+2tpa5eTkaPz48Vq+fLlaW1u/9PeIxWKKRqMJNwDAwJB0hJxzqqys1KxZs1RUVBS/v6ysTLt379bRo0e1detWnT59WvPmzVMsFuv296murlYwGIzf8vPzk10SACDD+JxzLpnB8vJyHTp0SCdOnNCYMWO+dLuWlhYVFBTo3Xff1aJFi7o8HovFEgIVjUYJEfq973znO55n/vSnP3memThxoucZSbp8+XJSc8Dntbe3Kysrq8dtkvqw6qpVq3Tw4EEdP368xwBJUjgcVkFBgRobG7t93O/3y+/3J7MMAECG8xQh55xWrVql/fv3q7a2VoWFhQ+daWtrU3Nzs8LhcNKLBAD0T55eEyovL9cf/vAH7dmzR4FAQJFIRJFIRLdu3ZIk3bx5U2vWrNE//vEPXb58WbW1tVqwYIFGjx6tZ599Ni1/AABA5vJ0JrRjxw5JUklJScL9O3fu1LJlyzR48GA1NDRo165dunHjhsLhsObOnau9e/cqEAikbNEAgP7B87fjejJixAgdOXLksRYEABg4kn53XLpEo1EFg0HrZQAAHtOjvDuOC5gCAMwQIQCAGSIEADBDhAAAZogQAMAMEQIAmCFCAAAzRAgAYIYIAQDMECEAgBkiBAAwQ4QAAGaIEADADBECAJghQgAAM0QIAGCGCAEAzBAhAIAZIgQAMEOEAABmiBAAwAwRAgCYIUIAADNECABgps9FyDlnvQQAQAo8yt/nfS5CHR0d1ksAAKTAo/x97nN97NTj3r17unr1qgKBgHw+X8Jj0WhU+fn5am5uVlZWltEK7bEf7mM/3Md+uI/9cF9f2A/OOXV0dCgvL0+DBvV8rjOkl9b0yAYNGqQxY8b0uE1WVtaAPsgeYD/cx364j/1wH/vhPuv9EAwGH2m7PvftOADAwEGEAABmMipCfr9fGzdulN/vt16KKfbDfeyH+9gP97Ef7su0/dDn3pgAABg4MupMCADQvxAhAIAZIgQAMEOEAABmMipCb775pgoLCzV8+HBNmTJFH374ofWSelVVVZV8Pl/CLRQKWS8r7Y4fP64FCxYoLy9PPp9PBw4cSHjcOaeqqirl5eVpxIgRKikp0fnz520Wm0YP2w/Lli3rcnxMnz7dZrFpUl1drWnTpikQCCgnJ0cLFy7UxYsXE7YZCMfDo+yHTDkeMiZCe/fuVUVFhTZs2KD6+no9/fTTKisr05UrV6yX1qsmTJiglpaW+K2hocF6SWnX2dmpSZMmafv27d0+vmXLFm3btk3bt2/X6dOnFQqFNH/+/H53HcKH7QdJeuaZZxKOj8OHD/fiCtOvrq5O5eXlOnXqlGpqanTnzh2Vlpaqs7Mzvs1AOB4eZT9IGXI8uAzxzW9+07344osJ933ta19z69atM1pR79u4caObNGmS9TJMSXL79++Pf33v3j0XCoXc66+/Hr/vs88+c8Fg0P32t781WGHv+OJ+cM65pUuXuu9973sm67HS2trqJLm6ujrn3MA9Hr64H5zLnOMhI86Ebt++rbNnz6q0tDTh/tLSUp08edJoVTYaGxuVl5enwsJCPf/887p06ZL1kkw1NTUpEokkHBt+v19z5swZcMeGJNXW1ionJ0fjx4/X8uXL1draar2ktGpvb5ckZWdnSxq4x8MX98MDmXA8ZESErl27prt37yo3Nzfh/tzcXEUiEaNV9b7i4mLt2rVLR44c0VtvvaVIJKKZM2eqra3NemlmHvz3H+jHhiSVlZVp9+7dOnr0qLZu3arTp09r3rx5isVi1ktLC+ecKisrNWvWLBUVFUkamMdDd/tBypzjoc9dRbsnX/zRDs65Lvf1Z2VlZfFfT5w4UTNmzNBTTz2ld955R5WVlYYrszfQjw1JWrx4cfzXRUVFmjp1qgoKCnTo0CEtWrTIcGXpsXLlSn388cc6ceJEl8cG0vHwZfshU46HjDgTGj16tAYPHtzlXzKtra1d/sUzkIwaNUoTJ05UY2Oj9VLMPHh3IMdGV+FwWAUFBf3y+Fi1apUOHjyoY8eOJfzol4F2PHzZfuhOXz0eMiJCw4YN05QpU1RTU5Nwf01NjWbOnGm0KnuxWEwXLlxQOBy2XoqZwsJChUKhhGPj9u3bqqurG9DHhiS1tbWpubm5Xx0fzjmtXLlS+/bt09GjR1VYWJjw+EA5Hh62H7rTZ48HwzdFePLuu++6oUOHurffftv961//chUVFW7UqFHu8uXL1kvrNS+//LKrra11ly5dcqdOnXLf/e53XSAQ6Pf7oKOjw9XX17v6+nonyW3bts3V19e7//znP845515//XUXDAbdvn37XENDg3vhhRdcOBx20WjUeOWp1dN+6OjocC+//LI7efKka2pqcseOHXMzZsxwTz75ZL/aDz/5yU9cMBh0tbW1rqWlJX779NNP49sMhOPhYfshk46HjImQc8698cYbrqCgwA0bNsxNnjw54e2IA8HixYtdOBx2Q4cOdXl5eW7RokXu/Pnz1stKu2PHjjlJXW5Lly51zt1/W+7GjRtdKBRyfr/fzZ492zU0NNguOg162g+ffvqpKy0tdU888YQbOnSoGzt2rFu6dKm7cuWK9bJTqrs/vyS3c+fO+DYD4Xh42H7IpOOBH+UAADCTEa8JAQD6JyIEADBDhAAAZogQAMAMEQIAmCFCAAAzRAgAYIYIAQDMECEAgBkiBAAwQ4QAAGaIEADAzP8DIA8bPIO4DwoAAAAASUVORK5CYII=","text/plain":["<Figure size 640x480 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":["!pip install numpy\n","!pip install matplotlib\n","\n","import numpy as np\n","import matplotlib.pyplot as plt\n","\n","%matplotlib inline \n","img = np.array(x_test[999], dtype='float')\n","pixels = img.reshape((28, 28))\n","plt.imshow(pixels, cmap='gray')\n","plt.show()"]},{"cell_type":"code","execution_count":42,"metadata":{"id":"de631607-92f1-4fc5-92d8-0243fcb7ab0b"},"outputs":[],"source":["scoring_data = {\"input_data\": [{\"values\": [x_test[999].tolist()]}]}\n","predictions = wml_client.deployments.score(deployment_uid, scoring_data)"]},{"cell_type":"markdown","metadata":{"id":"6975152c-faf4-495a-81f7-307f6cf523a8"},"source":["Please note that the scoring response stored in the predictions variable, should contain the fields `prediction, prediction_classes(index of the predicted class) and probability` and their corresponding values."]},{"cell_type":"code","execution_count":43,"metadata":{"id":"4a5b07f8-10d7-49b2-9657-e16b7358de88"},"outputs":[{"name":"stdout","output_type":"stream","text":["{\n","  \"predictions\": [\n","    {\n","      \"id\": \"dense_1\",\n","      \"fields\": [\n","        \"prediction\",\n","        \"prediction_classes\",\n","        \"probability\"\n","      ],\n","      \"values\": [\n","        [\n","          [\n","            0.08351116627454758,\n","            0.0708337128162384,\n","            0.10256966203451157,\n","            0.10903555899858475,\n","            0.11152926832437515,\n","            0.07864179462194443,\n","            0.0917503759264946,\n","            0.10757662355899811,\n","            0.1082470640540123,\n","            0.13630476593971252\n","          ],\n","          9,\n","          [\n","            0.08351116627454758,\n","            0.0708337128162384,\n","            0.10256966203451157,\n","            0.10903555899858475,\n","            0.11152926832437515,\n","            0.07864179462194443,\n","            0.0917503759264946,\n","            0.10757662355899811,\n","            0.1082470640540123,\n","            0.13630476593971252\n","          ]\n","        ]\n","      ]\n","    }\n","  ]\n","}\n"]}],"source":["print(json.dumps(predictions, indent=2))"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"9da9f44e-c20b-4c53-a5f4-7030ea11fb3b"},"outputs":[],"source":["wos_client.data_sets.show_records(payload_data_set_id)"]},{"cell_type":"markdown","metadata":{"id":"c3b7f282-2845-4798-8d53-b6ca4c897bd1"},"source":["<a id=\"explainability\"></a>\n","## 4. Explainability"]},{"cell_type":"markdown","metadata":{"id":"bee296ba-0583-43e3-bddf-41984f389335"},"source":["### 4.1 Configure Explainability"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"cd69a77a-4047-40d2-8535-ebb4817c416a"},"outputs":[],"source":["target = Target(\n","    target_type=TargetTypes.SUBSCRIPTION,\n","    target_id=subscription_id\n",")\n","parameters = {\n","    \"enabled\": True\n","}\n","explainability_details = wos_client.monitor_instances.create(\n","    data_mart_id=data_mart_id,\n","    background_mode=False,\n","    monitor_definition_id=wos_client.monitor_definitions.MONITORS.EXPLAINABILITY.ID,\n","    target=target,\n","    parameters=parameters\n",").result\n","\n","explainability_monitor_id = explainability_details.metadata.id"]},{"cell_type":"markdown","metadata":{"id":"beb6e09a-84e0-4417-b152-698a325af790"},"source":["### 4.2 Get explanation for the transaction"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"0bd7a0f2-a382-4544-9bbf-5903ee932e2c"},"outputs":[],"source":["pl_records_resp = wos_client.data_sets.get_list_of_records(data_set_id=payload_data_set_id, limit=1, offset=0).result\n","scoring_ids = [pl_records_resp[\"records\"][0][\"entity\"][\"values\"][\"scoring_id\"]]\n","print(\"Running explanations on scoring IDs: {}\".format(scoring_ids))\n","explanation_types = [\"lime\"]\n","result = wos_client.monitor_instances.explanation_tasks(scoring_ids=scoring_ids, explanation_types=explanation_types, subscription_id=subscription_id).result\n","print(result)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"e726fe0d-f00f-41c2-8227-6c19d0156d1a"},"outputs":[],"source":["explanation_task_id=result.to_dict()['metadata']['explanation_task_ids'][0]\n","explanation=wos_client.monitor_instances.get_explanation_tasks(explanation_task_id=explanation_task_id, subscription_id=subscription_id).result.to_dict()\n","explanation"]},{"cell_type":"markdown","metadata":{"id":"83710305-3c86-4b6e-b750-38209fdf8c1b"},"source":["### The explanation images can be obtained using the cells below"]},{"cell_type":"code","execution_count":49,"metadata":{"id":"d35b71eb-e68b-459b-9eaa-b99c94ab8ae1"},"outputs":[{"data":{"image/jpeg":"/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQgJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCAAcABwDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD5/rp7D4eeK9U0mPU7LRpZbaRWePEiB5FHUrGW3sPoDWFpiW0uq2cd7J5Vo06LNJjO1Cw3H8Bmuv8AiNca/Y/ESfUblZbJw4fTHif5Ft1OIjEw4xgDp3JoA4dlKsVYEMDgg9qSvSfjFYH/AISODVINO8iGa0t/tNwke1J7l1MjN6FsEZx7Z68+bUAFdNovjDXra3i0eNIdVtmYJb2N9ardKjngeWrAlSemFxnPSuZq1puo3WkalbajYy+VdW0glik2htrDkHBBB/GgDsvi/wCILnXPiFqED3LPa6e/2WCLokZUASbR7uGOfp2Arg6dJI80ryyMWd2LMx6knqabQB//2Q==","image/png":"iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAIAAAD9b0jDAAABvElEQVR4Ae1Vu4rCQBR1U4jarI1gKXaWYhEbwUJ7Bf/BQrDzWaTQVvwC/QEfnaWFlYKFLxBstFDwASIqaKPEPRgIQ2aYJLvVglOduefck5k7dyYWy2d8KsCqgNVqLZVK5/NZluXtdttoNLxeL0toOOZwOAqFAuzm83m1Wu31erfb7X6/B4NBwx6UMJ/Pv16vdrttt9sVMhKJXK9XWPt8PkpuIOB0Og+Hw3K5tNlspDwajcJ3sVjkcjkybgiHw2FsPJ1O0+p6vQ4Kg6bUiKAiEoii+Hw+m80mGVTwZrMBGA6HNKVG2KaJRKLT6ex2O1WnAWgGTYScsk2Px+P3e5BSEqMZyKkGs03L5XIoFOp2u/F4XJOgTC+XC7pCkiQmyzbt9/to0kAgUKvVYrEYnen3+0ejkcvloimdCE5/vV7joCuVCvJRD0EQZrPZ+/Dl6XSqk8+hs9msclMfj8dgMFAcATgpXxyOpIrFIp4C3LFkMul2u7Fkkv0rHo/Hp9OJ7/KbD04mE76paRY1TaVSptP4CTD1eDx8jYnt48VarVb7/R7NwDc1weKOYZmZTEY3x8RK8S/Ay99qtXRN/4/gByogwiIAoJ2BAAAAAElFTkSuQmCC","text/plain":["<PIL.PngImagePlugin.PngImageFile image mode=RGB size=28x28>"]},"execution_count":49,"metadata":{},"output_type":"execute_result"}],"source":["!pip install Pillow\n","from PIL import Image\n","import base64\n","import io\n","\n","time.sleep(10)\n","\n","img = explanation[\"entity\"]['explanations'][0]['predictions'][0][\"explanation\"][0][\"full_image\"]\n","img_data = base64.b64decode(img)\n","Image.open(io.BytesIO(img_data))"]},{"cell_type":"code","execution_count":64,"metadata":{"id":"6ac7d1a8-05ed-4c81-a4d1-54fe4084bf3f"},"outputs":[{"data":{"image/jpeg":"/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQgJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCAAcABwDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD5/qSaCSDYJV2l0DgEjOD0yO2evPYg9CKjrZ8Wa5/wknirUNWWPyop5f3Me0DZEoCxrgccIqj8KAMainvI0gQNj5F2jAAOPf169/YdqZQAUUUUALn5QO386SiigD//2Q==","image/png":"iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAIAAAD9b0jDAAAAiUlEQVR4Ae2SsQnAIBREJYWglYVLuJyt2jqT6ziAK4hoAmn9PyAWEb7lvTs5js8YPVpgtgDnPMY4I6ualDKEsJoGct77McYUGmOm+rdYSsk5Qz7nHIQwvfdurcUcC6zWupB6IxeUTClB6DRdCHFOZa315rL7f1RKIR3Bk0IyD2qt4QaitMDvF7gBkmUbwmjhhGUAAAAASUVORK5CYII=","text/plain":["<PIL.PngImagePlugin.PngImageFile image mode=RGB size=28x28>"]},"execution_count":64,"metadata":{},"output_type":"execute_result"}],"source":["img = explanation[\"entity\"]['explanations'][0]['predictions'][1][\"explanation\"][0][\"full_image\"]\n","img_data = base64.b64decode(img)\n","Image.open(io.BytesIO(img_data))"]},{"cell_type":"markdown","metadata":{"id":"d313e247-6f1b-4f2b-9854-a490ea5f317e"},"source":["<a id=\"Quality\"></a>\n","## 5. Quality"]},{"cell_type":"markdown","metadata":{"id":"55e9211c-98be-48db-bf94-2dd1f41c1a62"},"source":["### 5.1 Configure Quality"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"58c8c8b6-c76b-4198-9e27-afebf8af17a1"},"outputs":[],"source":["target = Target(\n","    target_type=TargetTypes.SUBSCRIPTION,\n","    target_id=subscription_id\n",")\n","parameters = {\n","    \"min_feedback_data_size\": 10\n","}\n","quality_details = wos_client.monitor_instances.create(\n","    data_mart_id=data_mart_id,\n","    background_mode=False,\n","    monitor_definition_id=wos_client.monitor_definitions.MONITORS.QUALITY.ID,\n","    target=target,\n","    parameters=parameters\n",").result\n","\n","quality_monitor_id = quality_details.metadata.id"]},{"cell_type":"markdown","metadata":{"id":"b3fa5713-97db-440b-966c-5f48f961532d"},"source":["### 5.2 Store feedback record"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"b00eb33f-79a7-4bbf-a325-446ed7e9377b"},"outputs":[],"source":["feedback_dataset_id = None\n","feedback_dataset = wos_client.data_sets.list(type=DataSetTypes.FEEDBACK, \n","                                                target_target_id=subscription_id, \n","                                                target_target_type=TargetTypes.SUBSCRIPTION).result\n","print(feedback_dataset)\n","feedback_dataset_id = feedback_dataset.data_sets[0].metadata.id\n","if feedback_dataset_id is None:\n","    print(\"Feedback data set not found. Please check quality monitor status.\")"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"267ece20-c35c-4f17-bd72-96d8f98e7b91"},"outputs":[],"source":["feedback = []\n","for i in range(10):\n","    element = {\"scoring_input\":x_test[i].tolist(), \"target\":list(y_test[i]).index(1)}\n","    feedback.append(element)\n","wos_client.data_sets.store_records(feedback_dataset_id, request_body=feedback, background_mode=False)"]},{"cell_type":"markdown","metadata":{"id":"137d698d-28ba-4188-bbeb-9e884d0df369"},"source":["### 5.3 Run Quality monitor"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"478e7318-fffc-4fcb-b389-9155173c36e7"},"outputs":[],"source":["run_details = wos_client.monitor_instances.run(monitor_instance_id=quality_monitor_id, background_mode=False).result"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"ca6efcb0-70a9-447b-8fe7-d186140180d5"},"outputs":[],"source":["wos_client.monitor_instances.show_metrics(monitor_instance_id=quality_monitor_id)"]}],"metadata":{"kernelspec":{"display_name":"Python 3.10","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.13"}},"nbformat":4,"nbformat_minor":4}