Monitor Azure ML With Watson OpenScale

In this Code Pattern, we will use a German Credit dataset to create a logistic regression model using Azure Machine Learning Studio. We will use Watson OpenScale to bind the ML model deployed in the Azure cloud, create a subscription, and perform payload and feedback logging.

When the reader has completed this Code Pattern, they will understand how to:

• Prepare data, train a model, and deploy using Azure Machine Learning Studio
• Score the model using sample scoring records and the scoring endpoint
• Setup Watson OpenScale Data Mart
• Bind the Azure model to the Watson OpenScale Data Mart
• Add subscriptions to the Data Mart
• Enable payload logging and performance monitoring for both subscribed assets
• Use Data Mart to access tables data via subscription

Flow

1. The developer creates a Jupyter Notebook.
2. The Jupyter Notebook is connected to a PostgreSQL database, which is used to store Watson OpenScale data.
3. An ML model is created using Azure ML Studio, using data from credit_risk_training.csv, and then it is deployed to the cloud.
4. Watson Open Scale is used by the notebook to log payload and monitor performance.

Prerequisites

• An IBM Cloud Account
• IBM Cloud CLI
• IBM Cloud Object Storage (COS)
• An account on Azure Machine Learning Studio

Steps

1. Clone the repository
2. Create a Watson OpenScale service
3. Create a Model on Azure Machine Learning Studio
4. Create a notebook in IBM Watson Studio on Cloud Pak for Data
5. Run the notebook

1. Clone the repository

git clone https://github.com/IBM/monitor-sagemaker-ml-with-ai-openscale
cd monitor-sagemaker-ml-with-ai-openscale

2. Create a Watson OpenScale service

Create Watson OpenScale, either on the IBM Cloud or using your On-Premise Cloud Pak for Data.

On IBM Cloud

• If you do not have an IBM Cloud account, register for an account

• Create a Watson OpenScale instance from the IBM Cloud catalog

• Select the Lite (Free) plan, enter a Service name, and click Create.

• Click Launch Application to start Watson OpenScale.

• Click Auto setup to automatically set up your Watson OpenScale instance with sample data.

  

• Click Start tour to tour the Watson OpenScale dashboard.

On IBM Cloud Pak for Data platform

Note: This assumes that your Cloud Pak for Data Cluster Admin has already installed and provisioned OpenScale on the cluster.

• In the Cloud Pak for Data instance, go the (☰) menu and under Services section, click on the Instances menu option.

  

• Find the OpenScale-default instance from the instances table and click the three vertical dots to open the action menu, then click on the Open option.

  

• If you need to give other users access to the OpenScale instance, go the (☰) menu and under Services section, click on the Instances menu option.

  

• Find the OpenScale-default instance from the instances table and click the three vertical dots to open the action menu, then click on the Manage access option.

  

• To add users to the service instance, click the Add users button.

  

• For all of the user accounts, select the Editor role for each user and then click the Add button.

  

3. Create a Model on Azure Machine Learning Studio

Note: Here we provide minimal instructions. It is beyond the scope of this Code Pattern to provide detailed support for the Azure platform.

• Copy credit risk experiment from Azure ML Studio Gallery

  

• Run the experiment to train a model.

  

• Create (deploy) web service.

  

4. Create a notebook in IBM Watson Studio on Cloud Pak for Data

• In Watson Studio or your on-premise Cloud Pak for Data, click New Project + under Projects or, at the top of the page click + New and choose the tile for Data Science and then Create Project.

• Using the project you've created, click on + Add to project and then choose the Notebook tile, OR in the Assets tab under Notebooks choose + New notebook to create a notebook.

• Select the From URL tab. [1]

• Enter a name for the notebook. [2]

• Optionally, enter a description for the notebook. [3]

• For Runtime select the Default Spark Python 3.7 option. [4]

• Under Notebook URL provide the following url: https://raw.githubusercontent.com/IBM/monitor-azure-ml-with-watson-openscale/master/notebooks/WatsonOpenScaleAndAzureMLengine.ipynb

Note: The current default (as of 8/11/2021) is Python 3.8. This will cause an error when installing the pyspark.sql SparkSession library, so make sure that you are using Python 3.7

• Click the Create notebook button. [6]

5. Run the notebook

• Move your cursor to each code cell and run the code in it. Read the comments for each cell to understand what the code is doing. Important when the code in a cell is still running, the label to the left changes to In [*]:. Do not continue to the next cell until the code is finished running.

• Follow the instructions for ACTION: Get Watson OpenScale instance_guid and apikey using the IBM Cloud CLI

How to get api key using ibmcloud console:

ibmcloud login --sso
ibmcloud iam api-key-create 'my_key'

• Enter this IBM Cloud API key in the cell for CLOUD_API_KEY.

• Leave the DB_CREDENTIALS as None to use the internal OpenScale database (recommended). Use the commented out fields to create the DB_CREDENTIALS if you wish to use an external database.

• In your IBM Cloud Object Storage instance, create a bucket with a globally unique name. The UI will let you know if there is a naming conflict.

• In your IBM Cloud Object Storage instance, get the Service Credentials for use as COS_API_KEY_ID, COS_RESOURCE_CRN, and COS_ENDPOINT:

  

• Insert you bucket name in the next cell as BUCKET_NAME.

How to get your Watson OpenScale instance GUID:

ibmcloud resource service-instance <WatsonOpenScale_instance_name>

• In the cell after 2. Bind machine learning engine enter the client_id, client_secret, subscription_id, and tenant for the AZURE_ENGINE_CREDENTIALS.

NOTE: Setting up Azure Active Directory for the AZURE_ENGINE_CREDENTIALS is beyond the scope of this document. See Azure documentation for help with this.

• After running the cell above 2.1 Add deployment ID from asset_deployment_details above you will get a {'metadata': {'guid': '<deployment_id>' to use in the cell below as deployment_id (without the angle brackets '<' and '>').

Sample Output

See the example notebook with output

License

This code pattern is licensed under the Apache License, Version 2. Separate third-party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 and the Apache License, Version 2.

Apache License FAQ