In this code pattern, we will analyse sellers quality by understanding the sentiments and emotions of reviews given by customers to the sellers using Watson Natural Language Understanding on Cloud Pak for Data or IBM Cloud and give a more impactful seller rating to the sellers.
In any E-Commerce website the product sellers have a rating between 0 to 5 stars which are explicitly given by the customers based on the product that have been purchased. Considering this rating into account the customers feel confident enough to purchase products from the particular seller.
The rating which is given to a seller on e-commerce platform is just a rating given explicitly by the customers and does not make more impact to the sellers quality. We are adding more parameters to this rating to make it stronger and more impactful. We read the product reviews of a seller and analyse the sentiment and emotion behind the review with Watson Natural Language Understanding and compute a score, we also analyse the delivery status of the product and whether the product is delivered on or before the estimated date of delivery and compute another score, finally we sum up these scores to get a seller quality rating between 0 to 5 stars which makes more impact to the seller ratings.
When you have completed this code pattern, you will understand how to:
- Use advanced NLP to analyze text and extract meta-data from content such as sentiment, emotion, relations, etc.
- Run small pieces of code to process your data and immediately view the results with Jupyter Notebook.
- Use Data Refinery to prepare training data for an Machine Learning task.
- Build Interactive dashboards and produce visualizations directly from your data in real-time with Embedded Dashboard.
-
Create a connection for the refined data in db2 into IBM Watson Studio project in Cloud Pak for Data or IBM Cloud.
-
Setup Jupyter Notebook that reads the dataset from the IBM db2 Connection.
-
Run the Algorithm from Jupyter notebook that computes the seller rating with the help of Watson Natural Language Understanding on Cloud Pak for Data or IBM Cloud.
-
Visualise insights from the data using Watson Embedded Dashboard on Cloud Pak for Data or IBM Cloud.
-
Implementation of this Code Pattern requires a basic understanding of Data Refinery and it also makes use of the dataset created in the tutorial Collect, cleanse, and enhance your data. Please complete the above tutorial before implementing this code pattern.
-
Any SQL Database.
In this Code Pattern we demonstrate using Db2 on Cloud Pak for Data
-
Create a Project (Already completed as a part of the Tutorial)
-
Add Db2 Connection to the Project (Already completed as a part of the Tutorial)
In this Code Pattern we are going to use Customised version of Brazilian E-Commerce Public Dataset by Olist that we created in the Tutorial Prepare your Dataset for your ML Models using Data Refinery from Db2.
We are also going to use Consumer Reviews of Amazon Products from Kaggle. Download the dataset from the link below.
After Downloading, Extract the consumer-reviews-of-amazon-products.zip file.
We’ll be using the following files:
Datafiniti_Amazon_Consumer_Reviews_of_Amazon_Products_May19.csv : This dataset is a list of over 28,000 consumer reviews for Amazon products like the Kindle, Fire TV Stick, and more from Datafiniti's Product Database updated between February 2019 and April 2019.
As we don’t have a proper dataset with product reviews and product delivery status, we are cooking up the dataset by assuming that the products from the Customised version of Brazilian E-Commerce Public Dataset by Olist and the Consumer Reviews of Amazon Products are the same and thereby we are merging these 2 sets of datasets as 1 dataset and using the Jupyter Notebook.
We will be using Watson Natural Language Understanding service to read the comments of the customer and analyse the Sentiment and Emotions of the Customers review.
-
Create a Watson Natural Language Understanding service on IBM Cloud.
-
Once the service is created copy the API Key and URL of the service as shown.
NOTE: These credentials are important as it will be used in step 5
IBM Cloud Pak for Data
- Create a Project in Cloud Pak for Data choose an Empty Project.
- Once The Project is Created you will see the below page.
IBM Cloud
- Create Watson Studio service.
-
Then click on Get Started.
-
In Watson Studio click
Create a project > Create an empty projectand name itRetail.
Now that we have created a project, we will start adding components to our project. We will start by adding Db2 Connection to our project first.
IBM Cloud Pak for Data
- Click on Add to Project and select Connection. If you have followed step 2 select Db2 from the list and add the credentials of your provisioned Db2 Instance. If you have a different database then you can select that and fill in the credentials.
- After filling the credentials click on Test Connection to make sure you have entered correct credentials. Finally select Create.
IBM Cloud
- Click on Add to Project and select Connection. If you have followed step 2 select Db2 from the list and add the credentials of your provisioned Db2 Instance. If you have a different database then you can select that and fill in the credentials.
- After filling the credentials click on Create.
NOTE: The Database Credentials will be provided by your Database administrator. If you have provisioned a Db2 instance on Cloud Pak for Data then you can follow the steps here to get the credentials.
We have successfully created a project and added Db2 Connection to our project. We will now add the required credentials to our python Jupyter notebook and run the notebook.
- Add Jupyter notebook to the Project by clicking on Add to Project and select Notebook. Click on New Notebook from URL and paste the below URL and create a Notebook.
- Once the notebook is created we will have to fill in 4 Cells with the following:
- Watson Natural Language Understanding Credentials
- Upload the Consumer Reviews of Amazon Products dataset and insert Pandas Dataframe
- Insert the Customised version of Brazilian E-Commerce Public Dataset by Olist from Db2
- Insert Db2 Credentials into the Notebook
- Insert the credentials copied from step 2 in the cell shown below.
- Click on the Assets tab, and select browse, from the file uploader select the extracted file from step 1 named Datafiniti_Amazon_Consumer_Reviews_of_Amazon_Products_May19.csv from the Consumer Reviews of Amazon Products.
-
Once the Dataset is uploaded, click on the cell which says "Insert Customer Review Dataset here", click on
Insert to code > Insert Pandas Dataframe. You will see the code to read the dataset in the cell. -
Finally replace the variable name to data as shown.
- Select the cell which says "Insert Customer Order Details Dataset here", click on Connections in the assets tab and select
Insert to code > Insert Pandas Dataframefrom your Db2 Connection variable. Select the Schema of your table and choose DERIVEDDATA and click on Select.
NOTE: If you have used Db2 Instance then the schema name is your Username.
- Once the credentials are inserted, replace the variable name to data2 as shown.
-
Select the cell which says "Insert Db2 Connection Credentials here", click on Connections in the assets tab and select
Insert to code > Insert Credentialsfrom your Db2 Connection variable. -
Once the credentials are inserted, replace the variable name to
credentials_1as shown.
After all the Preperations are done, we will run the Jupyter Notebook by Clicking on Cell > Run All as shown.
NOTE: It will take around 20 Min to complete the execution of entire notebook, please be Patient!
The Jupyter Notebook generated the Dataset for customer sentiments on seller quality. Further the data can be visualised in an Interactive Embedded Dashboard with IBM Embedded Dashboard Service.
IBM Cloud Pak for Data
- Add Embedded Dashboard to the Project by clicking on Add to Project and select Dashboard. Click on from file and upload the file
dashboard/dashboard.json
IBM Cloud
-
Create an Embedded Dashboard Service on IBM Cloud.
-
From the IBM Cloud Resources open the Watson Studio and load the project Retail.
-
Add Embedded Dashboard to the Project by clicking on Add to Project and select Dashboard. Click on from file and upload the file
dashboard/dashboard.jsonand select the newly created Embedded service.
- Once the
dashboard.jsonfile is loaded it may ask you to enter username and password. Enter the database username and password used in step 4.
- The Dashboard will give you a prompt to relink the data asset (as the database schema of my Db2 and your Db2 is different) choose the
SELLERQUALITYSCOREand relink.
- You will see a dashboard as shown.
NOTE: If the Dashboard takes along time to load, clear your browsing data & cache then reload.
-
The components in the dashboard are:
- Analysis by Sellers : You can click on any seller name to visualize ratings of that particular seller.
- Analysis by Products : Similar to analysis by sellers, you can click on products name to visualize the particular product's ratings.
- Seller Satisfaction Score : For selected seller and product, the seller satisfaction score can be viewed.
- Products Ratings by Seller Graph : You can see the graph of every sellers with their products and the respected ratings.
- Customers Emotion : For selected Product and seller, along with Seller Satisfaction Score, Customers Emotions can be visualized.
-
Conclusion that can be drawn from the Dashboard Visualization are:
-
For the product Alexa customers emotions were mixed i.e. joy, anger and sad. Based on the sentiments and emotions from the reviews and order delivery status, Watson Natural Language Understanding gave the highest seller satisfaction score of 3.38 for seller3 which can be visualized by selecting Alexa and Seller3 as shown.

-
For the product Batteries customers emotions were joy. Based on the sentiments and emotions from the reviews and order delivery status, Watson Natural Language Understanding gave the highest seller satisfaction score of 3.53 for seller4 which can be visualized by selecting Batteries and Seller4 as shown.

-
For the product Fire HD customers emotions were mixed i.e. joy, disgust and sad. Based on the sentiments and emotions from the reviews and order delivery status, Watson Natural Language Understanding gave the highest seller satisfaction score of 3.27 for seller4 which can be visualized by selecting Fire HD and Seller4 as shown.

-
For the product Kindle customers emotions were joy. Based on the sentiments and emotions from the reviews and order delivery status, Watson Natural Language Understanding gave the highest seller satisfaction score of 3.29 for seller2 which can be visualized by selecting Batteries and Seller2 as shown.

-
For the product Tablet 8in customers emotions were mixed i.e. joy, disgust and fear. Based on the sentiments and emotions from the reviews and order delivery status, Watson Natural Language Understanding gave the highest seller satisfaction score of 3.22 for seller3 which can be visualized by selecting Tablet 8in and Seller3 as shown.

-
-
Hence the following products can be brought from the following sellers.
Product Seller to buy from Seller Rating Alexa Seller3 3.38 Batteries Seller4 3.53 Fire HD Seller4 3.27 Kindle Seller2 3.29 Tablet 8in Seller3 3.22 -
Further if you want to play around with the dashboard you can do so by reffering to the following tutorial.
This code pattern is licensed under the Apache License, Version 2. Separate third-party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 and the Apache License, Version 2.






















