When the reader has completed this, they will understand how to:
- Connect to Mobile First Foundation using a mobile application.
- Take inputs from mobile application and do required processing on IBM Cloud.
- Use Watson Visual Recognition using a mobile application.
- Deploy and use cloud foundry applications.
- Access images from Cloud Object Storage using a mobile application.
- Connect and access Db2 on Cloud.
- Take input from user's mobile.
- The input is passed via Mobile First Foundation.
- Mobile First Foundation passes the user's input to the visual recognition application.
- The visual recognition application interacts with the Watson Visual Recognition service, which returns appropriate output.
- The output(age and gender of user) is then send to user's mobile application.
- The user clicks on "Get Recommendations" button.
- Requests the recommendation engine to return appropriate recommendation based the visual recognition applications output.
- Recommendation engine interacts with IBM Db2 to get the necessary product details for the recommended products.
- Images of the recommended products is retrieved from Cloud Object Storage.
- Images and details of the recommended products is retrieved by the recommendation engine.
- Recommendation engine returns the images and details of the recommended products to the user's mobile application.
- User can now add items to the cart and view them.
- IBM Cloud account : Create an IBM Cloud account.
- Python 3: Install python 3.
- Java 1.8.x: Make sure you have required version (Java 1.8.x).
Please follow the below to setup and run this code pattern.
Clone this git repo. Else, in a terminal, run:
$ git clone https://github.com/IBM/recommender-with-watson-visual-recognition.git
In this step we will be building a recommendation engine which takes users's age and gender as input ,and gives out a recommendation accordingly.
We use IBM Cloud Object Storage to store the jewellery images required for recommendation and the dataset.
- In the
IBM Cloud Dashboard, click onCatalogand selectObject Storageservice underInfrastructure->Storage. Click onCreateas shown below.
- The IBM Cloud Object Storage dashboard will get shown. In the
Bucketstab, click onCreate bucket. Give an unique name for the bucket. Set the selections for Resiliency (Cross Region), Location (us-geo) and Storage class (Standard), and click onCreateas shown below.
Note: Make a note of the Bucket Name as it is Important and will be used in step 4.4.2
-
Create Service ID
- In a separate browser tab/window, launch the IBM Cloud Identity & Access Management dashboard using URL https://cloud.ibm.com/iam/.
- In case you have multiple IBM Cloud accounts, then select the target Account, Region, Organization and Space.
- Under
Identity & Access(on the left side of the page), selectService IDsand clickCreateon the right top of the page. Give a name and description, and click Create. - Make a note of the name of the Service ID as shown below.
Note: Make a note of the name of the Service ID as it is Important and will be used in step 4.4.2
-
Add Cloud Object Storage Writer role to that service ID
- Back in IBM Cloud Object Storage dashboard, select
Bucket permissionsunderBucketsclick onpolicies. - Click on
Service IDstab. UnderSelect a service ID, select the service ID created in the above step. UnderAssign a role to this service ID for this bucket, selectWriter. Click Create policy as shown below.
- Back in IBM Cloud Object Storage dashboard, select
You should get a confirmation dialog saying “Service permission created“.
-
Create API Key
- Back in IBM Cloud Identity & Access Management dashboard, under
Service IDs, click on the service ID created earlier. UnderAccess policies, you should see theWriterrole for your bucket. - Click on
API keystab and then click onCreatebutton. In theCreate API keydialog, give a name and description for the API key and click onCreate. You should get a confirmation dialog sayingAPI key successfully createdas shown below. - Click on
Downloadand save the API key as shown below. Note: This is the only time you will see the key. You cannot retrieve it later. - You can now close the tab.
- Back in IBM Cloud Identity & Access Management dashboard, under
Note: Make a note of the API Key as it is Important and will be used in step 4.4.2
To access the Cloud Object Storage service programmatically, you need to copy in your credentials, which you can find in your IBM Cloud Object Storage service credentials in IBM Cloud.
-
Open your IBM Cloud Data Resource list. A list of your provisioned resources is displayed.
-
Locate your Cloud Object Storage instance under
Storagetab and click on that. -
Open the
Service Credentialstab on the right left side of the page, clicknew credentialand give a name. -
View your credentials by clicking
View Credentials.
-
Copy your credentials. Create a file
credentials1.jsonand paste the copied credentials into this file.Place this file in the directoryJewelleryRecommendationand also in the directoryUploadProductsCOS. -
Replace
xxxxxxin the place holderbucket_namewith your corresponding bucket name in the fileKMeans_200.py.
- Replace
xxxxxxin the place holderbucket_namewith your corresponding bucket name in the fileupload.py. - Run the file
upload.pylocally to upload images and dataset to Cloud Object Storage.
$ python3 upload.py
- Create a IBM Db2 instance IBM db2.
- Lanch your Db2 on cloud and click on
load, as shown below.
- Click on
browse filesand uploaddata.csv, as shown below.data.csvcan be found in the root folder ofProductDetailsDB2.
- Choose the default schema and create a table
PRODUCTS, as shown below.
- Click on
Next, as shown below.
-
Click on
Next. -
Click on
Begin Load, as shown below.
- Once the data is loaded, you can view the table which will look like the image, shown below.
NOTE: Make sure you note down the table name. In my case the table name is
ZJN44169.PRODUCTS.
- Replace the placeholder
username,password,sg_service_url,database,host,portundercredentials_1in the fileKMeans_200.py. - Replace
XXXX.YYYYin the place holderinsertwith your corresponding table name in the fileKMeans_200.py.
NOTE: You can get username, password, sg_service_url, hostname, port number and Database credentials by creating/clicking New Credentials from your Db2 service instance on cloud.
- Create a cloud foundry instance IBM Cloud Foundry Service and follow set of instructions for deploying python application to IBM Cloud Foundry.
NOTE: Make Sure the Cloud Foundry App gets at least 256MB of Memory. You can verify it by going to IBM Cloud Dashboard > Resources > Cloud Foundry Apps > YOUR_APP_NAME.
-
Use IBM Cloud command line interface to download, modify, and redeploy your Cloud Foundry applications and service instances.
-
Before you begin, download and install the IBM Cloud CLI.
-
After you install the command line interface, you can get started.
-
Change to the directory.
$ cd JewelleryRecommendation
Note : Make sure that
KMeans_200.py,credentials1.json,requirements.txt,manifest.ymlandProcfileis present in the directoryJewelleryRecommendation.
- Connect and log in to IBM Cloud.
$ ibmcloud api https://api.eu-gb.bluemix.net
$ ibmcloud login -u example@in.ibm.com -o example@in.ibm.com -s dev
NOTE: If you are using a federated ID, use the
-ssooption.
$ ibmcloud login -o example@in.ibm.com -s dev -sso
NOTE: You must add single or double quotes around
username,org_name, andspace_nameif the value contains a space, for example,-o "my org".
- Finally Deploy the application by following command.
$ ibmcloud cf push YOUR-APP-NAME
Example: ibmcloud cf push recommendation-engine
-
Once you have deployed the application Make a note of the
URLof the instance by right clicking on theVisit app URLand copying the link.
Note: This URL is Important as it will be used in step 4.4.2
To Test your deployment use any REST Clients like Postman. After Installing postman type https://YOUR-APP-URL/?age=40&name=Kavya&gender=F to test whether Recommendation engine works.
- Now click on
Sendbutton to run theGET /API. The API response should be shown in theResponse Bodyas shown in snapshot below.
The Watson Visual Recognition service take the user's Image as Input and predicts the age and gender of the user.
-
Create a Visual Recognition service in the IBM Cloud.
-
Copy the Api Key from the Visual Recognition Dashboard as shown bellow.
-
Go to the
VisualRecognitiondirectory.
$ cd VisualRecognition
- In the
credentials.jsonfile, for the keyvrapipaste the copied Api key value in the place ofYOUR-API-KEY-HEREas shown bellow.
{
"vrapi": "YOUR-API-KEY-HERE"
}
- Create a cloud foundry instance IBM Cloud Foundry Service and follow set of instructions for deploying JavaScript application to IBM Cloud Foundry.
NOTE: Make Sure the Cloud Foundry App gets at least 256MB of Memory. You can verify it by going to IBM Cloud Dashboard > Resources > Cloud Foundry Apps > YOUR_APP_NAME.
-
Use IBM Cloud command line interface to download, modify, and redeploy your Cloud Foundry applications and service instances.
-
Before you begin, download and install the IBM Cloud CLI.
-
After you install the command line interface, you can get started.
-
Go to the
VisualRecognitiondirectory.
$ cd VisualRecognition
- Connect and log in to IBM Cloud.
$ ibmcloud api https://api.eu-gb.bluemix.net
$ ibmcloud login -u example@in.ibm.com -o example@in.ibm.com -s dev
NOTE: If you are using a federated ID, use the
-ssooption.
$ ibmcloud login -o example@in.ibm.com -s dev -sso
NOTE: You must add single or double quotes around
username,org_name, andspace_nameif the value contains a space, for example,-o "my org".
- Finally Deploy the application by following command.
$ ibmcloud cf push YOUR_APP_NAME
Example: ibmcloud cf push watsonvr
-
Once you have deployed the application Make a note of the
URLof the instance by right clicking on theVisit app URLand copying the link.
Note: This URL is Important as it will be used in step 4.4.2.
The Mobile Application is the component that connects Virtual Mirror and Recommendation Engine.
- Install
Node.jsby downloading the setup from https://nodejs.org/en/ (Node.js 8.x or above)
$ node --version
v10.15.0
- Install Cordova
$ sudo npm install -g cordova@8.1.2
$ cordova --version
8.1.2
Note: Please refer MFP documentation for compatible versions of Cordova - https://mobilefirstplatform.ibmcloud.com/tutorials/en/foundation/8.0/application-development/sdk/cordova/
- Install Ionic
$ sudo npm install -g ionic@4.12.0
$ ionic --version
4.12.0
- Install IBM MobileFirst Platform CLI
$ sudo npm install -g mfpdev-cli
$ mfpdev --version
8.0.0-2018121711
Note: If you are on Windows, instead of using sudo, run the above command without sudo in a command prompt opened in administrative mode.
Note: While installing MFP CLI, if you hit an error saying
npm ERR! package.json npm can't find a package.json file in your current directory., then it is most likely due to MFP CLI not being supported in your npm version. In such a case, downgrade your npm as below, and then install MFP CLI.$ sudo npm install -g npm@3.10.10
- Install Java SDK 8 from https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
$ java -version
java version "1.8.0_101"
Note: Java version
1.8.xis required for cordova to compile apks. Do not Download Java version11.x. If you already have java version above1.8.xthen you can follow the guide inTROUBLESHOOTING.mdto uninstall the java and reinstall1.8.x.
- Install Maven:
On Mac, you can use
brew installfor installing Maven as shown below:
$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
$ brew install maven
$ mvn --version
Apache Maven 3.6.0 ...
On Windows, you can follow this Tutorial to install Maven.
- Install Gradle:
On Mac, you can use
brew installfor installing Maven as shown below:
$ brew install gradle
$ gradle --version
Gradle 5.1.1 ...
On Windows, you can follow this Tutorial to install Gradle.
-
In the IBM Cloud Dashboard, click on
Catalogand select Mobile Foundation service underPlatform->Mobile. Click onCreateas shown below.
-
In the Mobile Foundation service overview page that gets shown, click on
Service credentials. ExpandView credentialsand make a note of theurl,userandpasswordas shown below.
NOTE: The
user,passwordandurlis Important as it will be used in subsequent steps.
NOTE: Make Sure the Cloud Foundry App for Mobile Foundation-Server gets at least 768MB of Memory.(Recommended is 1GB) You can verify it by going to IBM Cloud Dashboard > Resources > Cloud Foundry Apps > MobileFoundation-Server as shown below.
Note: If Mobile Foundation service is not available with your current account type, then you can either:
- Upgrade your account, and avail the Mobile Foundation service's free Developer plan which allows the use of the service free for up to ten daily client devices for development and testing activities, or
- Back on your local machine, configure MFP CLI to work with Mobile Foundation server by running the following command in console.
Note: For
Enter the fully qualified URL of this server:, enter theurlmentioned in credentials followed by:443(the default HTTPS port).
$ mfpdev server add
- Follow the Instructions.
? Enter the name of the new server profile: MyServer
? Enter the fully qualified URL of this server: https://mobilefoundation-xxxx-xxxxx.xx-xx.mybluemix.net:443
? Enter the MobileFirst Server administrator login ID: admin
? Enter the MobileFirst Server administrator password: **********
? Save the administrator password for this server?: Yes
? Enter the context root of the MobileFirst administration services: mfpadmin
? Enter the MobileFirst Server connection timeout in seconds: 30
? Make this server the default?: Yes
Verifying server configuration...
The following runtimes are currently installed on this server: mfp
Server profile 'MyServer' added successfully.
- Next Verify If the Server is added.
$ mfpdev server info
Name URL
---------------------------------------------------------------------------------------
MyServer https://mobilefoundation-xxxx-xxxxxx.xx-xx.mybluemix.net:443 [Default]
---------------------------------------------------------------------------------------
Note: If this step fails check
TROUBLESHOOTING.mdto fix commonly occuring errors.
NOTE: This URL is Important as it will be required in subsequent Steps.
- Go to the
JewelleryStoreAppdirectory.
$ cd JewelleryStoreApp
- Update App ID, Name and Description
in
JewelleryStoreApp/config.xmlas below. Changeid,name,descriptionandauthordetails as shown bellow.
<?xml version='1.0' encoding='utf-8'?>
<widget id="com.ibm.mfpthejewellerystore" version="1.0.0" xmlns="http://www.w3.org/ns/widgets" xmlns:cdv="http://cordova.apache.org/ns/1.0" xmlns:mfp="http://www.ibm.com/mobilefirst/cordova-plugin-mfp">
<name>The Jewellery Store</name>
<description>A virtual mirror integration into ecommerce products with the help of mobilefirst foundation.</description>
<author email="example@in.ibm.com" href="/">Code Patterns Team </author>
...Specify Cloud Object Storage credentials in MFP Adapter
Recommendation Engine API & Virtual Mirror API in MFP Adapter
- Go to the
MobileFoundationAdapterdirectory insideJewelleryStoreAppdirectory.
$ cd MobileFoundationAdapter
$ cd ImagesFetch
- Add the
URLalong with the port number:443appended with/mfpadmin,UserandPasswordin thepom.xmlfile which is present inImagesFetchdirectory as show bellow.
...
<mfpfUrl>https://mobilefoundation-xxxx-xxxxxx.xx-xx.mybluemix.net:443/mfpadmin </mfpfUrl>
<mfpfUser>admin </mfpfUser>
<mfpfPassword>******** </mfpfPassword>
<mfpfRuntime>mfp </mfpfRuntime>
...
- Build the
ImagesFetchadapter as shown below.
$ mfpdev adapter build
Building adapter...
Successfully built adapter
- Deploy the adapter as shown bellow.
$ mfpdev adapter deploy
Verifying server configuration...
Deploying adapter to runtime mfp on https://mobilefoundation-xxxx-xxxxxx.xx-xx.mybluemix.net:443/mfpadmin...
Successfully deployed adapter
Note: In [Step 4.2], if you specified
NotoMake this server the default?, then you need to specify the name of your server profile (MyServerin our case) at the end ofmfpdev adapter deploycommand as shown below.
$ mfpdev adapter deploy MyServer
Launch MFP Dashboard as below:
- In the IBM Cloud dashboard, under
Cloud Foundry Services, click on theMobile Foundationservice you created in [Step 4.2]. The service overview page that gets shown, will have the MFP dashboard embedded within it. You can also open the MFP dashboard in a separate browser tab by appending/mfpconsoleto the urlhttps://mobilefoundation-xxxx-xxxxx.xx-xx.mybluemix.net.
Example:
https://mobilefoundation-xxxx-xxxxx.xx-xx.mybluemix.net/mfpconsole
NOTE:
username&passwordcan be found inService credentialsin step 4.2.
- Inside the MFP dashboard, in the list on the left, you will see the
VisualRecognitionadapter listed.
Update MFP Adapter configuration as below:
-
Inside the MFP dashboard, click on the
ImagesFetchadapter. UnderConfigurationstab, you should see the various properties for accessing Cloud Object Storage, recommendation Engine Api and Visual Recognition Api as shown below.
-
The
Cloud Object Storage Bucket Namecan be found in step 2.1.1,Cloud Object Storage API Keycan be found in step 2.1.2,Cloud Object Storage Endpointcan be found by going toIBM Cloud Dashboard > Resources > Storage > Cloud Object Storageand clicking onEndpointand the public link for Resiliency and Location as selected in step 2.1.1 andCloud Object Storage Service IDcan be found in step 2.1.2. -
The
Recommendation Engine API URLis your python app url and can be found in step 2.7,Visual Recognition API URLis your NodeJs app url and can be found in step 4.2. -
Save the changes by clicking
Save. -
Click on
Resourcestab. You should see the various REST APIs exposed byVisualRecognitionadapter as shown below.
To Test the adapter use any REST Clients like Postman.
After Installing postman type the url created in [step 4.2] and append it with /mfp/api/adapters/ImagesFetch/resource and /objectStorage to test whether the adapter is establishing connection with Cloud Object Storage.
Example:
https://mobilefoundation-xxxx-xxxxxx.xx-xx.mybluemix.net/mfp/api/adapters/ImagesFetch/resource/objectStorage
-
Now click on
Sendbutton to run theGET /API. The API response should get shown in theResponse Bodyas shown in snapshot below. -
The GET API on
/objectStorageshould return a JSON object containingbaseUrlandauthorizationHeaderas shown below.
- The GET API on
/recommendationEngineshould return a JSON object containingrecommendationEngineApias shown below.
Example:
https://mobilefoundation-xxxx-xxxxxx.xx-xx.mybluemix.net/mfp/api/adapters/ImagesFetch/resource/recommendationEngine
- The GET API on
/visualRecognitionshould return a JSON object containingVisualRecognitionApias shown below.
Example:
https://mobilefoundation-xxxx-xxxxxx.xx-xx.mybluemix.net/mfp/api/adapters/ImagesFetch/resource/visualRecognition
- Download and install Android Studio from https://developer.android.com/studio/
- Install Android SDK Platform 23 (or higher) as below:
- Launch Android Studio.
- Click on
Configure->SDK Manager - Make a note of the
Android SDK Location. - Under
SDK Platforms, selectAndroid 6.0 (Marshmallow) API Level 23or higher. ClickApplyand then clickOK. This will install Android SDK Platform on your machine.
- Enable USB debugging on your Android phone as per the steps in https://developer.android.com/studio/debug/dev-options
- Launch the Settings app on your phone. Select
About Device->Software Info. TapBuild number7 times to enable developer options. - Return to Settings list. Select
Developer optionsand enableUSB debugging.
- Launch the Settings app on your phone. Select
- If you are developing on Windows, then you need to install the appropriate USB driver as per instructions in https://developer.android.com/studio/run/oem-usb.
- Connect the Android phone to your development machine by USB cable, you will get a prompt displaying adb access required,
allowthe access.
Note: If you have android adb tools you can check whether your device is connected or not by entering
adb devices.
- Go back to
JewelleryStoreAppdirectory.
$ cd ../JewelleryStoreApp
- Register the app as Shown bellow.
$ mfpdev app register
Verifying server configuration...
Registering to server:'https://mobilefoundation-xxxx-xxxxxx.xx-xx.mybluemix.net:443' runtime:'mfp'
Updated config.xml file located at: .../Ionic-MFP-App/IonicMobileApp/config.xml
Run 'cordova prepare' to propagate changes.
Registered app for platform: android
Note: In [Step 4.2], if you specified
NotoMake this server the default?, then you need to specify the name of your server profile (MyServerin our case) at the end ofmfpdev app deploycommand as shown below.$ mfpdev app register MyServer
Note: To Propagate changes by running
cordova prepare
- Build Android application
$ ionic cordova build android
Note: The build & run commands should be executed in the JewelleryStoreApp directory and not else where.
Note: Make sure you Connect the Android phone to your development machine by USB cable, and accept the adb access permissions.
- Run application on Android device
$ ionic cordova run android
-
Allow the Camera Permission and Storage Permission when prompted. Without this the virtual mirror will not work.
NOTE: Storage Permission will be asked only if you choose to select a picture from camera.
NOTE: If there is not camera prompt in your mobile device follow step 4 from
TROUBLESHOOTING.mdto fix it. -
To get your age and gender either click a picture or choose an existing picture.
-
A list of Jewellery will be Recommended based on your age and gender.
-
You can add any Jewellery to the cart.
Reference: Automating Icons and Splash Screens https://blog.ionicframework.com/automating-icons-and-splash-screens/
Copy your desired app icon to JewelleryStoreApp/resources/icon.png and app splash to JewelleryStoreApp/resources/splash.png.
$ ionic cordova resources
For running ionic cordova resources command, you would need to sign up on ionicframework.com and specify the credentials on the command line.
Reference: https://ionicframework.com/docs/intro/deploying/
- Add following lines at the end of
JewelleryStoreApp/platforms/android/app/src/main/proguard-project-mfp.txt:
-dontwarn okhttp3.internal.huc.**
- Create release build as below:
$ cd ../JewelleryStoreApp
$ ionic cordova build android --prod --release
- Set
ANDROID_HOMEenvironment variable as per instructions in [Step x.x]. On Mac, this is usually:
export ANDROID_HOME=/Users/<username>/Library/Android/sdk
- Zip align release build as below:
$ cd ./platforms/android/build/outputs/apk/
$ ls
android-release-unsigned.apk
$ $ANDROID_HOME/build-tools/28.0.3/zipalign -v -p 4 android-release-unsigned.apk android-release-unsigned-aligned.apk
$ ls
android-release-unsigned-aligned.apk android-release-unsigned.apk
- Create self signing certificate as below:
Make a note of the Keystore password that you set. You would need it for signing your APK.
$ keytool -genkey -v -keystore my-release-key.jks -keyalg RSA -keysize 2048 -validity 10000 -alias my-alias
Enter keystore password:
Re-enter new password:
What is your first and last name?
[Unknown]: xxxxx xxxxxxxxx
What is the name of your organizational unit?
[Unknown]: XXX
What is the name of your organization?
[Unknown]: XXX
What is the name of your City or Locality?
[Unknown]: Xxxxxxxxxx
What is the name of your State or Province?
[Unknown]: Xxxxxxxxx
What is the two-letter country code for this unit?
[Unknown]: XX
Is CN=xxxxx xxxxxxxxx, OU=XXX, O=XXX, L=Xxxxxxxxxx, ST=Xxxxxxxxx, C=XX correct?
[no]: yes
Generating 2,048 bit RSA key pair and self-signed certificate (SHA256withRSA) with a validity of 10,000 days
for: CN=xxxx xxxxxxxxx, OU=XXX, O=XXX, L=Xxxxxxxxxx, ST=Xxxxxxxxx, C=XX
Enter key password for <my-alias>
(RETURN if same as keystore password):
[Storing my-release-key.jks]
$ ls
android-release-unsigned-aligned.apk android-release-unsigned.apk my-release-key.jks
- Self sign APK as below:
$ $ANDROID_HOME/build-tools/28.0.3/apksigner sign --ks my-release-key.jks --out thejewellerystore.apk android-release-unsigned-aligned.apk
Keystore password for signer #1:
$ ls
android-release-unsigned-aligned.apk my-release-key.jks
android-release-unsigned.apk thejewellerystore.apk
$
- Distribute
thejewellerystore.apkby uploading to Google Play Store or to your company's internal App store.
Please see troubleshooting guide for solutions to some commonly occuring problems.
-
Install Google Chrome
-
Open Google Chrome. Open URL
chrome://inspect/#devices -
Connect your mobile phone to the deployment device via USB cable and you should see your device name listed on the page as shown.
-
Under
Devices, click oninspectbelow your connected device.
-
You can see the console logs here for every action that the app performs.






















