Skip to content
This repository has been archived by the owner on Nov 20, 2019. It is now read-only.

Home

Jump to bottom
Dolph Mathews edited this page Oct 19, 2018 · 5 revisions

Short Name

Continuous Learning with WML and IBM DB2 Warehouse on Cloud

Short Description

Use IBM Watson Machine Learning and Watson Studio, which allows data scientists and analysts to quickly build and prototype models, to monitor deployments, and to learn over time as more data becomes available.

Offering Type

AI

Introduction

Use IBM Watson Machine Learning, Apache Spark and Watson Studio to quickly build and prototype models, to monitor deployments, and to learn over time as more data becomes available. Performance Monitoring and Continuous Learning enables machine learning models to re-train on new data supplied by the user or other data sources. Then, all of your applications and analysis tools which depend on the model are automatically updated as Watson Studio handles selecting and deploying the best model.

Author

By Sanjeev Ghimire and Adam Massachi

Code

https://github.com/IBM/continuous-learning-with-watson-ml-and-db2

Demo

TBD

Video

TBD

Overview

In this code pattern, we will use IBM Watson Machine Learning and Watson Studio, which allows data scientists and analysts to quickly build and prototype models, to monitor deployments, and to learn over time as more data become available. Performance Monitoring and Continuous Learning enable machine learning models to retrain on new data supplied by the user or another data source. Then, all of your applications and analysis tools which depend on the model are automatically updated as Watson Studio handles selecting and deploying the best model.

When the reader has completed this Code Pattern, they will understand how to:

  • Use Watson Studio to create a project and associate services
  • Use IBM Machine learning service to take advantage of machine learning models management (continuous learning system) and deployment (online, batch, streaming)
  • Use Apache Spark-as-a-service cluster computing framework optimized for extremely fast and large scale data processing.
  • Create and deploy self learning Watson Machine learning models

Flow

Architecture diagram

  1. Input Source Data is loaded into the IBM DB2 warehouse on the cloud.
  2. The source data is then added as data asset into Watson Studio.
  3. The Watson Machine Learning service uses the source data and computes the evaluation using Apache Spark-as-a-service to create a machine learning model and saves the evaluation back to the warehouse.
  4. Apache Spark-as-a-service to compute the evaluation.
  5. Feedback data is uploaded to Watson machine learning service to continuously learn and evaluate the new data.
  6. Once the evaluation is done the Watson Machine Learning service creates a machine learning model.
  7. The model data is exposed through an API.
  8. Application can use the API to evaluate new data and create new model based on continuous learning.

Included components

  • Watson Machine Learning: Use trusted data to put machine learning and deep learning models into production. Leverage an automated, collaborative workflow to grow intelligent business applications easily and with more confidencelaborate on building conversational AI solution.
  • Apache Spark: Apache Spark is an open source cluster computing framework optimized for extremely fast and large scale data processing, which you can access via the newly integrated notebook interface IBM Analytics for Apache Spark.
  • IBM DB2 Warehouse: IBM Db2 Warehouse is a software-defined data warehouse for private and virtual clouds that support Docker container technology. It is client managed and optimized for fast and flexible deployment with automated scaling to meet agile analytic workloads.
  • Watson Studio: IBM Watson Studio provides tools for data scientists, application developers and subject matter experts to collaboratively and easily work with data to build and train models at scale. It gives you the flexibility to build models where your data resides and deploy anywhere in a hybrid environment so you can operationalize data science faster.

Featured technologies

  • Artificial Intelligence: Artificial intelligence can be applied to disparate solution spaces to deliver disruptive technologies.
  • Machine Learning: Machine learning is a field of artificial intelligence that uses statistical techniques to give computer systems the ability to "learn" (e.g., progressively improve performance on a specific task) from data, without being explicitly programmed
  • DB2 Warehouse: IBM Db2 Warehouse is a software-defined data warehouse for private and virtual clouds that support Docker container technology. It is client managed and optimized for fast and flexible deployment with automated scaling to meet agile analytic workloads.
  • Continuous Learning: Continuous learning is a method of machine learning, in which input data is continuously used to extend the existing model's knowledge i.e. to further train the model.

Blog

Drafting blog post here: https://ibm.ent.box.com/notes/334872469594

Links

Clone this wiki locally