Classes

A Gentle Introduction to Machine Learning for Product Managers

I taught a machine learning (ML) class to Product Managers one year ago. The goals of this class are to learn the fundamentals of ML technologies so that a product manager can assess the business opportunities for an ML-powered application in an organization, and lead the development and launch of that product while interacting on the technology side with data scientists and machine learning engineers.

I have slightly updated the content of that class. A lot of Product Managers working on machine learning are contacting me on LinkedIn. So, I am hoping that the following materials might be useful to someone.

There are six classes. Class 7 is a final project where students present a machine learning application.

Each class includes a presentation, readings, videos to watch, and homework.

The most difficult concepts are presented and discussed in multiple classes in an incremental way.

There is some easy homework to do from Class 1 to Class 3. Students are expected to work from Class 4 to Class 6 on their final projects.

Class 1

ML University for PM Class 1

Videos:

• Professor Andrew Ng: Artificial Intelligence is the New Electricity at Stanford GSB: https://youtu.be/21EiKfQYZXc

Readings:

• Wikipedia: Linear regression https://en.wikipedia.org/wiki/Linear_regression
• Wikipedia: Logistic regression https://en.wikipedia.org/wiki/Logistic_regression
• Wikipedia: K-means clustering https://en.wikipedia.org/wiki/K-means_clustering 

Homework:

1. Select a few ML applications that you are using every day on your desktop or your mobile phone. Classify them into supervised or unsupervised learning. Try to identify the source of the data. Try to define how as a product/program manager or a product designer, you will consider those apps successful and which metrics, you might use to measure the app success. Please put your thoughts in a google document that you can share.
2. Propose a few applications that could be developed using an estimation (based on a regression), a classification (based on a logistic regression), and a segmentation (based on a K-means clustering). Please put your thoughts in a google document that you can share.

Class 2

ML University for PM Class 2

Videos:

• A machine learning primer: https://www.youtube.com/watch?v=1M09i0f3ruI&t=22s

(Please watch that video a couple of times, until you get it 🙂 We have not covered yet the K-nearest neighbor and Support Vector Machine algorithms but still try to understand those two algos)

Readings:

• Wikipedia: Decision trees learning https://en.wikipedia.org/wiki/Decision_tree_learning
• Wikipedia: Ensemble learning https://en.wikipedia.org/wiki/Ensemble_learning
• Wikipedia: Random forest https://en.wikipedia.org/wiki/Random_forest
• Wikipedia: Artificial neural network https://en.wikipedia.org/wiki/Artificial_neural_network

Optional (and advanced but interesting):

• Professor Michael Jordan from UC Berkeley, Perspectives and Challenges of ML: https://youtu.be/4inIBmY8dQI

Homework:

1. You have been hired to be the first Product Manager, Program Manager, or Designer for a new online store. Your first assignment is to lead the development of an ML application for classifying the products across various categories that will be available on the store. Consider that the store will provide up to 500 products from many suppliers across many types of categories for consumers such as home decoration, cooking, fashion, fitness…To get started on that assignment, please propose a small draft of your proposed plan for that app. Please put your thoughts in the same google document that you use for your previous homework.
2. Since you completed ahead of schedule and with a lower budget your classification app, you are now being asked to develop a segmentation app for the customers that will be visiting and purchasing on the store. That app could leverage the labels of your classification apps. Please put your thoughts in the same google document that you use for your previous homework.

Note: You can be as creative as you want in terms of the product plan that you will use to drive the development of those apps but at least describe the “what” and “why” of the apps, the data sources/features, the possible models, and any KPIs. If you are running out of time, just do something simple in a few sentences for each homework in ½ page. If you have time, do as much as you can 🙂

Class 3

ML University for PM Class 3

Videos and readings:

• Read and watch as much as you can 🙂

Google ML class:

• Descending into ML:
  • Videos: https://youtu.be/qAjFQLydY8E
  • Training and loss: https://developers.google.com/machine-learning/crash-course/descending-into-ml/training-and-loss
• Reducing loss:
  • Video: https://youtu.be/CaO0UDJirO4
  • Iterative approach: https://developers.google.com/machine-learning/crash-course/reducing-loss/an-iterative-approach
  • Gradient descent: https://developers.google.com/machine-learning/crash-course/reducing-loss/gradient-descent
  • Learning rate: https://developers.google.com/machine-learning/crash-course/reducing-loss/learning-rate
  • Optimizing learning rate: https://developers.google.com/machine-learning/crash-course/fitter/graph
  • Stochastic gradient descent: https://developers.google.com/machine-learning/crash-course/reducing-loss/stochastic-gradient-descent

Optional (for those with a math background):

• Understanding gradient descent: https://eli.thegreenplace.net/2016/understanding-gradient-descent/

Homework:

• Answer the following questions. Note: please limit your answers to a few concise sentences:

1. What is a labeled training set?
2. What are the two most common tasks that a supervised machine learning model can do?
3. Give the name of a common task that an unsupervised machine learning model can do?
4. What is the difference between a model parameter and a learning algorithm hyperparameter?
5. If your model performs great on the training data but generalizes poorly to new instances, what is happening? Can you mention some possible solutions?
6. What is a test data set and why would you want to use it?
7. Why feature engineering is a critical task for a data scientist to perform in order to develop a model that provides some good predictions?
8. When training a model, a data scientist might use an algorithm to reduce the cost or loss function of the model? What is in that case, the cost function?
9. What are the advantages of using a deep learning model compared to other traditional machine learning models?
10. Why deep learning models are somewhat considered as a “black box” model compared for instance to decision trees that are considered as a “white box” model?

Class 4

ML University for PM Class 4

Readings:

• Wikipedia: Support Vector Machine: https://en.wikipedia.org/wiki/Support_vector_machine
• Wikipedia: Hierarchical clustering: https://en.wikipedia.org/wiki/Hierarchical_clustering

Deep learning:

Videos:

Training a neural network:

• https://youtu.be/GvQwE2OhL8I
• https://youtu.be/9ZsyQZouOQ8
• https://youtu.be/5u0jaA3qAGk
• https://youtu.be/GlcnxUlrtek

Homework: Final project to be presented in Class 7

• Each student should select an ML application either from her or his organization or outside her or his organization
• The project deliverables shall include least the following sections:
  • ML application:
    • What, why, business case, business plan…
    • Use cases, user experiences, features, requirements, KPIs…
  • Models:
    • Machine learning tasks
    • Model predictions
    • Type of learning
    • Sucess metrics and KPIs
  • Features:
    • Feature sharing
  • Data sets
    • Data lakes
    • Data sources
    • Batch data
    • Streaming data
  • Machine learning pipeline:
    • Training
    • Inference
    • Experimentation
    • Feature store

Class 5

ML University for PM Class 5

Machine learning data pipelines

Google:

Readings:

• TFX: A TensorFlow-Based Production-Scale Machine Learning Platform

Videos:

• The Anatomy of a Production-Scale Continuously-Training Machine Learning Platform
• TensorFlow Extended (TFX) (TensorFlow Dev Summit 2018)
• TensorFlow Extended (TFX) Overview and Pre-training Workflow (TF Dev Summit ’19)
• TensorFlow Extended (TFX) Post-training Workflow (TF Dev Summit ’19)

Facebook:

Readings:

• FBLearner
• Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective

Uber:

Readings:

• Meet Michelangelo: Uber’s Machine Learning Platform
• Scaling Machine Learning at Uber with Michelangelo
• Michelangelo PyML: Introducing Uber’s Platform for Rapid Python ML Model Development
• Uber’s Big Data Platform: 100+ Petabytes with Minute Latency

Videos:

• Scaling Machine Learning as a Service at Uber

Twitter:

Readings:

• Cortex
• Productionizing ML with workflows at Twitter
• Twitter meets TensorFlow

Airbnb:

Video:

• Bighead: Airbnb’s end-to-end Machine Learning Platform

Recommender systems:

Videos:

• Coursera: Machine Learning Foundations from the University of Washington: A Case Study Approach – Week 5 Recommender Systems
• Presentation of the Coursera class

Readings:

• A field study of video recommendations: newest, most similar, most relevant (easy reading)
• Matrix factorization (not too difficult reading)
• Factorization machines (difficult reading)

Homework: Final project to be presented in Class 7

Class 6

ML University for PM Class 6

Videos:

• Emergence of Locomotion Behaviours in Rich Environments: https://youtu.be/hx_bgoTF7bs
• Everybody Dance Now: https://youtu.be/PCBTZh41Ris

Readings:

• Deep learning – Yann LeCun, Yoshua Bengio & Geoffrey Hinton; https://www.cs.toronto.edu/~hinton/absps/NatureDeepReview.pdf

Homework:

• Final project to be presented in Class 7

Class 7

Final project presentation: • Show time 🙂