Repository for the "Machine Learning for the Web" class at ITP, NYU, for Fall 2018
Branch: master
Clone or download
Latest commit 3eb0ddc Feb 17, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
week1-intro Update Nov 7, 2018
week2-ImageClassifier-KNN-Posenet Update Nov 18, 2018
week3-styleTransfer Update Nov 13, 2018
week4-pix2pix Update Nov 19, 2018
week5-moreExamples Update Nov 26, 2018
week6-soundClassifier-sketchRNN Update Feb 18, 2019
.gitignore gitignore Oct 23, 2018 Update Dec 6, 2018

Machine Learning for the Web

This is a repository for the "Machine Learning for the Web" class at ITP, NYU

Libraries like TensorFlow.js and ml5.js unlocked new opportunities for interactive machine learning projects in the browser. The goal of this class is to learn and understand common machine learning techniques and apply them to generate creative outputs in the browser.

This class will start with running models in the browser using high-level APIs from ml5.js, as well as explore the Layer APIs from TensorFlow.js to train models using custom data. This class will also cover preparing the dataset for training models.

At the completion of this course, students will have a better understanding of a few machine learning models, how do they work, how to train these models, and their use case to creative projects. Students will also be able to create interactive ML web applications with pre-trained models.


Get started

To run each examples, open your terminal, type in the following commands:

$ git clone
$ cd machine-learning-for-the-web
$ python -m SimpleHTTPServer     # $ python3 -m http.server (if you are using python 3)

Go to localhost:8000 in your browser, you will see a directory list like this:

  • week1-intro/
  • week2-knnClassifier/
  • week3-styleTransfer/
  • week4-pix2pix/
  • week5-soundClassifier/
  • week6-workshop/

Click into each week, you will see the example for each week.


Week 1 Introduction to Machine Learning, MobileNet

  • Notes: Introduction to Machine Learning

  • Coding session:

    • Installing ml5.js
    • Running Image Classification example with ml5.js
    • Installing tf.js
    • Running examples with tf.js
  • Homework

Week 2 Image Classification (KNN classifier, PoseNet)

  • Notes

  • Coding session:

    • Make a KNN Image Classifier
    • Regression with feature extractor
    • Posnet
    • poseNet + KNN Image Classifier
  • Homework

Week 3 Image Transformation (Part 1) Style transfer

  • Review: Showing the homework from last week, discuss any difficulties and other creative outputs

  • Notes

  • Coding session:

    • Setup training environment
    • Training a new Style Transfer model
    • Running Style Transfer model in ml5.js
  • Homework

Week 4 Image Transformation (Part 2) pix2pix

  • Review: Showing the homework from last week, discuss any difficulties when you are training your own style trasnfer model

  • Notes

  • Coding session:

    • Running pix2pix with ml5.js
    • Setup training environment
    • Prepare dataset for pix2pix
    • Training a new pix2pix model
  • Homework

Week 5 Final assignment proposal and guest lecture

  • More code examples and demo here
  • Review: Present your final assignment proposal and the experiments you did so far
  • Guest speaker: Runwayml

Week 6 Speech recognition, Sketch RNN, Body pix, Build your own model

  • Notes

  • Coding session:

    • Body pix from tf.js-models
    • Speech recognition from p5 speech.js
    • Sketch RNN from ml5.js
    • Build a doodle classifier with tf.js
  • Homework: Work on the final assignment, prepare the presentation

Week 7

Final Presentation



  • You are required to attend all class meetings and submit all weekly assignments.
  • Grading (pass/fail) will be based on a combination of factors:
    • Attendance, participation in class discussion, and engagement in other students' projects (25%)
    • Quality of weekly assignments (50%)
    • Last Assignment (25%)