KDD18 Tutorial: Deep Learning and Natural Language Processing with Apache MXNet (Incubating) Gluon
Switch branches/tags
Nothing to show
Clone or download
Latest commit 7f7570a Aug 23, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
00_setup KDD18 Aug 21, 2018
01_basics KDD18 Aug 21, 2018
02_neural_networks KDD18 Aug 21, 2018
03_computer_vision KDD18 Aug 21, 2018
04_distributed_training KDD18 Aug 21, 2018
05_data_pipeline KDD18 Aug 21, 2018
06_word_embedding KDD18 Aug 21, 2018
07_language_model KDD18 Aug 21, 2018
08_sequence_generation KDD18 Aug 21, 2018
09_sentiment_analysis KDD18 Aug 21, 2018
10_translation KDD18 Aug 21, 2018
env KDD18 Aug 21, 2018
slides upload keynotes Aug 21, 2018
LICENSE KDD18 Aug 21, 2018
README.md add social links Aug 24, 2018

README.md

KDD18 London: Deep Learning and Natural Language Processing with Apache MXNet Gluon

Time: Tuesday, August 21, 2018
Location: ICC Capital Suite Room 14+15+16

Presenter: Alex Smola, Leonard Lausen, Haibin Lin

Abstract

While deep learning has rapidly emerged as the dominant approach to training predictive models for large-scale machine learning problems, these algorithms push the limits of available hardware, requiring specialized frameworks optimized for GPUs and distributed cloud-based training. Moreover, especially in natural language processing (NLP), models contain a variety of moving parts: character-based encoders, pre-trained word embeddings, long-short term memory (LSTM) cells, and beam search for decoding sequential outputs, among others.

This tutorial introduces GluonNLP (GitHub), a powerful new toolkit that combines MXNet's speed, the user-friendly Gluon frontend, and an extensive new library automating the most painful aspects of deep learning for NLP. In this full-day tutorial, we will start off with a crash course on deep learning with Gluon, covering data, autodiff, and deep (convolutional and recurrent) neural networks. Then we'll dive into GluonNLP, demonstrating how to work with word embeddings (both pre-trained and from scratch), language models, and the popular Transformer model for machine translation.

Preparation

While most of the notebooks we prepared can run from the comfort of your laptop, some more interesting real-life problems benefit from more computation power. At the tutorial, we will provide each participant with a $50 AWS credit code for you to try out these problems yourself on the Amazon EC2 machines.

In preparation for the hands-on tutorial, please make sure that you have an AWS account with at least one p2.8xlarge AND one p3.2xlarge instance in EU (Ireland) available for launch. You may register an AWS account and follow the instructions on this page to verify and request for p2.8xlarge AND one p3.2xlarge instance limits in EU (Ireland) region.

Interested to learn more about GluonNLP? Sign-up and join our slack channel.

We'd love to hear from you! Have feedback to help us improve? Interested in career at AmazonAI? Contact us at amazonai-kdd18@amazon.com.

Agenda

Time Title Slides Notebooks
8:30-9:15 Installation and Basics (NDArray, AutoGrad, Libraries) link link link
9:15-9:30 Neural Networks 101 (MLP, ConvNet, LSTM, Loss, SGD) - Part I link link
9:30-10:00 Coffee Break
10:00-10:30 Neural Networks 101 (MLP, ConvNet, LSTM, Loss, SGD) - Part II link
10:30-11:00 Computer Vision 101 (GluonCV) link link
11:00-11:30 Parallel and Distributed Training link link
11:30-12:00 Data I/O in NLP (and Iterators) link
12:00-13:30 Lunch Break
13:30-14:15 Embeddings link link
14:15-15:00 Language Models (LM) link link
15:00-15:30 Sequence Generation from LM link link
15:30-16:00 Coffee Break
16:00-16:15 Sentiment Analysis link
16:15-17:00 Transformer Models and Machine Translation link link
17:00-17:30 Questions