This repository is home to the code that accompanies the Deep Learning with TensorFlow LiveLessons that are available within Safari. A high-level summary of these LiveLessons is available on Medium.
Working through these LiveLessons will be easiest if you are familiar with the Unix command line basics. A tutorial of these fundamentals can be found here.
In addition, if you're unfamiliar with using Python for data analysis (e.g., the pandas, scikit-learn, matplotlib packages), the data analyst path of DataQuest will quickly get you up to speed -- steps one (Introduction to Python) and two (Intermediate Python and Pandas) provide the bulk of the essentials.
Step-by-step guides for running the code in this repository can be found in the installation directory.
All of the code that I cover in the LiveLessons can be found in this directory as Jupyter notebooks.
Below is the lesson-by-lesson sequence in which I covered them:
- via analogy to their biological inspirations, this section introduces Artificial Neural Networks and how they developed to their predominantly deep architectures today
- goes over the installation directory mentioned above, discussing the options for working through my Jupyter notebooks
- details the step-by-step installation of TensorFlow on Mac OS X, a process that may be instructive for users of any Unix-like operating system
- get your hands dirty with a simple-as-possible neural network (shallow_net_in_keras.ipynb) for classifying handwritten digits
- introduces Jupyter notebooks and their most useful hot keys
- introduces a gentle quantity of deep learning terminology by whiteboarding through:
- the MNIST digit data set
- the preprocessing of images for analysis with a neural network
- a shallow network architecture
- talk through the function and popular applications of the predominant modern families of deep neural nets:
- Dense / Fully-Connected
- Convolutional Networks (ConvNets)
- Recurrent Neural Networks (RNNs) / Long Short-Term Memory units (LSTMs)
- Reinforcement Learning
- Generative Adversarial Networks
- the following essential deep learning concepts are explained with intuitive, graphical explanations:
- neural units and activation functions
- perceptron
- sigmoid (sigmoid_function.ipynb)
- tanh
- Rectified Linear Units (ReLU)
- neural units and activation functions
- cost functions
- quadratic
- cross-entropy (cross_entropy_cost.ipynb)
- gradient descent
- backpropagation via the chain rule
- layer types
- input
- dense / fully-connected
- softmax output (softmax_demo.ipynb)
- leverage TensorFlow Playground to interactively visualize the theory from the preceding section
- overview of canonical data sets for image classification and meta-resources for data sets ideally suited to deep learning
- apply the theory learned throughout Lesson Two to create an intermediate-depth image classifier (intermediate_net_in_keras.ipynb)
- builds on, and greatly outperforms, the shallow architecture from Section 1.3
- add to our state-of-the-art deep learning toolkit by delving further into essential theory, specifically:
- weight initialization
- uniform
- normal
- Xavier Glorot
- stochastic gradient descent
- learning rate
- batch size
- second-order gradient learning
- momentum
- Adam
- unstable gradients
- vanishing
- exploding
- avoiding overfitting / model generalization
- L1/L2 regularization
- dropout
- artificial data set expansion
- batch normalization
- more layers
- max-pooling
- flatten
- weight initialization
- apply the theory learned in the previous section to create a deep, dense net for image classification (deep_net_in_keras.ipynb)
- builds on, and outperforms, the intermediate architecture from Section 2.5
- whiteboard through an intuitive explanation of what convolutional layers are and how they're so effective
- apply the theory learned in the previous section to create a deep convolutional net for image classification (lenet_in_keras.ipynb) that is inspired by the classic LeNet-5 neural network introduced in section 1.1
- classify color images of flowers with two very deep convolutional networks inspired by contemporary prize-winning model architectures: AlexNet (alexnet_in_keras.ipynb) and VGGNet (vggnet_in_keras.ipynb)
- return to the networks from the previous section, adding code to output results to the TensorBoard deep learning results-visualization tool
- explore TensorBoard and explain how to interpret model results within it
- discuss the relative strengths, weaknesses, and common applications of the leading deep learning libraries:
- Caffe
- Torch
- Theano
- TensorFlow
- and the high-level APIs TFLearn and Keras
- conclude that, for the broadest set of applications, TensorFlow is the best option
- introduce TensorFlow graphs and related terminology:
- ops
- tensors
- Variables
- placeholders
- feeds
- fetches
- build simple TensorFlow graphs (first_tensorflow_graphs.ipynb)
- build neurons in TensorFlow (first_tensorflow_neurons.ipynb)
- fit a simple line in TensorFlow:
- by considering individual data points (point_by_point_intro_to_tensorflow.ipynb)
- while taking advantage of tensors (tensor-fied_intro_to_tensorflow.ipynb)
- with batches sampled from millions of data points (intro_to_tensorflow_times_a_million.ipynb)
- create a dense neural net (intermediate_net_in_tensorflow.ipynb) in TensorFlow with an architecture identical to the intermediate one built in Keras in Section 2.5
- create a deep convolutional neural net (lenet_in_tensorflow.ipynb) in TensorFlow with an architecture identical to the LeNet-inspired one built in Keras in Section 3.4
- detail systematic steps for improving the performance of deep neural nets, including by tuning hyperparameters
- specific steps for designing and evaluating your own deep learning project
- topics worth investing time in to become an expert deployer of deep learning models