Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time


Material for The Mathematical Engineering of Deep Learning. See the actual book content on or (when it is out) purchase the book from CRC press.

This repository contains general supporting material for the book.

Below is a detailed list of the source code used for creating figures and tables in the book. We use Julia, Python, or R and the code is sometimes in stand alone files, sometimes in Jupyter notebooks, sometimes as R Markdown, and sometimes in Google Colab. Many of our static illustrations were created using TikZ by Ajay Hemanth and Vishnu Prasath with the source of their illustrations also available so you can adapt it for purposes.

Chapter 1

Figure Topic Source Code
1.1 example Python Google Colab
1.3 Architectures TikZ(a), TikZ(b), TikZ(c), TikZ(d), TikZ(e), TikZ(f)
1.4 Neurons TikZ(b), TikZ(d)
1.5 Data on earth Julia

Chapter 2

Figure Topic Source Code
2.1 Supervised Learning TikZ
2.2 Unsupervised Learning TikZ
2.3 Simple regression R
2.4 Breast Cancer ROC curves R
2.5 Least Squares TikZ
2.6 Loss functions Julia
Table 2.1 Linear MNIST classification Julia
2.7 Gradient Descent Learning Rate Python
2.8 Loss Landscape R
2.9 Generalization and Training TikZ or Julia
2.10 Polynomial fit R
2.11 K-fold cross validation TikZ
2.12 K-means clustering R
2.13 K-means image segmentation R
2.14 Breast Cancer PCA R
2.15 SVD Compression Julia

Chapter 3 (Figures under construction)

Figure Topic Source Code
3.1 Logistic regression model curves R(a) R(b)
3.2 Linear decision boundary for logistic regression R
3.3 Components of an artificial neuron TikZ
3.4 Loss landscape of MSE vs. CE on logistic regression Python
3.5 Evolution of gradient descent learning in logistic regression R
3.6 Shallow multi-output neural network with softmax TikZ
3.7 Multinomial regression for classification R
Table 3.1 Different approaches for creating an MNIST digit classifier. Julia
3.8 Feature engineering in simple logistic regression R
3.9 Non-linear classification decision boundaries with feature engineering in logistic regression R
3.10 Non-linear classification decision boundaries with feature engineering in multinomial regression R
3.11 Single hidden layer autoencoder TikZ
3.12 Autoencoder projections of MNIST including using PCA R TikZ
3.13 Manifolds and autoencoders R TikZ
3.14 MNIST using autoencoders R
3.15 Denoising autoencoder TikZ
3.16 Interpolations with autoencoders Julia

Chapter 4 (Figures)

Figure Topic Source Code
4.1 Convexity and local/global extrema Python
4.2 Gradient descent with a time dependent learning rate Python
4.3 Stochastic gradient descent Python
4.4 Early stopping in deep learning Julia
4.5 Non-convex loss landscapes Python
4.6 Momentum enhancing gradient descent Python
4.7 The computational graph for automatic differentiation TikZ
4.8 Line search concepts Python
4.9 The zig-zagging property of line search (zoom in) Python
4.10 Newton's method in one dimension Python

Chapter 5 (Figures under construction)

Figure Topic Source Code
5.1 Fully Connected Feedforward Neural Networks TikZ(a), TikZ(b)
5.2 Arbitrary function approximation with neural nets Julia
5.3 Binary classification with increasing depth R
5.4 A continuous multiplication gate with 4 hidden units TikZ
5.5 Several common scalar activation functions Julia
5.6 Flow of information in general back propagation TikZ
5.7 Simple neural network hypothetical example TikZ
5.8 Flow of information in standard neural network back propagation TikZ
5.9 Computational graph for batch normalization TikZ
5.10 The effect of dropout TikZ

Chapter 6 (Figures under construction)

Figure Topic Source Code
6.1 TBD

Chapter 7 (Figures under construction)

Figure Topic Source Code
7.1 TBD

Chapter 8 (Figures under construction)

Figure Topic Source Code
8.1 TBD

Chapter 9 (Figures under construction)

Figure Topic Source Code
9.1 TBD

Chapter 10 (Figures under construction)

Figure Topic Source Code
10.1 TBD


Material for The Mathematical Engineering of Deep Learning. See







No releases published


No packages published