Basic PyTorch Tutorials in Python for beginners
This repository contains many resources I found during my study of this subject. This repository is a summary, note, resource, and information I found useful and would like to share with fellow beginners. I have no means of claiming the credits to myself, and I will be trying to cite as many things as possible.
To master deeplearning and AI, according to Andrew Ng, these areas of math are the most imporant, in decreasing order:
- Linear Algebra
- Probability and Statistics
- Calculus (including multivariate calculus)
- Optimization
My recommendation is to learn basic linear algebra then basic programming.
There are many online classes and toturials you can find online.
This will walk you through the PyTorch installation, including installing python and a toolkit.
A neural network is a mathematical function. It takes in one or multiple inputs, process it and produces one or more outputs, In PyTorch, neural networks are composed of PyTorch tensors.
You can initialize a tensor in many ways. Here is the reference source, TENSORS.
- Directly With Operator
- Directly From Arrays
- From a NumPy Array
- From Another Tensor
- With a Random/Constant Values
Attributes describe their shape, datatype, and the device on which the tensor is stored.
This is where you have to be careful when comverting and modifying tensors. As they often point to the same memory address. Like a C++ pointer, when you modify one variable, another variable will be modified as well.
- Loading a Dataset
- Iterating and Visualizing the Dataset
- Creating Custom Dataset
- Preparing Data for training with DataLoaders
- Iterate through the DataLoader
- Tensors, Functions and Computational graph
- Computing Gradients
- Disabling Gradient Tracking
- More on Computational Graphs
- Optional Reading: Tensor Gradients and Jacobian Products
- Hyperparameters
- Optimization Loop
- Loss Function
- Mean Squared Error(MSE)
- Negative Log Likelihood(NLL)
- Binary Crossentropy (BCE)
- Categorical Crossentropy (CC)
- Sparse Categorical Crossentropy (SCC)
- Optimizer
- Full Implementation