Skip to content
Thinking in tensors, writing in PyTorch (a hands-on deep learning intro)
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.
0 Before you start.ipynb
1 Vectors, matrices and tensors.ipynb
1_tech PyTorch aritmetics.ipynb
2 Gradient Descent.ipynb
3 Linear regression.ipynb
4 Nonlinear regression.ipynb

Thinking in tensors, writing in PyTorch

A hands-on deep learning introduction, from pieces.

By Piotr Migdał et al. (Weronika Ormaniec, possibly others)

“Study hard what interests you the most in the most undisciplined, irreverent and original manner possible.” ― Richard Feynman

“Scientists start out doing work that's perfect, in the sense that they're just trying to reproduce work someone else has already done for them. Eventually, they get to the point where they can do original work. Whereas hackers, from the start, are doing original work; it's just very bad. So hackers start original, and get good, and scientists start good, and get original.” - Paul Graham in Hackers and Painters

What's that?

Mathematical concepts behind deep learning using PyTorch 1.0.

  • All math equations as PyTorch code
  • Explicit, minimalistic examples
  • Jupyter Notebook for interactivity
  • “On the shoulders of giants” - I link and refer to the best materials I know
  • Fully open source & open for collaboration (I guess I will go with MIT for code, CC-BY for anything else)

Why not something else?

There are quite a few practical introductions to deep learning. I recommend Deep Learning in Python by François Chollet (the Keras author). Or you want, you can classify small pictures, or extraterrestrial beings, today.

When it comes to the mathematical background, Deep Learning Book by Ian Goodfellow et al. is a great starting point, giving a lot of overview. Though, it requires a lot of interest in maths. Convolutional networks start well after page 300.

I struggled to find something in the middle ground - showing mathematical foundations of deep learning, step by step, at the same time translating it into code. The closest example is CS231n: Convolutional Neural Networks for Visual Recognition (which is, IMHO, a masterpiece). Though, I believe that instead of using NumPy we can use PyTorch, giving a smooth transition between mathematic ideas and a practical, working code.

Of course, there are quite a few awesome posts, notebooks and visualizations


Crucially, this course is for you, the reader. If you are interested in one topic, let us know! There is nothing more inspiring that eager readers.


  • Start with concrete examples first
  • First 1d, then more
  • Equations in LaTeX AND PyTorch
  • x.matmul(y).pow(2).sum() not torch.sum(torch.matmul(x, y) ** 2)


A few links of mine:

You can’t perform that action at this time.