Skip to content


Folders and files

Last commit message
Last commit date

Latest commit



8 Commits

Repository files navigation

Neural Network from Scratch in Julia

Adapted from the original Python/PyTorch implementation at

May 2022 / Markus Konrad


The main file neuralnets.jl shows an example implementation of a single layer neural network with Julia by following a tutorial on "napkin math". It uses the AutoGrad package for calculating the loss function gradients for stochastic gradient descend (SGD). For a better understanding of AutoGrad two more files are added: autograd_tanh.jl shows how to get the derivate of a user defined function and autograd_lm.jl shows how to apply SGD for a simple linear model with two parameters.

You should run the code line-by-line with a Julia REPL in order to understand it.


The source-code is provided under Apache License 2.0 (see LICENSE file).