Autodiff (THIS PROJECT IS EXTREMLY BUGGY)

I want to learn autograd the right way
This is a project for my intellectual curiosity. reimplementation/reengineering autograd based on the full version of Autograd.

The goal:

To dive into Matthew Johnson autograd package, understand it the best I can, document, and reimplement.
This autograd will function like pytorch.

Rescores:

The video slides by Matthew
the full version of Autograd
tutorial implementation

Example:

from autograd.numpy.container import container, VJPNode
import numpy as _np
import autograd.numpy as anp

def Container(val,requires_grad=False):
    return container(val,requires_grad=requires_grad,_node=VJPNode.new_root())

x = Container(_np.linspace(-7,7,2),False)
i = Container(_np.linspace(-3,3,2),True)

p = 9+i*x+2/x
anp.sum(p).backward()
i.grad

Todo

• higher order derivates. fix grad pipline.
• Integrate cupy for gpu computation
• Unittest
• random modules
• ordinary differential equations