A pure-python/numpy autograd tensor library
Latest commit f544140 Apr 28, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
doc Apr 28, 2019
mygrad Apr 28, 2019
tests Apr 25, 2019
.gitattributes Nov 18, 2018
.gitignore Mar 3, 2019
.travis.yml Feb 9, 2019
LICENSE.txt Oct 27, 2018
MANIFEST.in Nov 18, 2018
README.md Mar 14, 2019
readthedocs.yml Nov 18, 2018
setup.cfg Nov 18, 2018
setup.py Mar 3, 2019
tox.ini Feb 3, 2019
versioneer.py Nov 18, 2018

mygrad is a simple, NumPy-centric autograd library. An autograd library enables you to automatically compute derivatives of mathematical functions. This library is designed to serve both as a tool for prototyping/testing and as an education tool for learning about gradient-based machine learning; it is easy to install, has a readable and easily customizable code base, and provides a sleek interface that mimics NumPy. Furthermore, it leverages NumPy's vectorization to achieve good performance despite the library's simplicity.

This is not meant to be a competitor to libraries like PyTorch (which mygrad most closely resembles) or TensorFlow. Rather, it is meant to serve as a useful tool for students who are learning about training neural networks using back propagation.

A more in-depth discussion of MyGrad and documentation for its internals can be found here.

## Installing mygrad (this project)

To install MyGrad, you can pip-install it:

pip install mygrad

or clone this repository and navigate to the MyGrad directory, then run:

python setup.py install

MyGrad requires numpy. It is highly recommended that you utilized numpy built with MKL for access to optimized math routines.

## A Simple Application

Let's use mygrad to compute the derivative of , which is .

mygrad.Tensor behaves nearly identically to NumPy's ndarray, in addition to having the machinery needed to compute the analytic derivatives of functions. Suppose we want to compute this derivative at x = 3. We can create a 0-dimensional tensor (a scalar) for x and compute f(x):

>>> import mygrad as mg
>>> x = mg.Tensor(3.0)
>>> f = x ** 2
>>> f
Tensor(9.0)

Invoking f.backward() instructs mygrad to trace through the computational graph that produced f and compute the derivatives of f with respect to all of its independent variables. Thus, executing f.backward() will compute and will store the value in x.grad:

>>> f.backward()  # triggers computation of df/dx
>>> x.grad  # df/dx = 2x = 6.0
array(6.0)

This is the absolute tip of the iceberg. mygrad can compute derivatives of multivariable composite functions of tensor-valued variables!

## Some Bells and Whistles

mygrad supports all of NumPy's essential features, including:

mygrad.Tensor plays nicely with NumPy-arrays, which behave as constants when they are used in computational graphs:

>>> import numpy as np
>>> x = mg.Tensor([2.0, 2.0, 2.0])
>>> y = np.array([1.0, 2.0, 3.0])
>>> f = x ** y  # (2 ** 1, 2 ** 2, 2 ** 3)
>>> f.backward()
array([ 1.,  4., 12.])

mygrad.nn supplies essential functions for machine learning, including:

• N-dimensional convolutions (with striding, dilation, and padding)
• N-dimensional pooling
• A gated recurrent unit for sequence-learning (with input-level dropout and variational hidden-hidden dropout)

It leverages a nice sliding window view function, which produces convolution-style windowed views of arrays/tensors without making copies of them, to intuitively (and quite efficiently) perform the neural network-style convolutions and pooling.

The following is an example of using mygrad to compute the hinge loss of classification scores and to "backpropagate" through (compute the gradient of) this loss. This example demonstrates some of mygrad's ability to perform backpropagation through broadcasted operations, basic indexing, advanced indexing, and in-place assignments.

>>> from mygrad import Tensor
>>> import numpy as np
>>> class_scores = Tensor(10 * np.random.rand(100, 10))         # 100 samples, 10 possible classes for each
>>> class_labels = np.random.randint(low=0, high=10, size=100)  # correct label for each datum
>>> class_labels = (range(len(class_labels)), class_labels)
>>> correct_class_scores = class_scores[class_labels]

>>> Lij = class_scores - correct_class_scores[:, np.newaxis] + 1.  # 100x10 margins
>>> Lij[Lij <= 0] = 0      # scores within the hinge incur no loss
>>> Lij[class_labels] = 0  # the score corresponding to the correct label incurs no loss

>>> loss = Lij.sum() / class_scores.shape[0]  # compute mean hinge loss
>>> loss.backward()    # compute gradient of loss w.r.t all dependent tensors
>>> class_scores.grad  # d(loss)/d(class_scores)
array([[ 0.  ,  0.01,  0.  , -0.04,  0.  ,  0.  ,  0.01,  0.  ,  0.01, 0.01], ...])

## Computational Graph Visualization

mygrad uses Graphviz and a Python interface for Graphviz to render the computational graphs built using tensors. These graphs can be rendered in Jupyter notebooks, allowing for quick checks of graph structure, or can be saved to file for later reference.

The dependencies can be installed with:

conda install graphviz
conda install python-graphviz
You can’t perform that action at this time.