This repository serves as a public issue tracker and documentation host for Gradient, full TensorFlow binding for .NET
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
LICENSE-TECH-PREVIEW.txt trivial license draft Jan 7, 2019
README.adoc added Limitations section and Porting section stub Jan 19, 2019

README.adoc

Gradient

Gradient is a fully-fledged, mostly typed binding to TensorFlow for .NET

He shall speak not reverbering injurance.
— Gradient-based Char-RNN trained on Shakespeare

Contents

Getting started

Installation

Tech Preview is currently limited to Windows x64 machines, and full .NET Framework 4.7.1+ targets.

Install Python + Tensorflow

Before installing Gradient, you should ensure, that you have TensorFlow installed and working:

  1. Install Python 3.6 64-bit. If you have Visual Studio 2017+, it is possible to install it as a component. Otherwise, get one from https://www.python.org/downloads/

  2. Install TensorFlow 1.10 using pip:

    1. Find python.exe in the installation directory (VS installs to: C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\python.exe)

    2. Open command line to the directory, containing python.exe

    3. Execute .\python.exe -m pip install "tensorflow==1.10.*" or .\python.exe -m pip install "tensorflow-gpu==1.10.*" if you want GPU acceleration via CUDA (NVidia only)

  3. Check the installation by launching python.exe, and running import tensorflow. It should succeed.

Add Nuget package to your project

Gradient packages are published on Nuget, but will not be "listed" until about Tech Preview 5, which means you won’t be able to find it in the Nuget Package Manager. Nuget page lists the commands, necessary to install the package into your project. As of preview1 and the new .csproj format, the command is

dotnet add package Gradient --version 0.1.10-tech-preview1

If using the new package management features of .csproj, this could also be achieved by adding the following line to it:

<PackageReference Include="Gradient" Version="0.1.10-tech-preview1"  />

See the example project file here.

First steps

Namespaces

In most cases, you will need to add using tensorflow; at the beginning of your file. In many cases you will also need using Gradient; and using numpy;.

using numpy;
using tensorflow;
using Gradient;

Logging

TensorFlow logging is separate from Gradient logging. This section discusses the later.

To enable Gradient logs, set appropriate properties of GradientLog static class, e.g.:

GradientLog.OutputWriter = Console.Out;

Old style TF

Prior to the recent changes, the main way to use TensorFlow was to contstruct a computation graph, and then run it in a session. Most of the existing examples will use this mode.

Constructing compute graph

Graph creation methods are located in the tf class from tensorflow namespace. For example:

var a = new dynamic[] { tf.constant(5.0, name: "a") };
var b = new dynamic[] { tf.constant(10.0, name: "b") };

var sum = new dynamic[] { tf.add(a, b, name: "sum") };
var div = new dynamic[] { tf.div(a, b, name: "div") };

In this case we wrap the resulting Tensor objects into an array, as many construction functions in Gradient like tf.add expect arrays (or rather IEnumerable), not individual Tensor values. (that might change in the future)

Running computation

Next, you need to create a Session to run your graph one or multiple times. Sessions allocate CPU, GPU and memory resources, and hold the states of variables.

Note
In GPU mode, TensorFlow will attempt to allocate all the GPU memory to itself at that stage, so ensure you don’t have any other programs extensively using it, or turn down TensorFlow memory allocation (has not been tested in Gradient Tech Preview)

Since TensorFlow sessions hold unmanaged resources, they have to be used similar to (but not idential to) IDisposable:

new Session().UseSelf(session => {
    ...do something with the session...
});

Now that you have a Session to work with, you can actually compute the values in the graph:

new Session().UseSelf(session => {
    Console.WriteLine($"a = {session.run(a)}");
    Console.WriteLine($"b = {session.run(b)}");
    Console.WriteLine($"a + b = {session.run(sum)}");
    Console.WriteLine($"a / b = {session.run(div)}");
});

Note, that Session.run also takes a sequence of Tensor-like objects.

The full code for this example is available at our samples repository

Porting Python code to Gradient

In most cases converting Python code, that uses TensorFlow, should be as easy as using C# syntax instead of Python one:

  • add new to class constructor calls: Class() to new Class().

Its easy to spot class construction vs simple function calls in Python: by convention function names there start with a lower case letter like min, while in class names the first letter is capitalized: Session

  • to pass named paramters, use : instead of =: make_layer(kernel_bias=2.0)make_layer(kernel_bias: 2.0)

  • to get a subrange of a Tensor , use C# 8 syntax (if available): tensor[1..-2]tensor[1..^2]. A single element can be addressed as usual: tensor[1]

Names of classes and functions

Generally, Gradient follows TensorFlow Python API naming.

Limitations of the current tech preview

This section may be outdated

Can’t inherit Gradient classes

While nothing will stop you from inheriting Gradient classes in C#, any new or overriden members will not be visible to TensorFlow. You may implement corresponding interfaces in C#, but don’t inherit anything from any classes in Gradient, tensorflow, or numpy namespaces.

Any callbacks, written in .NET are unable to call Gradient code.

Will NOT work:

tf.layer(activation: PythonFunctionContainer.Of<double, double>(v => tf.sqrt(v)))

Will work:

tf.layer(activation: PythonFunctionContainer.Of<double, double>(v => System.Math.Sqrt(v)))

Note the use of tf.sqrt function inside lambda.

Tips and Tricks

C# 8

Gradient supports the neat indexing feature of C# 8: if you are using Visual Studio 2019 Preview+ or the .NET Core SDK 3 Preview+, you can set appropriate language level like this in the project file: <LangVersion>8.0</LangVersion>

Then you can access numpy arrays with the new syntax, for example: arr[3..^4], which means "take a range from element at index 3, that excludes the last 4 elements".