Exporting a model

Exporting a model

In short

A network trained here can run somewhere else. The package DeepSharp.Export.Onnx writes a stack of steps as an ONNX graph, the format every runtime reads, and the graph answers what the network answers: every graph the tests write is run through ONNX Runtime and held to the network's own numbers. The rule to remember: the graph is written from the model and never read back as the model, and a layer that is written as no ONNX node is refused by name rather than written as something it is not.

You want to…	Write
Write the graph to a stream	new OnnxExport(shape).Write(network, loss, stream)
Have the graph as bytes	new OnnxExport(shape).ToBytes(network, loss)
Name the features a row is made of, beside the graph	new OnnxExport(shape) { Features = ["age", "fare"] }
Read a graph back	new OnnxFile(loss).Read(stream), in Importing a model

The rest of this page says what is written for each layer, which operator set it asks for, and what is refused.

The package

dotnet add package DeepSharp.Export.Onnx

It depends on the core package and on the generated messages of ONNX's own onnx.proto, and on no runtime: ONNX Runtime belongs to the package's test project, which answers every graph the package can write. How a number is laid out — a weight turned round, a kernel's axes, the gates of an LSTM in ONNX's order — is one source file compiled into the reader and into this package, so the two cannot come to disagree.

shape is the shape of one example, without the batch's axis — a row's length, or steps by channels, or rows by columns by channels. The graph takes a batch of them and gives a batch of what the network makes of them, then of its loss's output activation: a sigmoid for a binary cross-entropy, a softmax for a cross-entropy, written as the graph's last node.

using DeepSharp.Export.Onnx;
using DeepSharp.Networks;
using DeepSharp.Tensors;

var network = new Sequential().Dense(16).Relu().Dense(1).Lower(new Shape(14), new RandomStream(1));

using var file = File.Create("passenger.onnx");
new OnnxExport(new Shape(14)).Write(network, new BinaryCrossEntropy(), file);

What is written

A network is a stack of steps, and each step is written under the name it goes by, its numbers as initializers named by the slot's path (0.weight, encoder.self_attn.in_proj_weight).

layer	written as
dense	a MatMul by the weights as the layer keeps them, inputs by outputs, and an Add of the bias; a low-rank pair beside a held kernel as the products it is
the activations	the operator of the same name; a SiLU as the input times its own sigmoid
layer, root-mean-square, batch, group and instance normalisations	the operator of the same name (a batch normalisation as it answers, from its running statistics)
convolutions, transposed convolutions, poolings, pads	Conv, ConvTranspose, MaxPool, AveragePool, Pad, with the window's strides, dilations and the border worked out for the examples that reach each
global and adaptive pooling	GlobalAveragePool and GlobalMaxPool, or a reduction over the places; an adaptive pooling where the size divides the length
flatten, reshape	Flatten, Reshape
embeddings, the periodic embedding	the products and operators that look a vector up or turn a number round, as the layer does
a sum, a concatenation, a block of columns	Add, Concat, Slice
an LSTM or a GRU	one LSTM or GRU node for each layer, the two directions in the one node, the lengths of the series from the mask
attention, multi-head attention	the nodes below
the sine positions	an Add of the positions the formula works out
a dropout, a variation of the rows	nothing: a trained network does nothing with them

Series, images and volumes are held in the graph with their channels first, as ONNX's operators take them, and the network holds them with their channels last; the turns are written where an operator needs one.

Attention

Not the Attention operator of operator set 23, but the nodes of the oldest sets that hold it: the series projected into queries, keys and values by three products, the heads cut out by a reshape and a transpose, the scores one batched product scaled by one over the root of a head's width, the keys a mask leaves out put far below nothing by an Add, a Softmax over the last axis, and the values taken in those shares. A step with no key left to look at is made nothing, as the layer makes it. A recurrent layer's mask becomes the lengths of its series, and the graph gives nothing beyond them, as ONNX Runtime does.

The operator set

The graph asks for the oldest operator set that holds every node, and the IR version ONNX pairs with it: a graph of dense layers is operator set 7 and IR 3; a layer normalisation asks for 17, a GELU for 20, a root-mean-square normalisation for 23. A node that states its axes as an attribute in the older sets states them as an input when the graph's set is one that has them so.

What is refused

By name, and before anything is written:

• a network written as code — it declares no order its layers run in, so there is no graph to write it as;
• a layer that is written as no ONNX node here;
• a normalisation of the channels of every place that is a root-mean-square one: ONNX's operator for it normalises every axis from the one it names to the last;
• a row a flatten made of an image laid beside others, or added to another order: ONNX flattens an image channel by channel and the network place by place, so the two rows hold their values in two orders;
• a network that ends in a row a flatten made of an image, for the same reason;
• an adaptive pooling whose bins are of unequal sizes, which no ONNX operator pools.

Not a serving engine

A network trained behind a pipeline answers the rows the pipeline prepared. A runtime that takes the graph answers the rows it is handed, and the fitted verbs of a pipeline — a fill, a scale, an encoding — are not written into it. So the graph is the network and its output activation, and what it is handed is the pipeline's handover. A runtime that only answers is not an engine this library can train on, and nothing here asks it to be.

Next: Importing a model · The layers that are written: Networks