File Format

Complete documentation of the .nnn file format used by Neural Network Notions.

General Formatting Notes

• All multi-byte numbers use little-endian encoding.
• Each layer type's encoding includes different parameters - identified by Layer ID.
• Certain activations encode additional parameters (eg. Leaky ReLU's Tau) immediately after their ID - identified by Activation ID.
• Data appears in the file in the exact order as listed below.

File Header Format

• Magic Number -> int32 (4 bytes) -> 776883790 (spells .NNN in ASCII)
• Description -> string (see formatting) -> short description of the model in the file
• Parameter Count -> unsigned int64 (8 bytes) -> total number of parameter values across all parameter tensors in the model

Model Header Format

• Layer Count -> int32 (4 bytes) -> number of layers in the model

Layer Header Format

• Layer ID -> unsigned byte -> ID of the specific layer type (see ID list)

Layer Data Format

• Shared - always comes before type-specific data:

  • Activation ID -> unsigned byte -> ID of the layer's activation function (see ID list)
  • Dropout -> float (4 bytes) -> dropout parameter of the layer
  • Bias -> tensor (see formatting) -> bias parameter of the layer

• Dense:

  • Neuron Count -> int32 (4 bytes) -> number of neurons in the layer
  • Weights -> tensor (see formatting) -> weights parameter of the layer
  • Flatten -> boolean (see formatting) -> whether the layer flattens its input prior to applying weights

• Conv:

  • Filter Count -> int32 (4 bytes) -> number of filters in the layer
  • Kernels -> tensor (see formatting) -> kernels parameter of the layer ([f, h, w..., c] ordering)
  • PaddingType -> int32 (4 bytes) -> type of padding used by the layer
  • PaddingDims -> int32 array (see formatting) -> padding dimensions applied by the layer

Activation Function Data Format

Found immediately after Activation ID - for activation functions with parameters

• Leaky ReLU:

  • Tau -> float (4 bytes) -> tau parameter of the function

Tensor Format

• Dimensions -> int32 array (see formatting) -> dimensions of the tensor
• RequiresGrad -> boolean (see formatting) -> whether the tensor requires gradients to be calculated
• Data -> float array (see formatting) -> linear array of the tensor's data values (row-major ordering)

Primitive Data Type Formats

• String Format (UTF8):

  • Length -> int32 (4 bytes) -> number of characters in the string
  • Characters -> byte[Length] -> UTF8 character bytes

• Int32 Array Format:

  • Length -> int32 (4 bytes) -> number of elements in the array
  • Elements -> int32[Length] (4 bytes each) -> elements in the array

• Float Array Format:

  • Length -> int32 (4 bytes) -> number of elements in the array
  • Elements -> float[Length] (4 bytes each) -> elements in the array

• Boolean Format:

  • 1 bytes -> 1 = true, 0 = false

Layer IDs

• 0 -> Dense
• 1 -> Conv

Activation IDs

• 0 -> Linear
• 1 -> Sigmoid
• 2 -> Tanh
• 3 -> ReLU
• 4 -> Leaky ReLU
• 5 -> Softmax