Skip to content

Latest commit

 

History

History
1980 lines (1628 loc) · 164 KB

Function-Level_Support_Status.rst

File metadata and controls

1980 lines (1628 loc) · 164 KB

Function-Level Support Status

ONNX Support Status

Note

In this document, the numbers in the header of all tables represent the version of onnx opset.

Import

  • ✓: onnx specification defined, and supported.
  • X: onnx specification defined, but not support yet.
  • Empty: Not defined (Support status follows latest).

Total: 95/159

ONNX Operator 1 2 3 4 5 6 7 8 9 10 11 12 13 NNabla Func Description

Abs Acos Acosh Add And ArgMax ArgMin Asin Asinh Atan Atanh AveragePool BatchNormalization BitShift Cast Ceil Celu Clip Compress Concat ConcatFromSequence Constant ConstantOfShape Conv ConvInteger ConvTranspose Cos Cosh CumSum DepthToSpace DequantizeLinear Det Div Dropout DynamicQuantizeLinear Einsum Elu Equal Erf Exp Expand EyeLike Flatten Floor GRU Gather GatherElements GatherND Gemm GlobalAveragePool GlobalLpPool GlobalMaxPool Greater GreaterOrEqual HardSigmoid Hardmax Identity If InstanceNormalization IsInf IsNaN LRN LSTM LeakyRelu Less LessOrEqual Log LogSoftmax Loop LpNormalization LpPool MatMul MatMulInteger Max MaxPool MaxRoiPool MaxUnpool Mean MeanVarianceNormalization Min Mod Mul Multinomial Neg NonMaxSuppression NonZero Not OneHot Or PRelu Pad Pow QLinearConv QLinearMatMul QuantizeLinear RNN RandomNormal RandomNormalLike RandomUniform RandomUniformLike Range Reciprocal ReduceL1 ReduceL2 ReduceLogSum ReduceLogSumExp ReduceMax ReduceMean ReduceMin ReduceProd ReduceSum ReduceSumSquare Relu Reshape Resize ReverseSequence RoiAlign Round Scan Scatter ScatterElements ScatterND Selu SequenceAt SequenceConstruct SequenceErase SequenceInsert SequenceLength Shape Shrink Sigmoid Sign Sin Sinh Size Slice Softmax Softplus Softsign SpaceToDepth Split SplitToSequence Sqrt Squeeze StringNormalizer Sub Sum Tan Tanh TfIdfVectorizer ThresholdedRelu Tile TopK Transpose Unique Unsqueeze Upsample Where Xor

✓ ✓ ✓ ✓

✓ X

✓ ✓

✓ ✓

✓ ✓

✓ ✓ X ✓

✓ ✓ X X ✓

✓ ✓ ✓ X ✓

✓ X ✓ ✓

✓ ✓ X X X ✓

✓ ✓ X

✓ ✓ ✓ ✓

X X X X X

✓ X X X X ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

X

X ✓ ✓ X ✓ ✓ ✓

✓ ✓

✓ ✓

✓ X ✓

✓ X

X

X

X

X

✓ ✓

✓ ✓

✓ ✓

✓ ✓

✓ ✓ ✓ ✓ ✓

✓ X

✓ ✓

✓ ✓

✓ ✓

✓ ✓

✓ ✓ X ✓

✓ ✓

✓ ✓ ✓

✓ X ✓ ✓

✓ ✓

✓ ✓

✓ X ✓

✓ ✓ ✓ ✓

X

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ X ✓ ✓ ✓

✓ ✓

✓ ✓ ✓ ✓

✓ X

✓ X

X

X

X

X

X ✓

X

✓ X ✓

X ✓ X ✓

X

X

X

X X

X

X

X

✓ ✓

X X

X

X

X

X

X

X

X

X X ✓

X X X

X

X

X

✓ ✓

X

X

✓ X X X X

X

X

X ✓

X

X

X

✓ X X ✓

✓ X

X

X

X

X

X

X

X X X X ✓ ✓ ✓ ✓ ✓ ✓

X

✓ X X X X

X X X X X

X ✓

✓ X

X

X ✓

✓ ✓

X ✓ ✓ ✓

✓ ✓

✓ ✓

X

X X ✓ X

✓ ✓

✓ X X ✓

✓ ✓

✓ ✓ X

✓ ✓

✓ ✓ ✓ ✓

✓ ✓

✓ X ✓

X

✓ ✓ X

X

X

✓ ✓

X X ✓

✓ ✓

✓ ✓

Abs ACos ACosh Add2, Reshape LogicalAnd, Reshape Flip, Max, RSubScalar Flip, Min, RSubScalar ASin ASinh ATan ATanh AveragePooling, Pad BatchNormalization

Ceil Add2, Constant, Div2, ELU, Exp, MaximumScalar, MinimumScalar, Mul2, MulScalar, Reshape, Sub2 Identity

Concatenate

Identity Constant Convolution

Deconvolution, Pad Cos Cosh

Reshape, Transpose DequantizeLinear

Div2, Reshape Identity

ELU Equal, Reshape

Exp Broadcast, Reshape

Reshape Floor

Concatenate, Slice

Add2, BatchMatmul, MulScalar, Reshape GlobalAveragePooling

Greater, Reshape Equal, Greater, GreaterEqual, LogicalOr, Reshape AddScalar, HardSigmoid, MaximumScalar, MinimumScalar, MulScalar Max, OneHot, Transpose Identity

BatchNormalization, Concatenate, Reshape, Split IsInf IsNaN AddScalar, Div2, MulScalar, PowScalar, SumPooling, Transpose

LeakyReLU Less, Reshape Equal, Less, LessEqual, LogicalOr, Reshape Log Div2, Exp, Log, Max, Sub2, Sum

BatchMatmul, Reshape

Maximum2 MaxPooling, Pad

Identity, Mean, Stack

Minimum2

Mul2, Reshape

MulScalar

LogicalNot

LogicalOr, Reshape PReLU Pad Pow2, Reshape

QuantizeLinear

RDivScalar

Max Mean Min Prod Sum PowScalar, Sum ReLU Reshape

Round

SELU

Sigmoid Sign Sin Sinh

Slice Div2, Exp, Max, Sub2, Sum SoftPlus SoftSign Reshape, Transpose Split, Stack

PowScalar Reshape

Reshape, Sub2 AddN Tan Tanh

Constant, GreaterScalar, Where Tile

Transpose

Reshape Unpooling Where LogicalXor, Reshape

Not all features are verified. Those features can be verified by ONNXRuntime when opset > 6. Some feature is not supported by Nnabla such as Pad's edge mode. if opset >= 10, the ceil_mode is not supported.

Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented. Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented. Not yet implemented.

Not yet implemented. Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented.

Not yet implemented.

Not all features are verified. Those features can be verified by ONNXRuntime. if opset >= 10, the ceil_mode is not supported, dilations is not equal to 1 is not supported. Not yet implemented. Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented. Not yet implemented.

Not yet implemented.

Onnx required to support "edge" mode, while nnabla does not support it.

Not yet implemented. Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented.

Not yet implemented.

Export

  • ✓: Support to export this opset.
  • △: Partially support to export this opset (e.g. some cases cannot be supported, or not completely tested).
  • X: Supported, but test failed.
  • Empty: Not support corresponding opset version.

Total: 124/215

Neural Network Layer

Count 11/18

NNabla Function 7 9 10 11 13 ONNX Op Description

Affine RNN LSTM GRU Convolution FusedConvolution DepthwiseConvolution Deconvolution DepthwiseDeconvolution DeformableConvolution AdaptiveSeparableConvolution MaxPooling

✓ ✓ ✓

✓ ✓ ✓

✓ ✓ ✓

✓ ✓ ✓

✓ ✓ ✓

Gemm, Reshape

Conv, Reshape

Conv, Reshape ConvTranspose, Reshape ConvTranspose, Reshape

Constant, MaxPool, Pad, Reshape

Not yet implemented. Not yet implemented. Not yet implemented.

Not yet implemented.

Not yet implemented. Not yet implemented.

AveragePooling GlobalAveragePooling SumPooling Unpooling Embed RoiAlign

△ ✓ ✓ ✓ ✓

△ ✓ ✓ ✓ ✓

△ ✓ ✓ ✓ ✓

△ ✓ ✓ ✓ ✓

△ ✓ ✓ ✓ ✓

AveragePool, Constant, Pad, Reshape GlobalAveragePool AveragePool, Constant, Mul, Pad, Reshape Resize Gather

Currently only supports the cases where both ignore_border and including_pad are True.

Not yet implemented.

Neural Network Activation Functions

Count 20/22

NNabla Function 7 9 10 11 13 ONNX Op Description

Sigmoid Swish Tanh ReLU LeakyReLU Softmax LogSoftmax ELU SELU CReLU CELU PReLU GELU Mish ReLU6 HardSigmoid HardTanh LogSigmoid

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓

Sigmoid Mul, Sigmoid Tanh Relu LeakyRelu Div, Exp, ReduceMax, ReduceSum, Sub Exp, Log, ReduceMax, ReduceSum, Sub Elu Selu Concat, Neg, Relu Concat, Elu, Neg PRelu, Reshape Add, Constant, Div, Mul, Pow, Sqrt, Tanh

Constant, Min, Relu HardSigmoid Constant, Max, Min, Neg Log, Sigmoid

Not yet implemented.

SoftPlus SoftSign TanhShrink Sinc

X ✓ ✓ X

X ✓ ✓ X

X ✓ ✓ X

X ✓ ✓ ✓

X ✓ ✓ ✓

Softplus Softsign Sub, Tanh Constant, Div, Equal, Sin, Where

Not yet implemented.

Normalization

Count 7/14

NNabla Function 7 9 10 11 13 ONNX Op Description

FusedBatchNormalization BatchNormalization GroupNormalization InstanceNormalization LayerNormalization NormNormalization SyncBatchNormalization TensorNormalization WeightNormalization WeightStandardization SpectralNorm MeanSubtraction ClipGradByValue ClipGradByNorm

✓ ✓

✓ ✓

✓ ✓ ✓

✓ ✓

✓ ✓

✓ ✓ ✓

✓ ✓

✓ ✓

✓ ✓ ✓

✓ ✓

✓ ✓

✓ ✓ ✓

✓ ✓

✓ ✓

✓ ✓ ✓

Add, BatchNormalization, Constant, Div, Mul, ReduceMean, ReduceSum, Relu, Reshape, Squeeze, Sub BatchNormalization, Constant, Div, Mul, ReduceMean, ReduceSum, Reshape, Squeeze, Sub

Add, Constant, Div, Mul, Pow, ReduceMean, ReduceSum, Reshape, Sub Add, Constant, Div, Mul, Pow, ReduceMean, ReduceSum, Sub

Add, Constant, Mul, Pow, ReduceSum, Reshape Add, Constant, Div, Mul, Pow, ReduceMean, ReduceSum, Sub Add, Constant, Div, Gemm, Pow, ReduceSum, Reshape, Sqrt, Transpose

Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented.

Reduction

Count 5/10

NNabla Function 7 9 10 11 13 ONNX Op Description

Sum CumSum Mean Max Min Norm Prod CumProd ReduceSum ReduceMean

✓ ✓ ✓

✓ ✓ ✓

✓ ✓ ✓

✓ ✓ ✓

✓ ✓ ✓

ReduceSum

ReduceMean ReduceMax ReduceMin

ReduceProd

Not yet implemented.

Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented.

Arithmetic

Count 11/14

NNabla Function 7 9 10 11 13 ONNX Op Description

Add2 AddN BcAdd2 Sub2 Mul2 MulN Div2 Pow2 AddScalar MulScalar PowScalar RSubScalar RDivScalar RPowScalar

✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Add

Sub Mul

Div Pow Add, Constant Constant, Mul Constant, Pow Constant, Sub Constant, Div Constant, Pow

Not yet implemented. Not yet implemented.

Not yet implemented.

Logical

Count 29/30

NNabla Function 7 9 10 11 13 ONNX Op Description

Sign Minimum2 Maximum2 MinimumScalar MaximumScalar LogicalAnd LogicalOr LogicalXor Equal NotEqual GreaterEqual Greater LessEqual Less SearchSorted LogicalAndScalar LogicalOrScalar LogicalXorScalar EqualScalar NotEqualScalar GreaterEqualScalar GreaterScalar LessEqualScalar LessScalar LogicalNot IsNaN IsInf ResetNaN ResetInf Where

X ✓ ✓ ✓ ✓ ✓ ✓ ✓ X X ✓ ✓ ✓ ✓

✓ ✓ ✓ X X ✓ ✓ ✓ ✓ ✓ X X X X X

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ X X ✓ ✓ ✓ ✓

✓ ✓ ✓ X X ✓ ✓ ✓ ✓ ✓ ✓ X ✓ X ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ X X ✓ ✓ ✓ ✓

✓ ✓ ✓ X X ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Sign Add, Constant, Min Add, Constant, Max Add, Constant, Min Add, Constant, Max And Or Xor Equal Equal, Not Less, Not Greater Greater, Not Less

And, Constant Constant, Or Constant, Xor Constant, Equal Constant, Equal, Not Constant, Less, Not Constant, Greater Constant, Greater, Not Constant, Less Not IsNaN IsInf Constant, IsNaN, Where Constant, IsInf, Where Where

Not yet implemented.

Math

Count 22/22

NNabla Function 7 9 10 11 13 ONNX Op Description

Constant Arange Abs Exp Log Identity BatchMatmul Round Ceil Floor Sin Cos Tan Sinh Cosh ASin ACos ATan ATan2 ASinh ACosh ATanh

✓ ✓ ✓ ✓ ✓ ✓ ✓ X ✓ ✓ ✓ ✓ ✓ X X ✓ ✓ ✓ ✓ X X X

✓ ✓ ✓ ✓ ✓ ✓ ✓ X ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ X ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Constant, Identity Constant, Identity Abs Exp Log Identity MatMul, Transpose Round Ceil Floor Sin Cos Tan Sinh Cosh Asin Acos Atan Atan, Div Asinh Acosh Atanh

Array Manipulation

Count 12/30

NNabla Function 7 9 10 11 13 ONNX Op Description

Concatenate Split Stack

✓ ✓ ✓

✓ ✓ ✓

✓ ✓ ✓

✓ ✓ ✓

✓ ✓ ✓

Concat Split, Squeeze Concat, Unsqueeze

Slice

Constant, Slice ONNX slice cannot support step != 1 on opset < 10.

Pad Transpose Broadcast BroadcastTo Tile OneHot Flip Shift Sort Reshape MatrixDiag MatrixDiagPart Meshgrid BatchDet BatchInv BatchLogdet Assign Gather GatherNd BoolGather ScatterNd ScatterAdd BoolScatter BoolFill PackPaddedSequence PadPackedSequence

△ ✓ X ✓ ✓ X ✓

△ ✓ ✓ ✓ ✓ ✓ ✓

△ ✓ ✓ ✓ ✓ ✓ ✓

△ ✓ ✓ ✓ ✓ ✓ ✓

△ ✓ ✓ ✓ ✓ ✓ ✓

Constant, Pad Transpose

Constant, Reshape, Tile Flatten, Gather, Reshape Gather, Identity, Transpose

Constant, Reshape

When the mode of the pad is reflect, if the size of the pad exceeds the input size, onnxruntime cannot handle it.

Not yet implemented. Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Signal Processing

Count 1/5

NNabla Function 7 9 10 11 13 ONNX Op Description

Interpolate FFT IFFT STFT ISTFT

X

X

Resize

Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Stochasticity

Count 0/15

NNabla Function 7 9 10 11 13 ONNX Op Description

Dropout TopKData TopKGrad Rand Randint Randn RandBinomial RandBeta RandGamma RandomChoice RandomCrop RandomFlip RandomShift RandomErase ImageAugmentation

X

X

X

X

X

Dropout

The Dropout in nnabla has no test mode and contains random parameters, so the test result is not the same as onnx. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Loss Functions

Count 0/9

NNabla Function 7 9 10 11 13 ONNX Op Description

SigmoidCrossEntropy BinaryCrossEntropy SoftmaxCrossEntropy CategoricalCrossEntropy SquaredError AbsoluteError HuberLoss EpsilonInsensitiveLoss KLMultinomial

Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Geometric Neural Network Layers

Count 0/3

NNabla Function 7 9 10 11 13 ONNX Op Description

AffineGrid WarpByGrid WarpByFlow

Not yet implemented. Not yet implemented. Not yet implemented.

Quantization Neural Network Layers

Count 6/14

NNabla Function 7 9 10 11 13 ONNX Op Description

BinarySigmoid BinaryTanh BinaryConnectAffine BinaryConnectConvolution BinaryWeightAffine BinaryWeightConvolution INQAffine INQConvolution FixedPointQuantize MinMaxQuantize Pow2Quantize Prune QuantizeLinear DequantizeLinear

X X ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓

✓ ✓

✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓

✓ ✓

Constant, Greater, Where Constant, Greater, Where Gemm, Reshape Conv, Reshape Add, MatMul, Mul, Reshape Add, Conv, Mul, Reshape

Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Validation

Count 0/3

NNabla Function 7 9 10 11 13 ONNX Op Description

TopNError BinaryError ConfusionMatrix

Not yet implemented. Not yet implemented. Not yet implemented.

Unsupported, Special Use

Count 0/6

NNabla Function 7 9 10 11 13 ONNX Op Description

VATNoise Unlink Sink NmsDetection2d MaxPoolingBackward PatchCorrelation

Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Tensorflow Support Status

Import

  • ✓: Supported
  • △: Partially supported
  • X: Supported, but test failed.
  • Empty: Not support yet.

Total: 106/122

Tensorflow support status
Tensorflow Function Status NNabla Func Description

Abs Acos Acosh Add AddN All Any ArgMax ArgMin Asin Asinh Atan Atan2 Atanh AvgPool AvgPool3D BatchMatMul BatchNormalization BiasAdd BroadcastTo

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ △ △ ✓ ✓ ✓ ✓

Abs ACos ACosh Add2 AddN Greater, Min, Reshape Greater, Reshape, Sum Max Min ASin ASinh ATan ATan, Add2, Div2, Mul2, Reshape, Sign, Sub2 ATanh AveragePooling, Pad, Transpose AveragePooling, Pad, Transpose BatchMatmul, Transpose Add2, Mul2, PowScalar, RDivScalar, Reshape, Sub2 Add2, Reshape

Cast Ceil ClipByValue Concat ConcatV2 Const Conv1D Conv1DTranspose Conv2D Conv2DBackpropInput Conv3D Conv3DBackpropInput Cos Cosh Crelu Cumsum DepthToSpace DepthwiseConv2d Div Elu Equal Erf Erfc Exp ExpandDims Floor FloorDiv FloorMod GatherNd

X ✓ ✓ ✓ ✓ ✓ △ △ △ △ △ △ ✓ ✓ ✓ X △ △ ✓ ✓ ✓ X X ✓ ✓ ✓ ✓ ✓ X

NA Ceil Maximum2, Minimum2, Reshape Concatenate Concatenate NA Convolution, Pad, Reshape, Transpose Deconvolution, Reshape, Transpose Convolution, Pad, Transpose Deconvolution, Transpose Convolution, Pad, Transpose Deconvolution, Pad, Transpose Cos Cosh Concatenate, MulScalar, ReLU

Reshape, Transpose Convolution, Pad, Reshape, Transpose Div2 ELU Equal

Exp Reshape Floor Div2, Floor Div2, Floor, Mul2, Sub2

Not yet implemented.

Not yet implemented.

Not yet implemented. Not yet implemented.

Not yet implemented.

GatherV2 Greater GreaterEqual Identity IsInf IsNan LeakyRelu Less LessEqual Log

X ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Concatenate, Slice Greater Less, LogicalNot Identity IsInf IsNaN LeakyReLU Less Greater, LogicalNot Log

Not yet implemented.

LogSigmoid LogSoftmax LogicalAnd LogicalNot LogicalOr LogicalXor Max MaxPool MaxPool3D MaxPoolWithArgmax Maximum Mean Min Minimum Mul Neg NotEqual Pack Pad Pow Prod RealDiv Reciprocal Relu Relu6 Reshape ReverseSequence ReverseV2 Round Rsqrt Selu Shape Sigmoid Sign Sin Sinh Size Slice Softmax

X ✓ ✓ ✓ ✓ ✓ ✓ △ △ X ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ △ ✓ ✓ ✓ ✓ ✓ ✓ ✓ X X ✓ ✓ ✓ X ✓ ✓ ✓ ✓ X ✓ ✓

MulScalar, SoftPlus Add2, Exp, Log, Max, Reshape, Sub2, Sum, Transpose LogicalAnd LogicalNot LogicalOr LogicalAnd, LogicalNot, LogicalOr Max MaxPooling, Pad, Reshape, Transpose MaxPooling, Pad, Transpose

Maximum2 Mean Min Minimum2 Mul2 MulScalar Equal, LogicalNot Concatenate, Reshape Pad Pow2 Prod Div2 RDivScalar ReLU MaximumScalar, MinimumScalar Reshape

Round PowScalar, RDivScalar SELU

Sigmoid Sign Sin Sinh

Slice Div2, Exp, Max, Reshape, Sub2, Sum, Transpose

Not yet implemented.

Not yet implemented.

Not yet implemented. Not yet implemented.

Not yet implemented.

Not yet implemented.

Softplus Softsign SpaceToDepth Split SplitV Sqrt Square SquaredDifference Squeeze StopGradient StridedSlice Sub Sum Swish Tan Tanh Tile TopKV2 Transpose

X ✓ △ ✓ ✓ ✓ ✓ ✓ ✓ ✓ △ ✓ ✓ ✓ ✓ ✓ ✓ X ✓

SoftPlus SoftSign Reshape, Transpose Split, Stack Split, Stack PowScalar Mul2 Mul2, Sub2 Reshape Identity Slice Sub2 Sum Mul2, Sigmoid Tan Tanh Tile

Transpose

Not yet implemented.

Not yet implemented.

TruncateDiv TruncateMod Unpack Where ZerosLike

X X ✓ △ ✓

Div2

Reshape, Split, Stack Where NA

Not yet implemented. Not yet implemented.

Export

  • ✓: Supported
  • △: Partially supported
  • X: Supported, but test failed.
  • Empty: Not support yet.

Total: 124/215

Neural Network Layer

Count 11/18

NNabla Function Status Description

Affine RNN LSTM GRU

Not yet implemented. Not yet implemented. Not yet implemented.

Convolution FusedConvolution

The cases dilations and strides larger than 1 are not supported by tensorflow. Not yet implemented.

DepthwiseConvolution

The cases dilations and strides larger than 1 are not supported by tensorflow.

Deconvolution

The cases dilations larger than 1 are not supported by tensorflow.

DepthwiseDeconvolution DeformableConvolution AdaptiveSeparableConvolution MaxPooling

The cases dilations larger than 1 are not supported by tensorflow. Not yet implemented. Not yet implemented.

AveragePooling GlobalAveragePooling SumPooling

△ ✓ ✓

Currently only supports the cases both ignore_border and including_pad are True.

Unpooling Embed RoiAlign

△ ✓

The kernel only supports 2d.

Not yet implemented.

Neural Network Activation Functions

Count 20/22

NNabla Function Status Description

Sigmoid Swish Tanh ReLU LeakyReLU Softmax LogSoftmax ELU SELU CReLU CELU PReLU GELU Mish ReLU6 HardSigmoid HardTanh LogSigmoid

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ △ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓

Not yet implemented.

SoftPlus SoftSign TanhShrink Sinc

X ✓ ✓ ✓

Not yet implemented.

Normalization

Count 7/14

NNabla Function Status Description

FusedBatchNormalization BatchNormalization GroupNormalization InstanceNormalization LayerNormalization NormNormalization SyncBatchNormalization TensorNormalization WeightNormalization WeightStandardization SpectralNorm MeanSubtraction ClipGradByValue ClipGradByNorm

✓ ✓

✓ ✓

✓ ✓ ✓

Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented.

Reduction

Count 5/10

NNabla Function Status Description

Sum CumSum Mean Max Min Norm Prod CumProd ReduceSum ReduceMean

✓ ✓ ✓

Not yet implemented.

Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented.

Arithmetic

Count 11/14

NNabla Function Status Description

Add2 AddN BcAdd2 Sub2 Mul2 MulN Div2 Pow2 AddScalar MulScalar PowScalar RSubScalar RDivScalar RPowScalar

✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Not yet implemented. Not yet implemented.

Not yet implemented.

Logical

Count 29/30

NNabla Function Status Description

Sign Minimum2 Maximum2 MinimumScalar MaximumScalar LogicalAnd LogicalOr LogicalXor Equal NotEqual GreaterEqual Greater LessEqual Less SearchSorted LogicalAndScalar LogicalOrScalar LogicalXorScalar EqualScalar NotEqualScalar GreaterEqualScalar GreaterScalar LessEqualScalar LessScalar LogicalNot IsNaN IsInf ResetNaN ResetInf Where

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Not yet implemented.

Math

Count 22/22

NNabla Function Status Description

Constant Arange Abs Exp Log Identity BatchMatmul Round Ceil Floor Sin Cos Tan Sinh Cosh ASin ACos ATan ATan2 ASinh ACosh ATanh

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Array Manipulation

Count 12/30

NNabla Function Status Description

Concatenate Split Stack Slice

✓ ✓ ✓ ✓

Pad Transpose Broadcast BroadcastTo Tile OneHot Flip Shift Sort Reshape MatrixDiag MatrixDiagPart Meshgrid BatchDet BatchInv BatchLogdet Assign Gather GatherNd BoolGather ScatterNd ScatterAdd BoolScatter BoolFill PackPaddedSequence PadPackedSequence

△ ✓ ✓ ✓ ✓ ✓ ✓

When the mode of the pad is reflect, if the size of the pad exceeds the input size, tensorflow cannot handle it.

Not yet implemented. Not yet implemented.

Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Signal Processing

Count 1/5

NNabla Function Status Description

Interpolate FFT IFFT STFT ISTFT

Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Stochasticity

Count 0/15

NNabla Function Status Description

Dropout TopKData TopKGrad Rand Randint Randn RandBinomial RandBeta RandGamma RandomChoice RandomCrop RandomFlip RandomShift RandomErase ImageAugmentation

X

The Dropout in nnabla has no test mode and contains random parameters, so the test result is not the same as tensorflow. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Loss Functions

Count 0/9

NNabla Function Status Description

SigmoidCrossEntropy BinaryCrossEntropy SoftmaxCrossEntropy CategoricalCrossEntropy SquaredError AbsoluteError HuberLoss EpsilonInsensitiveLoss KLMultinomial

Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Geometric Neural Network Layers

Count 0/3

NNabla Function Status Description

AffineGrid WarpByGrid WarpByFlow

Not yet implemented. Not yet implemented. Not yet implemented.

Quantization Neural Network Layers

Count 6/14

NNabla Function Status Description

BinarySigmoid BinaryTanh BinaryConnectAffine

✓ ✓ ✓

BinaryConnectConvolution BinaryWeightAffine

△ ✓

The cases dilations and strides larger than 1 are not supported by tensorflow.

BinaryWeightConvolution INQAffine INQConvolution FixedPointQuantize MinMaxQuantize Pow2Quantize Prune QuantizeLinear DequantizeLinear

The cases dilations and strides larger than 1 are not supported by tensorflow. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Validation

Count 0/3

NNabla Function Status Description

TopNError BinaryError ConfusionMatrix

Not yet implemented. Not yet implemented. Not yet implemented.

Unsupported, Special Use

Count 0/6

NNabla Function Status Description

VATNoise Unlink Sink NmsDetection2d MaxPoolingBackward PatchCorrelation

Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented. Not yet implemented.

Tensorflow Lite Support Status

Export

  • ✓: Supported
  • △: Partially supported
  • X: Supported, but test failed.
  • Empty: Not support yet.

Total: 82/215

Neural Network Layer

Count 11/18

NNabla Function Status

Affine RNN LSTM GRU

Convolution FusedConvolution

DepthwiseConvolution

Deconvolution

DepthwiseDeconvolution DeformableConvolution AdaptiveSeparableConvolution

MaxPooling

AveragePooling

GlobalAveragePooling

SumPooling

Unpooling

Embed RoiAlign

Neural Network Activation Functions

Count 10/22

NNabla Function Status

Sigmoid

Swish

Tanh

ReLU

LeakyReLU

Softmax

LogSoftmax

ELU

SELU

X

CReLU

X

CELU

X

PReLU

GELU Mish

X

ReLU6

HardSigmoid

X

HardTanh

X

LogSigmoid

X

SoftPlus

X

SoftSign

X

TanhShrink

X

Sinc

X

Normalization

Count 1/14

NNabla Function Status

FusedBatchNormalization

X

BatchNormalization GroupNormalization

InstanceNormalization

X

LayerNormalization NormNormalization SyncBatchNormalization TensorNormalization

X

WeightNormalization

X

WeightStandardization

X

SpectralNorm MeanSubtraction ClipGradByValue ClipGradByNorm

X

Reduction

Count 5/10

NNabla Function Status

Sum CumSum

Mean

Max

Min Norm

Prod CumProd ReduceSum ReduceMean

Arithmetic

Count 11/14

NNabla Function Status

Add2 AddN BcAdd2

Sub2

Mul2 MulN

Div2

Pow2

AddScalar

MulScalar

PowScalar

RSubScalar

RDivScalar

RPowScalar

Logical

Count 23/30

NNabla Function Status

Sign

X

Minimum2

Maximum2

MinimumScalar

MaximumScalar

LogicalAnd

LogicalOr

LogicalXor

Equal

NotEqual

GreaterEqual

Greater

LessEqual

Less SearchSorted

LogicalAndScalar

LogicalOrScalar

LogicalXorScalar

EqualScalar

NotEqualScalar

GreaterEqualScalar

GreaterScalar

LessEqualScalar

LessScalar

LogicalNot

IsNaN

X

IsInf

X

ResetNaN

X

ResetInf

X

Where

X

Math

Count 10/22

NNabla Function Status

Constant

X

Arange

X

Abs

Exp

Log

Identity

X

BatchMatmul

Round

Ceil

Floor

Sin

Cos

Tan

Sinh

X

Cosh

X

ASin

X

ACos

X

ATan

X

ATan2

X

ASinh

X

ACosh

X

ATanh

X

Array Manipulation

Count 10/30

NNabla Function Status

Concatenate

Split

Stack

Slice

Pad

Transpose

Broadcast

BroadcastTo

X

Tile

OneHot

X

Flip Shift Sort

Reshape MatrixDiag MatrixDiagPart Meshgrid BatchDet BatchInv BatchLogdet Assign Gather GatherNd BoolGather ScatterNd ScatterAdd BoolScatter BoolFill PackPaddedSequence PadPackedSequence

Signal Processing

Count 1/5

NNabla Function Status

Interpolate FFT IFFT STFT ISTFT

Stochasticity

Count 0/15

NNabla Function Status

Dropout TopKData TopKGrad Rand Randint Randn RandBinomial RandBeta RandGamma RandomChoice RandomCrop RandomFlip RandomShift RandomErase ImageAugmentation

X

Loss Functions

Count 0/9

NNabla Function Status

SigmoidCrossEntropy BinaryCrossEntropy SoftmaxCrossEntropy CategoricalCrossEntropy SquaredError AbsoluteError HuberLoss EpsilonInsensitiveLoss KLMultinomial

Geometric Neural Network Layers

Count 0/3

NNabla Function Status

AffineGrid WarpByGrid WarpByFlow

Quantization Neural Network Layers

Count 0/14

NNabla Function Status

BinarySigmoid

X

BinaryTanh

X

BinaryConnectAffine

X

BinaryConnectConvolution

X

BinaryWeightAffine

X

BinaryWeightConvolution INQAffine INQConvolution FixedPointQuantize MinMaxQuantize Pow2Quantize Prune QuantizeLinear DequantizeLinear

X

Validation

Count 0/3

NNabla Function Status

TopNError BinaryError ConfusionMatrix

Unsupported, Special Use

Count 0/6

NNabla Function Status

VATNoise Unlink Sink NmsDetection2d MaxPoolingBackward PatchCorrelation

NNabla C Runtime Support Status

NNabla version: None

  • ✓: Supported
  • △: Partially supported
  • X: Supported, but test failed or no test data.
  • Empty: Not support yet.

Export

Total: 56/215

Neural Network Layer

Count 8/18

NNabla Function Status Description

Affine RNN LSTM GRU Convolution FusedConvolution DepthwiseConvolution Deconvolution DepthwiseDeconvolution DeformableConvolution AdaptiveSeparableConvolution MaxPooling AveragePooling GlobalAveragePooling SumPooling Unpooling Embed RoiAlign

✓ △

△ ✓

✓ ✓

Neural Network Activation Functions

Count 11/22

NNabla Function Status Description

Sigmoid Swish Tanh ReLU LeakyReLU Softmax LogSoftmax ELU SELU CReLU CELU PReLU GELU Mish ReLU6 HardSigmoid HardTanh LogSigmoid SoftPlus SoftSign TanhShrink Sinc

✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ △ ✓

Normalization

Count 1/14

NNabla Function Status Description

FusedBatchNormalization BatchNormalization GroupNormalization InstanceNormalization LayerNormalization NormNormalization SyncBatchNormalization TensorNormalization WeightNormalization WeightStandardization SpectralNorm MeanSubtraction ClipGradByValue ClipGradByNorm

X

X

Reduction

Count 1/10

NNabla Function Status Description

Sum CumSum Mean Max Min Norm Prod CumProd ReduceSum ReduceMean

Arithmetic

Count 11/14

NNabla Function Status Description

Add2 AddN BcAdd2 Sub2 Mul2 MulN Div2 Pow2 AddScalar MulScalar PowScalar RSubScalar RDivScalar RPowScalar

✓ X

✓ ✓ X ✓ ✓ ✓ ✓ ✓ △ ✓ ✓

Logical

Count 5/30

NNabla Function Status Description

Sign Minimum2 Maximum2 MinimumScalar MaximumScalar LogicalAnd LogicalOr LogicalXor Equal NotEqual GreaterEqual Greater LessEqual Less SearchSorted LogicalAndScalar LogicalOrScalar LogicalXorScalar EqualScalar NotEqualScalar GreaterEqualScalar GreaterScalar LessEqualScalar LessScalar LogicalNot IsNaN IsInf ResetNaN ResetInf Where

✓ ✓ ✓ ✓ ✓

Math

Count 6/22

NNabla Function Status Description

Constant Arange Abs Exp Log Identity BatchMatmul Round Ceil Floor Sin Cos Tan Sinh Cosh ASin ACos ATan ATan2 ASinh ACosh ATanh

✓ ✓ ✓ ✓ △ ✓

Array Manipulation

Count 7/30

NNabla Function Status Description

Concatenate Split Stack Slice Pad Transpose Broadcast BroadcastTo Tile OneHot Flip Shift Sort Reshape MatrixDiag MatrixDiagPart Meshgrid BatchDet BatchInv BatchLogdet Assign Gather GatherNd BoolGather ScatterNd ScatterAdd BoolScatter BoolFill PackPaddedSequence PadPackedSequence

✓ ✓ △ △

✓ X

✓ X X

Signal Processing

Count 0/5

NNabla Function Status Description

Interpolate FFT IFFT STFT ISTFT

Stochasticity

Count 0/15

NNabla Function Status Description

Dropout TopKData TopKGrad Rand Randint Randn RandBinomial RandBeta RandGamma RandomChoice RandomCrop RandomFlip RandomShift RandomErase ImageAugmentation

X

Loss Functions

Count 0/9

NNabla Function Status Description

SigmoidCrossEntropy BinaryCrossEntropy SoftmaxCrossEntropy CategoricalCrossEntropy SquaredError AbsoluteError HuberLoss EpsilonInsensitiveLoss KLMultinomial

Geometric Neural Network Layers

Count 0/3

NNabla Function Status Description

AffineGrid WarpByGrid WarpByFlow

Quantization Neural Network Layers

Count 6/14

NNabla Function Status Description

BinarySigmoid BinaryTanh BinaryConnectAffine BinaryConnectConvolution BinaryWeightAffine BinaryWeightConvolution INQAffine INQConvolution FixedPointQuantize MinMaxQuantize Pow2Quantize Prune QuantizeLinear DequantizeLinear

✓ ✓ ✓ ✓ ✓ △

Validation

Count 0/3

NNabla Function Status Description

TopNError BinaryError ConfusionMatrix

Unsupported, Special Use

Count 0/6

NNabla Function Status Description

VATNoise Unlink Sink NmsDetection2d MaxPoolingBackward PatchCorrelation