Constant have no input which is weird for an operator

[owulveryck]

I have a PR against onnx-go to implement the Constant Operator in the Gorgonia backend.

The test I have fails because it creates an empty graph.
It is because the Constant Operator does not have any input.

Is it an expected behavior to be able to define an operator without input? It sounds counter-intuitive. My understanding is that its usage can be avoided by using Initializers.

Are they any other operators "input-less"?

WDYT?

[gramalingam]

Yes, it is reasonable to have an operator with no inputs. Of course, the only case where it makes sense is where the output is a constant or is a randomly-generated tensor. Thus, RandomUniform and RandomNormal are two other examples of an operator with no inputs.

Why is your test failing? There are "optimizer" passes that convert uses of "Constant" to replace it with an Initializer (just like you mention). It is possible that this creates an empty graph, which causes something to fail. But it sounds like an error in some other code.

[owulveryck]

Thanks for your reply.

Why is your test failing?

onnx-go works somehow like Keras; it is independent of the execution backend.
It is decoding the protobuf structure. Then it creates a graph object in Go (a weighted directed graph, the weights of the edges allows to keep the order of the input for non-commutative operations).

Each node of the graph is either a DataCarrier or an Operation.

Then a backend (here, a particular implementation based on Gorgonia) walks the graph to create its internal structure.
The problem arises then Gorgonia tries to run the graph. In the test of the constant, the graph is made of a single node, which is an operator. It does not have any placeholder for a tensor (a DataCarrier object); therefore, it panics because it does not know how to process this weird graph.

The main issue is a design problem. Some checks are made against the arity of the operators. But I never considered that any operator could have an arity of zero (neither did Gorgonia I think).

There are "optimizer" passes that convert uses of "Constant" to replace it with an Initializer

This can be implemented as a workaround in the main decoder package (onnx-go). Thanks for the tip.

[owulveryck]

I am closing this issue as I have all the required information to implement this in onnx-go.
Thanks