AttributeHasValue and StaticOptionalHasValue

[gramalingam]

This is a proposal to add a couple of primitive operators, whose primary purpose is to allow us to define other ops as functions. It is intended to specifically help address scenarios where the op/function has optional attributes or optional inputs, and we need to check if the optional attribute/input was supplied or not.

New Operator

(a) AttributeHasValue() : an operator that returns true if it has an attribute specified.
(b) StaticOptionalHasValue(X): an operator that returns true if an optional input is specified.

Describe the operator

As mentioned above, this is intended to enable function-definition for ops that have optional
attributes or inputs which determine the behavior of the function.

Can this operator be constructed using existing onnx operators?

No.

Is this operator used by any model currently? Which one?

There are examples of ops: eg., CumSum has two optional attributes. Squeeze is an example of an op with an optional input.

Are you willing to contribute it? (Y/N)

Y

Notes

The existing mechanism of context-dependent functions is a workaround for the above issue. However, context-dependent functions have their own disadvantages. Eg., it is not easy to embed their function expansion as a model-local function. The above primitives will allow us to express such functions as context-free functions, which are easy to export as a model-local function.

[gramalingam]

Some more info about part (b) described above. There are two kinds of optional inputs in ONNX. (Eg: see #4301 ). The first kind originally used in ONNX is not part of the type of the input (eg., input ratio of Dropout), while the second kind incorporates optional as part of the type (see ops Optional, OptionalHasElement and OptionalGetElement).

The second kind (and associated ops) are meant to serve a similar purpose as described in this issue (to write conditional logic dependent on whether an optional input is specified or not)). However, these ops cannot be used with optional inputs of the first kind because of their current typing.

Proposal: generalize the ops OptionalHasElement and OptionalGetElement to allow optional inputs of the first kind as well. This will help unify the two kinds of optional inputs.

[gramalingam]

PR 4421 completes part (b).

As for part (a), the conclusion for now is that it can be managed using suitable default-values for attributes, and, hence, this extension is not required.

[gramalingam]

Reopening this for part (a).

A good example of where this is required is in defining the function body for ReduceSumSquare, which can be defined in terms of ReduceSum. This, however, requires promoting the optional attribute axes into an optional input. There is no easy way of doing this. Providing AttributeHasValue as an op will enable this. (Note that this op can be defined as a context-dependent function in ONNX.)