diff --git a/docs/AddNewOp.md b/docs/AddNewOp.md index bae367d78f9..37307eacdc7 100644 --- a/docs/AddNewOp.md +++ b/docs/AddNewOp.md @@ -66,7 +66,7 @@ Once the criteria of proposing new operator/function has been satisfied, you wil 1. Write a detailed description about the operator, and its expected behavior. Pretty much, the description should be clear enough to avoid confusion between implementors. 2. Add an example in the description to illustrate the usage. 3. Add reference to the source of the operator in the corresponding framework in the description (if possible). - 4. Write the mathematic formula or a pseudocode in the description. The core algorithm needs to be very clear. + 4. Write the mathematical formula or a pseudocode in the description. The core algorithm needs to be very clear. 2. Write a reference implementation in Python, this reference implementation should cover all the expected behavior of the operator. Only in extremely rare case, we will waive this requirement. 3. Operator version: check out our [versioning doc](/docs/Versioning.md#operator-versioning) diff --git a/docs/Changelog-ml.md b/docs/Changelog-ml.md index 209ebdea821..de2e76e8e09 100644 --- a/docs/Changelog-ml.md +++ b/docs/Changelog-ml.md @@ -1226,3 +1226,124 @@ This version of the operator has been available since version 4 of the 'ai.onnx.
Output type is determined by the specified 'values_*' attribute.
+## Version 5 of the 'ai.onnx.ml' operator set +### **ai.onnx.ml.TreeEnsemble-5** + + Tree Ensemble operator. Returns the regressed values for each input in a batch. + Inputs have dimensions `[N, F]` where `N` is the input batch size and `F` is the number of input features. + Outputs have dimensions `[N, num_targets]` where `N` is the batch size and `num_targets` is the number of targets, which is a configurable attribute. + + The encoding of this attribute is split along interior nodes and the leaves of the trees. Notably, attributes with the prefix `nodes_*` are associated with interior nodes, and attributes with the prefix `leaf_*` are associated with leaves. + The attributes `nodes_*` must all have the same length and encode a sequence of tuples, as defined by taking all the `nodes_*` fields at a given position. + + All fields prefixed with `leaf_*` represent tree leaves, and similarly define tuples of leaves and must have identical length. + + This operator can be used to implement both the previous `TreeEnsembleRegressor` and `TreeEnsembleClassifier` nodes. + The `TreeEnsembleRegressor` node maps directly to this node and requires changing how the nodes are represented. + The `TreeEnsembleClassifier` node can be implemented by adding a `ArgMax` node after this node to determine the top class. + To encode class labels, a `LabelEncoder` or `GatherND` operator may be used. + +#### Version + +This version of the operator has been available since version 5 of the 'ai.onnx.ml' operator set. + +#### Attributes + +
+
aggregate_function : int (default is 1)
+
Defines how to aggregate leaf values within a target.
One of 'AVERAGE' (0) 'SUM' (1) 'MIN' (2) 'MAX (3) defaults to 'SUM' (1)
+
leaf_targetids : list of ints (required)
+
The index of the target that this leaf contributes to (this must be in range `[0, n_targets)`).
+
leaf_weights : tensor (required)
+
The weight for each leaf.
+
membership_values : tensor
+
Members to test membership of for each set membership node. List all of the members to test again in the order that the 'BRANCH_MEMBER' mode appears in `node_modes`, delimited by `NaN`s. Will have the same number of sets of values as nodes with mode 'BRANCH_MEMBER'. This may be omitted if the node doesn't contain any 'BRANCH_MEMBER' nodes.
+
n_targets : int
+
The total number of targets.
+
nodes_falseleafs : list of ints (required)
+
1 if false branch is leaf for each node and 0 if an interior node. To represent a tree that is a leaf (only has one node), one can do so by having a single `nodes_*` entry with true and false branches referencing the same `leaf_*` entry
+
nodes_falsenodeids : list of ints (required)
+
If `nodes_falseleafs` is false at an entry, this represents the position of the false branch node. This position can be used to index into a `nodes_*` entry. If `nodes_falseleafs` is false, it is an index into the leaf_* attributes.
+
nodes_featureids : list of ints (required)
+
Feature id for each node.
+
nodes_hitrates : tensor
+
Popularity of each node, used for performance and may be omitted.
+
nodes_missing_value_tracks_true : list of ints
+
For each node, define whether to follow the true branch (if attribute value is 1) or false branch (if attribute value is 0) in the presence of a NaN input feature. This attribute may be left undefined and the default value is false (0) for all nodes.
+
nodes_modes : tensor (required)
+
The comparison operation performed by the node. This is encoded as an enumeration of 0 ('BRANCH_LEQ'), 1 ('BRANCH_LT'), 2 ('BRANCH_GTE'), 3 ('BRANCH_GT'), 4 ('BRANCH_EQ'), 5 ('BRANCH_NEQ'), and 6 ('BRANCH_MEMBER'). Note this is a tensor of type uint8.
+
nodes_splits : tensor (required)
+
Thresholds to do the splitting on for each node with mode that is not 'BRANCH_MEMBER'.
+
nodes_trueleafs : list of ints (required)
+
1 if true branch is leaf for each node and 0 an interior node. To represent a tree that is a leaf (only has one node), one can do so by having a single `nodes_*` entry with true and false branches referencing the same `leaf_*` entry
+
nodes_truenodeids : list of ints (required)
+
If `nodes_trueleafs` is false at an entry, this represents the position of the true branch node. This position can be used to index into a `nodes_*` entry. If `nodes_trueleafs` is false, it is an index into the leaf_* attributes.
+
post_transform : int (default is 0)
+
Indicates the transform to apply to the score.
One of 'NONE' (0), 'SOFTMAX' (1), 'LOGISTIC' (2), 'SOFTMAX_ZERO' (3) or 'PROBIT' (4), defaults to 'NONE' (0)
+
tree_roots : list of ints (required)
+
Index into `nodes_*` for the root of each tree. The tree structure is derived from the branching of each node.
+
+ +#### Inputs + +
+
X : T
+
Input of shape [Batch Size, Number of Features]
+
+ +#### Outputs + +
+
Y : T
+
Output of shape [Batch Size, Number of targets]
+
+ +#### Type Constraints + +
+
T : tensor(float), tensor(double), tensor(float16)
+
The input type must be a tensor of a numeric type.
+
+ +### **ai.onnx.ml.TreeEnsembleClassifier-5** (deprecated) + + This operator is DEPRECATED. Please use TreeEnsemble with provides similar functionality. + In order to determine the top class, the ArgMax node can be applied to the output of TreeEnsemble. + To encode class labels, use a LabelEncoder operator. + Tree Ensemble classifier. Returns the top class for each of N inputs.
+ The attributes named 'nodes_X' form a sequence of tuples, associated by + index into the sequences, which must all be of equal length. These tuples + define the nodes.
+ Similarly, all fields prefixed with 'class_' are tuples of votes at the leaves. + A leaf may have multiple votes, where each vote is weighted by + the associated class_weights index.
+ One and only one of classlabels_strings or classlabels_int64s + will be defined. The class_ids are indices into this list. + All fields ending with _as_tensor can be used instead of the + same parameter without the suffix if the element type is double and not float. + +#### Version + +This version of the operator has been deprecated since version 5 of the 'ai.onnx.ml' operator set. + +### **ai.onnx.ml.TreeEnsembleRegressor-5** (deprecated) + + This operator is DEPRECATED. Please use TreeEnsemble instead which provides the same + functionality.
+ Tree Ensemble regressor. Returns the regressed values for each input in N.
+ All args with nodes_ are fields of a tuple of tree nodes, and + it is assumed they are the same length, and an index i will decode the + tuple across these inputs. Each node id can appear only once + for each tree id.
+ All fields prefixed with target_ are tuples of votes at the leaves.
+ A leaf may have multiple votes, where each vote is weighted by + the associated target_weights index.
+ All fields ending with _as_tensor can be used instead of the + same parameter without the suffix if the element type is double and not float. + All trees must have their node ids start at 0 and increment by 1.
+ Mode enum is BRANCH_LEQ, BRANCH_LT, BRANCH_GTE, BRANCH_GT, BRANCH_EQ, BRANCH_NEQ, LEAF + +#### Version + +This version of the operator has been deprecated since version 5 of the 'ai.onnx.ml' operator set. + diff --git a/docs/Operators-ml.md b/docs/Operators-ml.md index c1fa51123ba..c45635fc274 100644 --- a/docs/Operators-ml.md +++ b/docs/Operators-ml.md @@ -26,8 +26,9 @@ For an operator input/output's differentiability, it can be differentiable, |ai.onnx.ml.SVMClassifier|1| |ai.onnx.ml.SVMRegressor|1| |ai.onnx.ml.Scaler|1| -|ai.onnx.ml.TreeEnsembleClassifier|3, 1| -|ai.onnx.ml.TreeEnsembleRegressor|3, 1| +|ai.onnx.ml.TreeEnsemble|5| +|ai.onnx.ml.TreeEnsembleClassifier (deprecated)|5, 3, 1| +|ai.onnx.ml.TreeEnsembleRegressor (deprecated)|5, 3, 1| |ai.onnx.ml.ZipMap|1| @@ -918,102 +919,238 @@ This version of the operator has been available since version 1 of the 'ai.onnx. -### **ai.onnx.ml.TreeEnsembleClassifier** +### **ai.onnx.ml.TreeEnsemble** - Tree Ensemble classifier. Returns the top class for each of N inputs.
- The attributes named 'nodes_X' form a sequence of tuples, associated by - index into the sequences, which must all be of equal length. These tuples - define the nodes.
- Similarly, all fields prefixed with 'class_' are tuples of votes at the leaves. - A leaf may have multiple votes, where each vote is weighted by - the associated class_weights index.
- One and only one of classlabels_strings or classlabels_int64s - will be defined. The class_ids are indices into this list. - All fields ending with _as_tensor can be used instead of the - same parameter without the suffix if the element type is double and not float. + Tree Ensemble operator. Returns the regressed values for each input in a batch. + Inputs have dimensions `[N, F]` where `N` is the input batch size and `F` is the number of input features. + Outputs have dimensions `[N, num_targets]` where `N` is the batch size and `num_targets` is the number of targets, which is a configurable attribute. -#### Version + The encoding of this attribute is split along interior nodes and the leaves of the trees. Notably, attributes with the prefix `nodes_*` are associated with interior nodes, and attributes with the prefix `leaf_*` are associated with leaves. + The attributes `nodes_*` must all have the same length and encode a sequence of tuples, as defined by taking all the `nodes_*` fields at a given position. + + All fields prefixed with `leaf_*` represent tree leaves, and similarly define tuples of leaves and must have identical length. -This version of the operator has been available since version 3 of the 'ai.onnx.ml' operator set. + This operator can be used to implement both the previous `TreeEnsembleRegressor` and `TreeEnsembleClassifier` nodes. + The `TreeEnsembleRegressor` node maps directly to this node and requires changing how the nodes are represented. + The `TreeEnsembleClassifier` node can be implemented by adding a `ArgMax` node after this node to determine the top class. + To encode class labels, a `LabelEncoder` or `GatherND` operator may be used. -Other versions of this operator: 1 +#### Version + +This version of the operator has been available since version 5 of the 'ai.onnx.ml' operator set. #### Attributes
-
base_values : list of floats
-
Base values for classification, added to final class score; the size must be the same as the classes or can be left unassigned (assumed 0)
-
base_values_as_tensor : tensor
-
Base values for classification, added to final class score; the size must be the same as the classes or can be left unassigned (assumed 0)
-
class_ids : list of ints
-
The index of the class list that each weight is for.
-
class_nodeids : list of ints
-
node id that this weight is for.
-
class_treeids : list of ints
-
The id of the tree that this node is in.
-
class_weights : list of floats
-
The weight for the class in class_id.
-
class_weights_as_tensor : tensor
-
The weight for the class in class_id.
-
classlabels_int64s : list of ints
-
Class labels if using integer labels.
One and only one of the 'classlabels_*' attributes must be defined.
-
classlabels_strings : list of strings
-
Class labels if using string labels.
One and only one of the 'classlabels_*' attributes must be defined.
-
nodes_falsenodeids : list of ints
-
Child node if expression is false.
-
nodes_featureids : list of ints
+
aggregate_function : int (default is 1)
+
Defines how to aggregate leaf values within a target.
One of 'AVERAGE' (0) 'SUM' (1) 'MIN' (2) 'MAX (3) defaults to 'SUM' (1)
+
leaf_targetids : list of ints (required)
+
The index of the target that this leaf contributes to (this must be in range `[0, n_targets)`).
+
leaf_weights : tensor (required)
+
The weight for each leaf.
+
membership_values : tensor
+
Members to test membership of for each set membership node. List all of the members to test again in the order that the 'BRANCH_MEMBER' mode appears in `node_modes`, delimited by `NaN`s. Will have the same number of sets of values as nodes with mode 'BRANCH_MEMBER'. This may be omitted if the node doesn't contain any 'BRANCH_MEMBER' nodes.
+
n_targets : int
+
The total number of targets.
+
nodes_falseleafs : list of ints (required)
+
1 if false branch is leaf for each node and 0 if an interior node. To represent a tree that is a leaf (only has one node), one can do so by having a single `nodes_*` entry with true and false branches referencing the same `leaf_*` entry
+
nodes_falsenodeids : list of ints (required)
+
If `nodes_falseleafs` is false at an entry, this represents the position of the false branch node. This position can be used to index into a `nodes_*` entry. If `nodes_falseleafs` is false, it is an index into the leaf_* attributes.
+
nodes_featureids : list of ints (required)
Feature id for each node.
-
nodes_hitrates : list of floats
-
Popularity of each node, used for performance and may be omitted.
-
nodes_hitrates_as_tensor : tensor
+
nodes_hitrates : tensor
Popularity of each node, used for performance and may be omitted.
nodes_missing_value_tracks_true : list of ints
-
For each node, define what to do in the presence of a missing value: if a value is missing (NaN), use the 'true' or 'false' branch based on the value in this array.
This attribute may be left undefined, and the default value is false (0) for all nodes.
-
nodes_modes : list of strings
-
The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf node.
One of 'BRANCH_LEQ', 'BRANCH_LT', 'BRANCH_GTE', 'BRANCH_GT', 'BRANCH_EQ', 'BRANCH_NEQ', 'LEAF'
-
nodes_nodeids : list of ints
-
Node id for each node. Ids may restart at zero for each tree, but it not required to.
-
nodes_treeids : list of ints
-
Tree id for each node.
-
nodes_truenodeids : list of ints
-
Child node if expression is true.
-
nodes_values : list of floats
-
Thresholds to do the splitting on for each node.
-
nodes_values_as_tensor : tensor
-
Thresholds to do the splitting on for each node.
-
post_transform : string (default is NONE)
-
Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' or 'PROBIT.'
+
For each node, define whether to follow the true branch (if attribute value is 1) or false branch (if attribute value is 0) in the presence of a NaN input feature. This attribute may be left undefined and the default value is false (0) for all nodes.
+
nodes_modes : tensor (required)
+
The comparison operation performed by the node. This is encoded as an enumeration of 0 ('BRANCH_LEQ'), 1 ('BRANCH_LT'), 2 ('BRANCH_GTE'), 3 ('BRANCH_GT'), 4 ('BRANCH_EQ'), 5 ('BRANCH_NEQ'), and 6 ('BRANCH_MEMBER'). Note this is a tensor of type uint8.
+
nodes_splits : tensor (required)
+
Thresholds to do the splitting on for each node with mode that is not 'BRANCH_MEMBER'.
+
nodes_trueleafs : list of ints (required)
+
1 if true branch is leaf for each node and 0 an interior node. To represent a tree that is a leaf (only has one node), one can do so by having a single `nodes_*` entry with true and false branches referencing the same `leaf_*` entry
+
nodes_truenodeids : list of ints (required)
+
If `nodes_trueleafs` is false at an entry, this represents the position of the true branch node. This position can be used to index into a `nodes_*` entry. If `nodes_trueleafs` is false, it is an index into the leaf_* attributes.
+
post_transform : int (default is 0)
+
Indicates the transform to apply to the score.
One of 'NONE' (0), 'SOFTMAX' (1), 'LOGISTIC' (2), 'SOFTMAX_ZERO' (3) or 'PROBIT' (4), defaults to 'NONE' (0)
+
tree_roots : list of ints (required)
+
Index into `nodes_*` for the root of each tree. The tree structure is derived from the branching of each node.
#### Inputs
-
X : T1
-
Input of shape [N,F]
+
X : T
+
Input of shape [Batch Size, Number of Features]
#### Outputs
-
Y : T2
-
N, Top class for each point
-
Z : tensor(float)
-
The class score for each class, for each point, a tensor of shape [N,E].
+
Y : T
+
Output of shape [Batch Size, Number of targets]
#### Type Constraints
-
T1 : tensor(float), tensor(double), tensor(int64), tensor(int32)
+
T : tensor(float), tensor(double), tensor(float16)
The input type must be a tensor of a numeric type.
-
T2 : tensor(string), tensor(int64)
-
The output type will be a tensor of strings or integers, depending on which of the classlabels_* attributes is used.
-### **ai.onnx.ml.TreeEnsembleRegressor** +#### Examples + +
+tree_ensemble_set_membership + +```python +node = onnx.helper.make_node( + "TreeEnsemble", + ["X"], + ["Y"], + domain="ai.onnx.ml", + n_targets=4, + aggregate_function=1, + membership_values=make_tensor( + "membership_values", + onnx.TensorProto.FLOAT, + (8,), + [1.2, 3.7, 8, 9, np.nan, 12, 7, np.nan], + ), + nodes_missing_value_tracks_true=None, + nodes_hitrates=None, + post_transform=0, + tree_roots=[0], + nodes_modes=make_tensor( + "nodes_modes", + onnx.TensorProto.UINT8, + (3,), + np.array([0, 6, 6], dtype=np.uint8), + ), + nodes_featureids=[0, 0, 0], + nodes_splits=make_tensor( + "nodes_splits", + onnx.TensorProto.FLOAT, + (3,), + np.array([11, 232344.0, np.nan], dtype=np.float32), + ), + nodes_trueleafs=[0, 1, 1], + nodes_truenodeids=[1, 0, 1], + nodes_falseleafs=[1, 0, 1], + nodes_falsenodeids=[2, 2, 3], + leaf_targetids=[0, 1, 2, 3], + leaf_weights=make_tensor( + "leaf_weights", onnx.TensorProto.FLOAT, (4,), [1, 10, 1000, 100] + ), +) + +x = np.array([1.2, 3.4, -0.12, np.nan, 12, 7], np.float32).reshape(-1, 1) +expected = np.array( + [ + [1, 0, 0, 0], + [0, 0, 0, 100], + [0, 0, 0, 100], + [0, 0, 1000, 0], + [0, 0, 1000, 0], + [0, 10, 0, 0], + ], + dtype=np.float32, +) +expect( + node, + inputs=[x], + outputs=[expected], + name="test_ai_onnx_ml_tree_ensemble_set_membership", +) +``` + +
+ + +
+tree_ensemble_single_tree + +```python +node = onnx.helper.make_node( + "TreeEnsemble", + ["X"], + ["Y"], + domain="ai.onnx.ml", + n_targets=2, + membership_values=None, + nodes_missing_value_tracks_true=None, + nodes_hitrates=None, + aggregate_function=1, + post_transform=0, + tree_roots=[0], + nodes_modes=make_tensor( + "nodes_modes", + onnx.TensorProto.UINT8, + (3,), + np.array([0, 0, 0], dtype=np.uint8), + ), + nodes_featureids=[0, 0, 0], + nodes_splits=make_tensor( + "nodes_splits", + onnx.TensorProto.DOUBLE, + (3,), + np.array([3.14, 1.2, 4.2], dtype=np.float64), + ), + nodes_truenodeids=[1, 0, 1], + nodes_trueleafs=[0, 1, 1], + nodes_falsenodeids=[2, 2, 3], + nodes_falseleafs=[0, 1, 1], + leaf_targetids=[0, 1, 0, 1], + leaf_weights=make_tensor( + "leaf_weights", + onnx.TensorProto.DOUBLE, + (4,), + np.array([5.23, 12.12, -12.23, 7.21], dtype=np.float64), + ), +) - Tree Ensemble regressor. Returns the regressed values for each input in N.
+x = np.array([1.2, 3.4, -0.12, 1.66, 4.14, 1.77], np.float64).reshape(3, 2) +y = np.array([[5.23, 0], [5.23, 0], [0, 12.12]], dtype=np.float64) +expect( + node, + inputs=[x], + outputs=[y], + name="test_ai_onnx_ml_tree_ensemble_single_tree", +) +``` + +
+ + +### **ai.onnx.ml.TreeEnsembleClassifier** (deprecated) + + This operator is DEPRECATED. Please use TreeEnsemble with provides similar functionality. + In order to determine the top class, the ArgMax node can be applied to the output of TreeEnsemble. + To encode class labels, use a LabelEncoder operator. + Tree Ensemble classifier. Returns the top class for each of N inputs.
+ The attributes named 'nodes_X' form a sequence of tuples, associated by + index into the sequences, which must all be of equal length. These tuples + define the nodes.
+ Similarly, all fields prefixed with 'class_' are tuples of votes at the leaves. + A leaf may have multiple votes, where each vote is weighted by + the associated class_weights index.
+ One and only one of classlabels_strings or classlabels_int64s + will be defined. The class_ids are indices into this list. + All fields ending with _as_tensor can be used instead of the + same parameter without the suffix if the element type is double and not float. + +#### Version + +This version of the operator has been deprecated since version 5 of the 'ai.onnx.ml' operator set. + +Other versions of this operator: 1, 3 + + +### **ai.onnx.ml.TreeEnsembleRegressor** (deprecated) + + This operator is DEPRECATED. Please use TreeEnsemble instead which provides the same + functionality.
+ Tree Ensemble regressor. Returns the regressed values for each input in N.
All args with nodes_ are fields of a tuple of tree nodes, and it is assumed they are the same length, and an index i will decode the tuple across these inputs. Each node id can appear only once @@ -1028,77 +1165,9 @@ Other versions of this operator: 1 - -#### Attributes - -
-
aggregate_function : string (default is SUM)
-
Defines how to aggregate leaf values within a target.
One of 'AVERAGE,' 'SUM,' 'MIN,' 'MAX.'
-
base_values : list of floats
-
Base values for regression, added to final prediction after applying aggregate_function; the size must be the same as the classes or can be left unassigned (assumed 0)
-
base_values_as_tensor : tensor
-
Base values for regression, added to final prediction after applying aggregate_function; the size must be the same as the classes or can be left unassigned (assumed 0)
-
n_targets : int
-
The total number of targets.
-
nodes_falsenodeids : list of ints
-
Child node if expression is false
-
nodes_featureids : list of ints
-
Feature id for each node.
-
nodes_hitrates : list of floats
-
Popularity of each node, used for performance and may be omitted.
-
nodes_hitrates_as_tensor : tensor
-
Popularity of each node, used for performance and may be omitted.
-
nodes_missing_value_tracks_true : list of ints
-
For each node, define what to do in the presence of a NaN: use the 'true' (if the attribute value is 1) or 'false' (if the attribute value is 0) branch based on the value in this array.
This attribute may be left undefined and the default value is false (0) for all nodes.
-
nodes_modes : list of strings
-
The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf node.
One of 'BRANCH_LEQ', 'BRANCH_LT', 'BRANCH_GTE', 'BRANCH_GT', 'BRANCH_EQ', 'BRANCH_NEQ', 'LEAF'
-
nodes_nodeids : list of ints
-
Node id for each node. Node ids must restart at zero for each tree and increase sequentially.
-
nodes_treeids : list of ints
-
Tree id for each node.
-
nodes_truenodeids : list of ints
-
Child node if expression is true
-
nodes_values : list of floats
-
Thresholds to do the splitting on for each node.
-
nodes_values_as_tensor : tensor
-
Thresholds to do the splitting on for each node.
-
post_transform : string (default is NONE)
-
Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' or 'PROBIT'
-
target_ids : list of ints
-
The index of the target that each weight is for
-
target_nodeids : list of ints
-
The node id of each weight
-
target_treeids : list of ints
-
The id of the tree that each node is in.
-
target_weights : list of floats
-
The weight for each target
-
target_weights_as_tensor : tensor
-
The weight for each target
-
- -#### Inputs - -
-
X : T
-
Input of shape [N,F]
-
- -#### Outputs +This version of the operator has been deprecated since version 5 of the 'ai.onnx.ml' operator set. -
-
Y : tensor(float)
-
N classes
-
- -#### Type Constraints - -
-
T : tensor(float), tensor(double), tensor(int64), tensor(int32)
-
The input type must be a tensor of a numeric type.
-
+Other versions of this operator: 1, 3 ### **ai.onnx.ml.ZipMap** diff --git a/docs/TestCoverage-ml.md b/docs/TestCoverage-ml.md index 47196a8f725..89b6771d73f 100644 --- a/docs/TestCoverage-ml.md +++ b/docs/TestCoverage-ml.md @@ -6,7 +6,7 @@ * [Overall Test Coverage](#overall-test-coverage) # Node Test Coverage ## Summary -Node tests have covered 3/18 (16.67%, 0 generators excluded) common operators. +Node tests have covered 4/19 (21.05%, 0 generators excluded) common operators. Node tests have covered 0/0 (N/A) experimental operators. @@ -166,6 +166,128 @@ expect( +### TreeEnsemble +There are 2 test cases, listed as following: +
+tree_ensemble_set_membership + +```python +node = onnx.helper.make_node( + "TreeEnsemble", + ["X"], + ["Y"], + domain="ai.onnx.ml", + n_targets=4, + aggregate_function=1, + membership_values=make_tensor( + "membership_values", + onnx.TensorProto.FLOAT, + (8,), + [1.2, 3.7, 8, 9, np.nan, 12, 7, np.nan], + ), + nodes_missing_value_tracks_true=None, + nodes_hitrates=None, + post_transform=0, + tree_roots=[0], + nodes_modes=make_tensor( + "nodes_modes", + onnx.TensorProto.UINT8, + (3,), + np.array([0, 6, 6], dtype=np.uint8), + ), + nodes_featureids=[0, 0, 0], + nodes_splits=make_tensor( + "nodes_splits", + onnx.TensorProto.FLOAT, + (3,), + np.array([11, 232344.0, np.nan], dtype=np.float32), + ), + nodes_trueleafs=[0, 1, 1], + nodes_truenodeids=[1, 0, 1], + nodes_falseleafs=[1, 0, 1], + nodes_falsenodeids=[2, 2, 3], + leaf_targetids=[0, 1, 2, 3], + leaf_weights=make_tensor( + "leaf_weights", onnx.TensorProto.FLOAT, (4,), [1, 10, 1000, 100] + ), +) + +x = np.array([1.2, 3.4, -0.12, np.nan, 12, 7], np.float32).reshape(-1, 1) +expected = np.array( + [ + [1, 0, 0, 0], + [0, 0, 0, 100], + [0, 0, 0, 100], + [0, 0, 1000, 0], + [0, 0, 1000, 0], + [0, 10, 0, 0], + ], + dtype=np.float32, +) +expect( + node, + inputs=[x], + outputs=[expected], + name="test_ai_onnx_ml_tree_ensemble_set_membership", +) +``` + +
+
+tree_ensemble_single_tree + +```python +node = onnx.helper.make_node( + "TreeEnsemble", + ["X"], + ["Y"], + domain="ai.onnx.ml", + n_targets=2, + membership_values=None, + nodes_missing_value_tracks_true=None, + nodes_hitrates=None, + aggregate_function=1, + post_transform=0, + tree_roots=[0], + nodes_modes=make_tensor( + "nodes_modes", + onnx.TensorProto.UINT8, + (3,), + np.array([0, 0, 0], dtype=np.uint8), + ), + nodes_featureids=[0, 0, 0], + nodes_splits=make_tensor( + "nodes_splits", + onnx.TensorProto.DOUBLE, + (3,), + np.array([3.14, 1.2, 4.2], dtype=np.float64), + ), + nodes_truenodeids=[1, 0, 1], + nodes_trueleafs=[0, 1, 1], + nodes_falsenodeids=[2, 2, 3], + nodes_falseleafs=[0, 1, 1], + leaf_targetids=[0, 1, 0, 1], + leaf_weights=make_tensor( + "leaf_weights", + onnx.TensorProto.DOUBLE, + (4,), + np.array([5.23, 12.12, -12.23, 7.21], dtype=np.float64), + ), +) + +x = np.array([1.2, 3.4, -0.12, 1.66, 4.14, 1.77], np.float64).reshape(3, 2) +y = np.array([[5.23, 0], [5.23, 0], [0, 12.12]], dtype=np.float64) +expect( + node, + inputs=[x], + outputs=[y], + name="test_ai_onnx_ml_tree_ensemble_single_tree", +) +``` + +
+ +
## 💔No Cover Common Operators diff --git a/onnx/backend/test/case/node/ai_onnx_ml/tree_ensemble.py b/onnx/backend/test/case/node/ai_onnx_ml/tree_ensemble.py new file mode 100644 index 00000000000..a799456e37a --- /dev/null +++ b/onnx/backend/test/case/node/ai_onnx_ml/tree_ensemble.py @@ -0,0 +1,122 @@ +# Copyright (c) ONNX Project Contributors + +# SPDX-License-Identifier: Apache-2.0 + +import numpy as np + +import onnx +from onnx.backend.test.case.base import Base +from onnx.backend.test.case.node import expect +from onnx.helper import make_tensor + + +class TreeEnsemble(Base): + @staticmethod + def export_tree_ensemble_single_tree() -> None: + node = onnx.helper.make_node( + "TreeEnsemble", + ["X"], + ["Y"], + domain="ai.onnx.ml", + n_targets=2, + membership_values=None, + nodes_missing_value_tracks_true=None, + nodes_hitrates=None, + aggregate_function=1, + post_transform=0, + tree_roots=[0], + nodes_modes=make_tensor( + "nodes_modes", + onnx.TensorProto.UINT8, + (3,), + np.array([0, 0, 0], dtype=np.uint8), + ), + nodes_featureids=[0, 0, 0], + nodes_splits=make_tensor( + "nodes_splits", + onnx.TensorProto.DOUBLE, + (3,), + np.array([3.14, 1.2, 4.2], dtype=np.float64), + ), + nodes_truenodeids=[1, 0, 1], + nodes_trueleafs=[0, 1, 1], + nodes_falsenodeids=[2, 2, 3], + nodes_falseleafs=[0, 1, 1], + leaf_targetids=[0, 1, 0, 1], + leaf_weights=make_tensor( + "leaf_weights", + onnx.TensorProto.DOUBLE, + (4,), + np.array([5.23, 12.12, -12.23, 7.21], dtype=np.float64), + ), + ) + + x = np.array([1.2, 3.4, -0.12, 1.66, 4.14, 1.77], np.float64).reshape(3, 2) + y = np.array([[5.23, 0], [5.23, 0], [0, 12.12]], dtype=np.float64) + expect( + node, + inputs=[x], + outputs=[y], + name="test_ai_onnx_ml_tree_ensemble_single_tree", + ) + + @staticmethod + def export_tree_ensemble_set_membership() -> None: + node = onnx.helper.make_node( + "TreeEnsemble", + ["X"], + ["Y"], + domain="ai.onnx.ml", + n_targets=4, + aggregate_function=1, + membership_values=make_tensor( + "membership_values", + onnx.TensorProto.FLOAT, + (8,), + [1.2, 3.7, 8, 9, np.nan, 12, 7, np.nan], + ), + nodes_missing_value_tracks_true=None, + nodes_hitrates=None, + post_transform=0, + tree_roots=[0], + nodes_modes=make_tensor( + "nodes_modes", + onnx.TensorProto.UINT8, + (3,), + np.array([0, 6, 6], dtype=np.uint8), + ), + nodes_featureids=[0, 0, 0], + nodes_splits=make_tensor( + "nodes_splits", + onnx.TensorProto.FLOAT, + (3,), + np.array([11, 232344.0, np.nan], dtype=np.float32), + ), + nodes_trueleafs=[0, 1, 1], + nodes_truenodeids=[1, 0, 1], + nodes_falseleafs=[1, 0, 1], + nodes_falsenodeids=[2, 2, 3], + leaf_targetids=[0, 1, 2, 3], + leaf_weights=make_tensor( + "leaf_weights", onnx.TensorProto.FLOAT, (4,), [1, 10, 1000, 100] + ), + ) + + x = np.array([1.2, 3.4, -0.12, np.nan, 12, 7], np.float32).reshape(-1, 1) + expected = np.array( + [ + [1, 0, 0, 0], + [0, 0, 0, 100], + [0, 0, 0, 100], + [0, 0, 1000, 0], + [0, 0, 1000, 0], + [0, 10, 0, 0], + ], + dtype=np.float32, + ) + expect( + node, + inputs=[x], + outputs=[expected], + name="test_ai_onnx_ml_tree_ensemble_set_membership", + ) diff --git a/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_set_membership/model.onnx b/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_set_membership/model.onnx new file mode 100644 index 00000000000..c51dfadd588 Binary files /dev/null and b/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_set_membership/model.onnx differ diff --git a/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_set_membership/test_data_set_0/input_0.pb b/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_set_membership/test_data_set_0/input_0.pb new file mode 100644 index 00000000000..377ef7600a7 Binary files /dev/null and b/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_set_membership/test_data_set_0/input_0.pb differ diff --git a/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_set_membership/test_data_set_0/output_0.pb b/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_set_membership/test_data_set_0/output_0.pb new file mode 100644 index 00000000000..431fafb9341 Binary files /dev/null and b/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_set_membership/test_data_set_0/output_0.pb differ diff --git a/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_single_tree/model.onnx b/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_single_tree/model.onnx new file mode 100644 index 00000000000..d099958cb68 Binary files /dev/null and b/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_single_tree/model.onnx differ diff --git a/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_single_tree/test_data_set_0/input_0.pb b/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_single_tree/test_data_set_0/input_0.pb new file mode 100644 index 00000000000..a727fce1b88 --- /dev/null +++ b/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_single_tree/test_data_set_0/input_0.pb @@ -0,0 +1 @@ + BXJ0333333?333333 @Q(\?(\@RQ? \ No newline at end of file diff --git a/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_single_tree/test_data_set_0/output_0.pb b/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_single_tree/test_data_set_0/output_0.pb new file mode 100644 index 00000000000..815f1f41480 Binary files /dev/null and b/onnx/backend/test/data/node/test_ai_onnx_ml_tree_ensemble_single_tree/test_data_set_0/output_0.pb differ diff --git a/onnx/defs/__init__.py b/onnx/defs/__init__.py index 137c104d218..55fb5cf4133 100644 --- a/onnx/defs/__init__.py +++ b/onnx/defs/__init__.py @@ -38,6 +38,11 @@ def onnx_opset_version() -> int: return C.schema_version_map()[ONNX_DOMAIN][1] +def onnx_ml_opset_version() -> int: + """Return current opset for domain `ai.onnx.ml`.""" + return C.schema_version_map()[ONNX_ML_DOMAIN][1] + + @property # type: ignore def _function_proto(self): # type: ignore func_proto = FunctionProto() diff --git a/onnx/defs/operator_sets_ml.h b/onnx/defs/operator_sets_ml.h index 9c22334d1e6..d1b52999a90 100644 --- a/onnx/defs/operator_sets_ml.h +++ b/onnx/defs/operator_sets_ml.h @@ -84,11 +84,25 @@ class OpSet_OnnxML_ver4 { } }; +class ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(OnnxML, 5, TreeEnsemble); +class ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(OnnxML, 5, TreeEnsembleRegressor); +class ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(OnnxML, 5, TreeEnsembleClassifier); + +class OpSet_OnnxML_ver5 { + public: + static void ForEachSchema(std::function fn) { + fn(GetOpSchema()); + fn(GetOpSchema()); + fn(GetOpSchema()); + } +}; + inline void RegisterOnnxMLOperatorSetSchema() { RegisterOpSetSchema(); RegisterOpSetSchema(); RegisterOpSetSchema(); RegisterOpSetSchema(); + RegisterOpSetSchema(); } } // namespace ONNX_NAMESPACE diff --git a/onnx/defs/schema.h b/onnx/defs/schema.h index 05bee7e5de9..c7b2029109d 100644 --- a/onnx/defs/schema.h +++ b/onnx/defs/schema.h @@ -1154,7 +1154,7 @@ class OpSchemaRegistry final : public ISchemaRegistry { // operator schema on specific domain. Update the lowest version when it's // determined to remove too old version history. map_[ONNX_DOMAIN] = std::make_pair(1, 21); - map_[AI_ONNX_ML_DOMAIN] = std::make_pair(1, 4); + map_[AI_ONNX_ML_DOMAIN] = std::make_pair(1, 5); map_[AI_ONNX_TRAINING_DOMAIN] = std::make_pair(1, 1); // ONNX's preview domain contains operators subject to change, so // versining is not meaningful and that domain should have only one diff --git a/onnx/defs/shape_inference.cc b/onnx/defs/shape_inference.cc index 8aa2249c818..c90a5df7a6b 100644 --- a/onnx/defs/shape_inference.cc +++ b/onnx/defs/shape_inference.cc @@ -511,6 +511,23 @@ std::string stringify(const Container& elements) { return ss.str(); } +std::pair getAttributeProtoElemTypeAndLength(const AttributeProto* attr_proto) { + if (attr_proto->ints_size()) { + return {TensorProto_DataType_INT64, attr_proto->ints_size()}; + } else if (attr_proto->floats_size()) { + return {TensorProto_DataType_FLOAT, attr_proto->floats_size()}; + } else if (attr_proto->strings_size()) { + return {TensorProto_DataType_STRING, attr_proto->strings_size()}; + } else if (attr_proto->has_t()) { + if (attr_proto->t().dims_size() != 1) { + fail_type_inference( + "Attribute ", attr_proto->name(), " expected to be a 1D tensor but was ", attr_proto->t().dims_size(), "D"); + } + return {attr_proto->t().data_type(), attr_proto->t().dims(0)}; + } + return {TensorProto::UNDEFINED, 0}; +} + std::pair getAttributeElementTypeAndLength( const InferenceContext& ctx, const std::initializer_list& attribute_names) { @@ -524,23 +541,7 @@ std::pair getAttributeElementTypeAndLength( // Another attribute was already set fail_shape_inference("One and only one attribute must be set out of ", stringify(attribute_names)); } - if (attr_proto->ints_size()) { - elem_type = TensorProto_DataType_INT64; - length = attr_proto->ints_size(); - } else if (attr_proto->floats_size()) { - elem_type = TensorProto_DataType_FLOAT; - length = attr_proto->floats_size(); - } else if (attr_proto->strings_size()) { - elem_type = TensorProto_DataType_STRING; - length = attr_proto->strings_size(); - } else if (attr_proto->has_t()) { - if (attr_proto->t().dims_size() != 1) { - fail_type_inference( - "Attribute ", attribute, " expected to be a 1D tensor but was ", attr_proto->t().dims_size(), "D"); - } - elem_type = attr_proto->t().data_type(); - length = attr_proto->t().dims(0); - } + std::tie(elem_type, length) = getAttributeProtoElemTypeAndLength(attr_proto); } } return {elem_type, length}; diff --git a/onnx/defs/shape_inference.h b/onnx/defs/shape_inference.h index f949f59c35b..a80473b386c 100644 --- a/onnx/defs/shape_inference.h +++ b/onnx/defs/shape_inference.h @@ -187,6 +187,8 @@ inline TensorShapeProto::Dimension operator*(TensorShapeProto::Dimension dim1, T template std::string stringify(const Container& elements); +std::pair getAttributeProtoElemTypeAndLength(const AttributeProto* attr_proto); + std::pair getAttributeElementTypeAndLength( const InferenceContext& ctx, const std::initializer_list& attribute_names); diff --git a/onnx/defs/traditionalml/defs.cc b/onnx/defs/traditionalml/defs.cc index 04af5e32d93..146b290db83 100644 --- a/onnx/defs/traditionalml/defs.cc +++ b/onnx/defs/traditionalml/defs.cc @@ -3,6 +3,7 @@ */ #include "onnx/defs/schema.h" +#include "onnx/defs/traditionalml/utils.h" #ifdef ONNX_ML namespace ONNX_NAMESPACE { @@ -48,7 +49,8 @@ ONNX_ML_OPERATOR_SET_SCHEMA( num_indices *= indices_shape.dim(i).dim_value(); } else if (indices_shape.dim(i).has_dim_param()) { if (single_symbolic_dim.empty()) { - // it is possible to set symbolic dimension param if the rest dim values are all value 1 + // it is possible to set symbolic dimension param if the rest dim values are all + // value 1 single_symbolic_dim = indices_shape.dim(i).dim_param(); } else { return; @@ -111,12 +113,14 @@ ONNX_ML_OPERATOR_SET_SCHEMA( "The output is a 1-D tensor of string, float, or integer.") .Attr( "cast_to", - "A string indicating the desired element type of the output tensor, one of 'TO_FLOAT', 'TO_STRING', 'TO_INT64'.", + "A string indicating the desired element type of the output tensor, one of 'TO_FLOAT', 'TO_STRING', " + "'TO_INT64'.", AttributeProto::STRING, std::string("TO_FLOAT")) .Attr( "map_form", - "Indicates whether to only output as many values as are in the input (dense), or position the input based on using the key of the map as the index of the output (sparse).
One of 'DENSE', 'SPARSE'.", + "Indicates whether to only output as many values as are in the input (dense), or position the input based " + "on using the key of the map as the index of the output (sparse).
One of 'DENSE', 'SPARSE'.", AttributeProto::STRING, std::string("DENSE")) .Attr( @@ -179,12 +183,14 @@ ONNX_ML_OPERATOR_SET_SCHEMA( OPTIONAL_VALUE) .Attr( "default_string", - "A string to use when an input integer value is not found in the map.
One and only one of the 'default_*' attributes must be defined.", + "A string to use when an input integer value is not found in the map.
One and only one of the " + "'default_*' attributes must be defined.", AttributeProto::STRING, std::string("_Unused")) .Attr( "default_int64", - "An integer to use when an input string value is not found in the map.
One and only one of the 'default_*' attributes must be defined.", + "An integer to use when an input string value is not found in the map.
One and only one of the " + "'default_*' attributes must be defined.", AttributeProto::INT, static_cast(-1)) .TypeAndShapeInferenceFunction([](InferenceContext& ctx) { @@ -230,11 +236,13 @@ ONNX_ML_OPERATOR_SET_SCHEMA( "map(int64, double)", "map(string, float)", "map(string, double)"}, - "The input must be a map from strings or integers to either strings or a numeric type. The key and value types cannot be the same.") + "The input must be a map from strings or integers to either strings or a numeric type. The key and value " + "types cannot be the same.") .TypeConstraint( "T2", {"tensor(int64)", "tensor(float)", "tensor(double)", "tensor(string)"}, - "The output will be a tensor of the value type of the input map. It's shape will be [1,C], where C is the length of the input dictionary.") + "The output will be a tensor of the value type of the input map. It's shape will be [1,C], where C is the " + "length of the input dictionary.") .Attr( "string_vocabulary", "A string vocabulary array.
One and only one of the vocabularies must be defined.", @@ -292,7 +300,8 @@ ONNX_ML_OPERATOR_SET_SCHEMA( .TypeConstraint( "T", {"tensor(float)", "tensor(double)", "tensor(int64)", "tensor(int32)"}, - "The input type must be a tensor of a numeric type, either [N,C] or [C]. The output type will be of the same tensor type and shape.") + "The input type must be a tensor of a numeric type, either [N,C] or [C]. The output type will be of the " + "same tensor type and shape.") .Attr("imputed_value_floats", "Value(s) to change to", AttributeProto::FLOATS, OPTIONAL_VALUE) .Attr("replaced_value_float", "A value that needs replacing.", AttributeProto::FLOAT, 0.f) .Attr("imputed_value_int64s", "Value(s) to change to.", AttributeProto::INTS, OPTIONAL_VALUE) @@ -356,7 +365,8 @@ ONNX_ML_OPERATOR_SET_SCHEMA( .Attr("default_float", "A float.", AttributeProto::FLOAT, -0.f) .Attr( "default_tensor", - "A default tensor. {\"_Unused\"} if values_* has string type, {-1} if values_* has integral type, and {-0.f} if values_* has float type.", + "A default tensor. {\"_Unused\"} if values_* has string type, {-1} if values_* has integral type, and " + "{-0.f} if values_* has float type.", AttributeProto::TENSOR, OPTIONAL_VALUE) .TypeAndShapeInferenceFunction([](InferenceContext& ctx) { @@ -427,7 +437,8 @@ ONNX_ML_OPERATOR_SET_SCHEMA( .TypeConstraint( "T1", {"tensor(float)", "tensor(double)", "tensor(int64)", "tensor(int32)"}, - "The input must be a tensor of a numeric type, and of shape [N,C] or [C]. In the latter case, it will be treated as [1,C]") + "The input must be a tensor of a numeric type, and of shape [N,C] or [C]. In the latter case, it will be " + "treated as [1,C]") .TypeConstraint( "T2", {"tensor(string)", "tensor(int64)"}, @@ -451,7 +462,8 @@ ONNX_ML_OPERATOR_SET_SCHEMA( OPTIONAL_VALUE) .Attr( "post_transform", - "Indicates the transform to apply to the scores vector.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' or 'PROBIT'", + "Indicates the transform to apply to the scores vector.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' " + "'SOFTMAX_ZERO,' or 'PROBIT'", AttributeProto::STRING, std::string("NONE")) .TypeAndShapeInferenceFunction([](InferenceContext& ctx) { @@ -529,7 +541,8 @@ ONNX_ML_OPERATOR_SET_SCHEMA( "The input must be a tensor of a numeric type.") .Attr( "post_transform", - "Indicates the transform to apply to the regression output vector.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' or 'PROBIT'", + "Indicates the transform to apply to the regression output vector.
One of 'NONE,' 'SOFTMAX,' " + "'LOGISTIC,' 'SOFTMAX_ZERO,' or 'PROBIT'", AttributeProto::STRING, std::string("NONE")) .Attr("coefficients", "Weights of the model(s).", AttributeProto::FLOATS, OPTIONAL_VALUE) @@ -600,7 +613,8 @@ ONNX_ML_OPERATOR_SET_SCHEMA( OPTIONAL_VALUE) .Attr( "zeros", - "If true and category is not present, will return all zeros; if false and a category if not found, the operator will fail.", + "If true and category is not present, will return all zeros; if false and a category if not found, the " + "operator will fail.", AttributeProto::INT, static_cast(1)) .TypeAndShapeInferenceFunction([](InferenceContext& ctx) { @@ -637,12 +651,14 @@ ONNX_ML_OPERATOR_SET_SCHEMA( "The input must be a tensor of a numeric type.") .Attr( "offset", - "First, offset by this.
Can be length of features in an [N,F] tensor or length 1, in which case it applies to all features, regardless of dimension count.", + "First, offset by this.
Can be length of features in an [N,F] tensor or length 1, in which case it " + "applies to all features, regardless of dimension count.", AttributeProto::FLOATS, OPTIONAL_VALUE) .Attr( "scale", - "Second, multiply by this.
Can be length of features in an [N,F] tensor or length 1, in which case it applies to all features, regardless of dimension count.
Must be same length as 'offset'", + "Second, multiply by this.
Can be length of features in an [N,F] tensor or length 1, in which case it " + "applies to all features, regardless of dimension count.
Must be same length as 'offset'", AttributeProto::FLOATS, OPTIONAL_VALUE)); @@ -660,7 +676,8 @@ ONNX_ML_OPERATOR_SET_SCHEMA( .Output( 1, "Z", - "Class scores (one per class per example), if prob_a and prob_b are provided they are probabilities for each class, otherwise they are raw scores.", + "Class scores (one per class per example), if prob_a and prob_b are provided they are probabilities for " + "each class, otherwise they are raw scores.", "tensor(float)") .TypeConstraint( "T1", @@ -669,7 +686,8 @@ ONNX_ML_OPERATOR_SET_SCHEMA( .TypeConstraint( "T2", {"tensor(string)", "tensor(int64)"}, - "The output type will be a tensor of strings or integers, depending on which of the classlabels_* attributes is used. Its size will match the bactch size of the input.") + "The output type will be a tensor of strings or integers, depending on which of the classlabels_* " + "attributes is used. Its size will match the bactch size of the input.") .Attr( "kernel_type", "The kernel type, one of 'LINEAR,' 'POLY,' 'RBF,' 'SIGMOID'.", @@ -686,23 +704,27 @@ ONNX_ML_OPERATOR_SET_SCHEMA( .Attr("prob_a", "First set of probability coefficients.", AttributeProto::FLOATS, OPTIONAL_VALUE) .Attr( "prob_b", - "Second set of probability coefficients. This array must be same size as prob_a.
If these are provided then output Z are probability estimates, otherwise they are raw scores.", + "Second set of probability coefficients. This array must be same size as prob_a.
If these are provided " + "then output Z are probability estimates, otherwise they are raw scores.", AttributeProto::FLOATS, OPTIONAL_VALUE) .Attr("rho", "", AttributeProto::FLOATS, OPTIONAL_VALUE) .Attr( "post_transform", - "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' or 'PROBIT'", + "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' " + "or 'PROBIT'", AttributeProto::STRING, std::string("NONE")) .Attr( "classlabels_strings", - "Class labels if using string labels.
One and only one of the 'classlabels_*' attributes must be defined.", + "Class labels if using string labels.
One and only one of the 'classlabels_*' attributes must be " + "defined.", AttributeProto::STRINGS, OPTIONAL_VALUE) .Attr( "classlabels_ints", - "Class labels if using integer labels.
One and only one of the 'classlabels_*' attributes must be defined.", + "Class labels if using integer labels.
One and only one of the 'classlabels_*' attributes must be " + "defined.", AttributeProto::INTS, OPTIONAL_VALUE) .TypeAndShapeInferenceFunction([](InferenceContext& ctx) { @@ -752,12 +774,16 @@ ONNX_ML_OPERATOR_SET_SCHEMA( .Attr("n_supports", "The number of support vectors.", AttributeProto::INT, static_cast(0)) .Attr( "post_transform", - "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' or 'PROBIT.'", + "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' " + "or 'PROBIT.'", AttributeProto::STRING, std::string("NONE")) .Attr("rho", "", AttributeProto::FLOATS, OPTIONAL_VALUE)); -static const char* TreeEnsembleClassifier_ver3_doc = R"DOC( +static const char* TreeEnsembleClassifier_ver5_doc = R"DOC( + This operator is DEPRECATED. Please use TreeEnsemble with provides similar functionality. + In order to determine the top class, the ArgMax node can be applied to the output of TreeEnsemble. + To encode class labels, use a LabelEncoder operator. Tree Ensemble classifier. Returns the top class for each of N inputs.
The attributes named 'nodes_X' form a sequence of tuples, associated by index into the sequences, which must all be of equal length. These tuples @@ -773,9 +799,10 @@ static const char* TreeEnsembleClassifier_ver3_doc = R"DOC( ONNX_ML_OPERATOR_SET_SCHEMA( TreeEnsembleClassifier, - 3, + 5, OpSchema() - .SetDoc(TreeEnsembleClassifier_ver3_doc) + .Deprecate() + .SetDoc(TreeEnsembleClassifier_ver5_doc) .Input(0, "X", "Input of shape [N,F]", "T1") .Output(0, "Y", "N, Top class for each point", "T2") .Output(1, "Z", "The class score for each class, for each point, a tensor of shape [N,E].", "tensor(float)") @@ -786,7 +813,8 @@ ONNX_ML_OPERATOR_SET_SCHEMA( .TypeConstraint( "T2", {"tensor(string)", "tensor(int64)"}, - "The output type will be a tensor of strings or integers, depending on which of the classlabels_* attributes is used.") + "The output type will be a tensor of strings or integers, depending on which of the classlabels_* " + "attributes is used.") .Attr("nodes_treeids", "Tree id for each node.", AttributeProto::INTS, OPTIONAL_VALUE) .Attr( "nodes_nodeids", @@ -816,14 +844,17 @@ ONNX_ML_OPERATOR_SET_SCHEMA( OPTIONAL_VALUE) .Attr( "nodes_modes", - "The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf node.
One of 'BRANCH_LEQ', 'BRANCH_LT', 'BRANCH_GTE', 'BRANCH_GT', 'BRANCH_EQ', 'BRANCH_NEQ', 'LEAF'", + "The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf " + "node.
One of 'BRANCH_LEQ', 'BRANCH_LT', 'BRANCH_GTE', 'BRANCH_GT', 'BRANCH_EQ', 'BRANCH_NEQ', 'LEAF'", AttributeProto::STRINGS, OPTIONAL_VALUE) .Attr("nodes_truenodeids", "Child node if expression is true.", AttributeProto::INTS, OPTIONAL_VALUE) .Attr("nodes_falsenodeids", "Child node if expression is false.", AttributeProto::INTS, OPTIONAL_VALUE) .Attr( "nodes_missing_value_tracks_true", - "For each node, define what to do in the presence of a missing value: if a value is missing (NaN), use the 'true' or 'false' branch based on the value in this array.
This attribute may be left undefined, and the default value is false (0) for all nodes.", + "For each node, define what to do in the presence of a missing value: if a value is missing (NaN), use the " + "'true' or 'false' branch based on the value in this array.
This attribute may be left undefined, and " + "the default value is false (0) for all nodes.", AttributeProto::INTS, OPTIONAL_VALUE) .Attr("class_treeids", "The id of the tree that this node is in.", AttributeProto::INTS, OPTIONAL_VALUE) @@ -837,85 +868,38 @@ ONNX_ML_OPERATOR_SET_SCHEMA( OPTIONAL_VALUE) .Attr( "classlabels_strings", - "Class labels if using string labels.
One and only one of the 'classlabels_*' attributes must be defined.", + "Class labels if using string labels.
One and only one of the 'classlabels_*' attributes must be " + "defined.", AttributeProto::STRINGS, OPTIONAL_VALUE) .Attr( "classlabels_int64s", - "Class labels if using integer labels.
One and only one of the 'classlabels_*' attributes must be defined.", + "Class labels if using integer labels.
One and only one of the 'classlabels_*' attributes must be " + "defined.", AttributeProto::INTS, OPTIONAL_VALUE) .Attr( "post_transform", - "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' or 'PROBIT.'", + "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' " + "or 'PROBIT.'", AttributeProto::STRING, std::string("NONE")) .Attr( "base_values", - "Base values for classification, added to final class score; the size must be the same as the classes or can be left unassigned (assumed 0)", + "Base values for classification, added to final class score; the size must be the same as the classes or " + "can be left unassigned (assumed 0)", AttributeProto::FLOATS, OPTIONAL_VALUE) .Attr( "base_values_as_tensor", - "Base values for classification, added to final class score; the size must be the same as the classes or can be left unassigned (assumed 0)", + "Base values for classification, added to final class score; the size must be the same as the classes or " + "can be left unassigned (assumed 0)", AttributeProto::TENSOR, - OPTIONAL_VALUE) - .TypeAndShapeInferenceFunction([](InferenceContext& ctx) { - auto* nodes_values = ctx.getAttribute("nodes_values"); - auto* nodes_values_as_tensor = ctx.getAttribute("nodes_values_as_tensor"); - auto* nodes_hitrates = ctx.getAttribute("nodes_hitrates"); - auto* nodes_hitrates_as_tensor = ctx.getAttribute("nodes_hitrates_as_tensor"); - auto* class_weights = ctx.getAttribute("class_weights"); - auto* class_weights_as_tensor = ctx.getAttribute("class_weights_as_tensor"); - auto* base_values = ctx.getAttribute("base_values"); - auto* base_values_as_tensor = ctx.getAttribute("base_values_as_tensor"); - - if (nullptr != nodes_values && nullptr != nodes_values_as_tensor) { - fail_shape_inference( - "Only one of the attributes 'nodes_values', 'nodes_values_as_tensor' should be specified."); - } - if (nullptr != nodes_hitrates && nullptr != nodes_hitrates_as_tensor) { - fail_shape_inference( - "Only one of the attributes 'nodes_hitrates', 'nodes_hitrates_as_tensor' should be specified."); - } - if (nullptr != class_weights && nullptr != class_weights_as_tensor) { - fail_shape_inference( - "Only one of the attributes 'class_weights', 'class_weights_as_tensor' should be specified."); - } - if (nullptr != base_values && nullptr != base_values_as_tensor) { - fail_shape_inference( - "Only one of the attributes 'base_values', 'base_values_as_tensor' should be specified."); - } - - std::vector classlabels_strings; - auto result = getRepeatedAttribute(ctx, "classlabels_strings", classlabels_strings); - bool using_strings = (result && !classlabels_strings.empty()); - if (using_strings) { - updateOutputElemType(ctx, 0, TensorProto::STRING); - } else { - updateOutputElemType(ctx, 0, TensorProto::INT64); - } - updateOutputElemType(ctx, 1, TensorProto::FLOAT); - - checkInputRank(ctx, 0, 2); - Dim N, E; - unifyInputDim(ctx, 0, 0, N); - - if (using_strings) { - unifyDim(E, classlabels_strings.size()); - } else { - std::vector classlabels_int64s; - result = getRepeatedAttribute(ctx, "classlabels_int64s", classlabels_int64s); - if (!result || classlabels_int64s.empty()) { - fail_shape_inference("Non of classlabels_int64s or classlabels_strings is set."); - } - unifyDim(E, classlabels_int64s.size()); - } - updateOutputShape(ctx, 0, {N}); - updateOutputShape(ctx, 1, {N, E}); - })); + OPTIONAL_VALUE)); -static const char* TreeEnsembleRegressor_ver3_doc = R"DOC( +static const char* TreeEnsembleRegressor_ver5_doc = R"DOC( + This operator is DEPRECATED. Please use TreeEnsemble instead which provides the same + functionality.
Tree Ensemble regressor. Returns the regressed values for each input in N.
All args with nodes_ are fields of a tuple of tree nodes, and it is assumed they are the same length, and an index i will decode the @@ -932,9 +916,10 @@ static const char* TreeEnsembleRegressor_ver3_doc = R"DOC( ONNX_ML_OPERATOR_SET_SCHEMA( TreeEnsembleRegressor, - 3, + 5, OpSchema() - .SetDoc(TreeEnsembleRegressor_ver3_doc) + .Deprecate() + .SetDoc(TreeEnsembleRegressor_ver5_doc) .Input(0, "X", "Input of shape [N,F]", "T") .Output(0, "Y", "N classes", "tensor(float)") .TypeConstraint( @@ -970,14 +955,17 @@ ONNX_ML_OPERATOR_SET_SCHEMA( OPTIONAL_VALUE) .Attr( "nodes_modes", - "The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf node.
One of 'BRANCH_LEQ', 'BRANCH_LT', 'BRANCH_GTE', 'BRANCH_GT', 'BRANCH_EQ', 'BRANCH_NEQ', 'LEAF'", + "The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf " + "node.
One of 'BRANCH_LEQ', 'BRANCH_LT', 'BRANCH_GTE', 'BRANCH_GT', 'BRANCH_EQ', 'BRANCH_NEQ', 'LEAF'", AttributeProto::STRINGS, OPTIONAL_VALUE) .Attr("nodes_truenodeids", "Child node if expression is true", AttributeProto::INTS, OPTIONAL_VALUE) .Attr("nodes_falsenodeids", "Child node if expression is false", AttributeProto::INTS, OPTIONAL_VALUE) .Attr( "nodes_missing_value_tracks_true", - "For each node, define what to do in the presence of a NaN: use the 'true' (if the attribute value is 1) or 'false' (if the attribute value is 0) branch based on the value in this array.
This attribute may be left undefined and the default value is false (0) for all nodes.", + "For each node, define what to do in the presence of a NaN: use the 'true' (if the attribute value is 1) " + "or 'false' (if the attribute value is 0) branch based on the value in this array.
This attribute may " + "be left undefined and the default value is false (0) for all nodes.", AttributeProto::INTS, OPTIONAL_VALUE) .Attr("target_treeids", "The id of the tree that each node is in.", AttributeProto::INTS, OPTIONAL_VALUE) @@ -988,7 +976,8 @@ ONNX_ML_OPERATOR_SET_SCHEMA( .Attr("n_targets", "The total number of targets.", AttributeProto::INT, OPTIONAL_VALUE) .Attr( "post_transform", - "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' or 'PROBIT'", + "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' " + "or 'PROBIT'", AttributeProto::STRING, std::string("NONE")) .Attr( @@ -998,48 +987,215 @@ ONNX_ML_OPERATOR_SET_SCHEMA( std::string("SUM")) .Attr( "base_values", - "Base values for regression, added to final prediction after applying aggregate_function; the size must be the same as the classes or can be left unassigned (assumed 0)", + "Base values for regression, added to final prediction after applying aggregate_function; the size must be " + "the same as the classes or can be left unassigned (assumed 0)", AttributeProto::FLOATS, OPTIONAL_VALUE) .Attr( "base_values_as_tensor", - "Base values for regression, added to final prediction after applying aggregate_function; the size must be the same as the classes or can be left unassigned (assumed 0)", + "Base values for regression, added to final prediction after applying aggregate_function; the size must be " + "the same as the classes or can be left unassigned (assumed 0)", + AttributeProto::TENSOR, + OPTIONAL_VALUE)); + +static const char* TreeEnsemble_ver5_doc = R"DOC( + Tree Ensemble operator. Returns the regressed values for each input in a batch. + Inputs have dimensions `[N, F]` where `N` is the input batch size and `F` is the number of input features. + Outputs have dimensions `[N, num_targets]` where `N` is the batch size and `num_targets` is the number of targets, which is a configurable attribute. + + The encoding of this attribute is split along interior nodes and the leaves of the trees. Notably, attributes with the prefix `nodes_*` are associated with interior nodes, and attributes with the prefix `leaf_*` are associated with leaves. + The attributes `nodes_*` must all have the same length and encode a sequence of tuples, as defined by taking all the `nodes_*` fields at a given position. + + All fields prefixed with `leaf_*` represent tree leaves, and similarly define tuples of leaves and must have identical length. + + This operator can be used to implement both the previous `TreeEnsembleRegressor` and `TreeEnsembleClassifier` nodes. + The `TreeEnsembleRegressor` node maps directly to this node and requires changing how the nodes are represented. + The `TreeEnsembleClassifier` node can be implemented by adding a `ArgMax` node after this node to determine the top class. + To encode class labels, a `LabelEncoder` or `GatherND` operator may be used. +)DOC"; + +ONNX_ML_OPERATOR_SET_SCHEMA( + TreeEnsemble, + 5, + OpSchema() + .SetDoc(TreeEnsemble_ver5_doc) + .Input(0, "X", "Input of shape [Batch Size, Number of Features]", "T") + .Output(0, "Y", "Output of shape [Batch Size, Number of targets]", "T") + .TypeConstraint( + "T", + {"tensor(float)", "tensor(double)", "tensor(float16)"}, + "The input type must be a tensor of a numeric type.") + .Attr("nodes_featureids", "Feature id for each node.", AttributeProto::INTS, true) + .Attr( + "nodes_splits", + "Thresholds to do the splitting on for each node with mode that is not 'BRANCH_MEMBER'.", + AttributeProto::TENSOR, + true) + .Attr( + "nodes_hitrates", + "Popularity of each node, used for performance and may be omitted.", AttributeProto::TENSOR, OPTIONAL_VALUE) + .Attr( + "nodes_modes", + "The comparison operation performed by the node. This is encoded as an enumeration of 0 ('BRANCH_LEQ'), 1 " + "('BRANCH_LT'), 2 ('BRANCH_GTE'), 3 ('BRANCH_GT'), 4 ('BRANCH_EQ'), 5 ('BRANCH_NEQ'), and 6 " + "('BRANCH_MEMBER'). Note this is a tensor of type uint8.", + AttributeProto::TENSOR, + true) + .Attr( + "nodes_truenodeids", + "If `nodes_trueleafs` is false at an entry, this represents the position of the true branch node. This " + "position can be used to index into a `nodes_*` entry. If `nodes_trueleafs` is false, it is an index into " + "the leaf_* attributes.", + AttributeProto::INTS, + true) + .Attr( + "nodes_falsenodeids", + "If `nodes_falseleafs` is false at an entry, this represents the position of the false branch node. This " + "position can be used to index into a `nodes_*` entry. If `nodes_falseleafs` is false, it is an index into " + "the leaf_* attributes.", + AttributeProto::INTS, + true) + .Attr( + "nodes_trueleafs", + "1 if true branch is leaf for each node and 0 an interior node. To represent a tree that is a leaf (only " + "has one node), one can do so by having a single `nodes_*` entry with true and false branches referencing " + "the same `leaf_*` entry", + AttributeProto::INTS, + true) + .Attr( + "nodes_falseleafs", + "1 if false branch is leaf for each node and 0 if an interior node. To represent a tree that is a leaf " + "(only has one node), one can do so by having a single `nodes_*` entry with true and false branches " + "referencing the same `leaf_*` entry", + AttributeProto::INTS, + true) + .Attr( + "nodes_missing_value_tracks_true", + "For each node, define whether to follow the true branch (if attribute value is 1) or false branch (if " + "attribute value is 0) in the presence of a NaN input feature. This attribute may be left undefined and " + "the default value is false (0) for all nodes.", + AttributeProto::INTS, + OPTIONAL_VALUE) + .Attr( + "tree_roots", + "Index into `nodes_*` for the root of each tree. The tree structure is derived from the branching of each " + "node.", + AttributeProto::INTS, + true) + .Attr( + "membership_values", + "Members to test membership of for each set membership node. List all of the members to test again in the " + "order that the 'BRANCH_MEMBER' mode appears in `node_modes`, delimited by `NaN`s. Will have the same " + "number " + "of sets of values as nodes with mode 'BRANCH_MEMBER'. This may be omitted if the node doesn't contain any " + "'BRANCH_MEMBER' nodes.", + AttributeProto::TENSOR, + OPTIONAL_VALUE) + .Attr( + "leaf_targetids", + "The index of the target that this leaf contributes to (this must be in range `[0, n_targets)`).", + AttributeProto::INTS, + true) + .Attr("leaf_weights", "The weight for each leaf.", AttributeProto::TENSOR, true) + .Attr("n_targets", "The total number of targets.", AttributeProto::INT, OPTIONAL_VALUE) + .Attr( + "post_transform", + "Indicates the transform to apply to the score.
One of 'NONE' (0), 'SOFTMAX' (1), 'LOGISTIC' (2), " + "'SOFTMAX_ZERO' (3) or 'PROBIT' (4), defaults to 'NONE' (0)", + AttributeProto::INT, + static_cast(0)) + .Attr( + "aggregate_function", + "Defines how to aggregate leaf values within a target.
One of 'AVERAGE' (0) 'SUM' (1) 'MIN' (2) 'MAX " + "(3) defaults to 'SUM' (1)", + AttributeProto::INT, + static_cast(1)) .TypeAndShapeInferenceFunction([](InferenceContext& ctx) { - auto* nodes_values = ctx.getAttribute("nodes_values"); - auto* nodes_values_as_tensor = ctx.getAttribute("nodes_values_as_tensor"); - auto* nodes_hitrates = ctx.getAttribute("nodes_hitrates"); - auto* nodes_hitrates_as_tensor = ctx.getAttribute("nodes_hitrates_as_tensor"); - auto* target_weights = ctx.getAttribute("target_weights"); - auto* target_weights_as_tensor = ctx.getAttribute("target_weights_as_tensor"); - auto* base_values = ctx.getAttribute("base_values"); - auto* base_values_as_tensor = ctx.getAttribute("base_values_as_tensor"); - - if (nullptr != nodes_values && nullptr != nodes_values_as_tensor) { - fail_shape_inference( - "Only one of the attributes 'nodes_values', 'nodes_values_as_tensor' should be specified."); + checkInputRank(ctx, 0, 2); + auto* nodes_splits = ctx.getAttribute("nodes_splits"); + if (nullptr == nodes_splits) { + fail_shape_inference("Attribute 'nodes_splits' is required."); } - if (nullptr != nodes_hitrates && nullptr != nodes_hitrates_as_tensor) { + if (nodes_splits->t().dims_size() != 1) { + fail_shape_inference("Attribute 'nodes_splits' must be 1D."); + } + auto input_type = ctx.getInputType(0)->tensor_type().elem_type(); + // Check that input type is same as split type + if (input_type != nodes_splits->t().data_type()) { fail_shape_inference( - "Only one of the attributes 'nodes_hitrates', 'nodes_hitrates_as_tensor' should be specified."); + "Attribute 'nodes_splits' must have same type as input. Input type is ", + input_type, + " and attribute type is ", + nodes_splits->t().data_type()); } - if (nullptr != target_weights && nullptr != target_weights_as_tensor) { + + // Expected nodes_* length + auto expected_length = nodes_splits->t().dims(0); + // Validate all nodes_* attributes that are set have the same length and are 1D. + AssertAttributeProtoTypeAndLength( + ctx.getAttribute("nodes_featureids"), expected_length, TensorProto_DataType_INT64, true); + AssertAttributeProtoTypeAndLength( + ctx.getAttribute("nodes_hitrates"), expected_length, TensorProto_DataType_FLOAT, false); + AssertAttributeProtoTypeAndLength( + ctx.getAttribute("nodes_modes"), expected_length, TensorProto_DataType_UINT8, true); + AssertAttributeProtoTypeAndLength( + ctx.getAttribute("nodes_truenodeids"), expected_length, TensorProto_DataType_INT64, true); + AssertAttributeProtoTypeAndLength( + ctx.getAttribute("nodes_falsenodeids"), expected_length, TensorProto_DataType_INT64, true); + AssertAttributeProtoTypeAndLength( + ctx.getAttribute("nodes_trueleafs"), expected_length, TensorProto_DataType_INT64, true); + AssertAttributeProtoTypeAndLength( + ctx.getAttribute("nodes_falseleafs"), expected_length, TensorProto_DataType_INT64, true); + AssertAttributeProtoTypeAndLength( + ctx.getAttribute("nodes_missing_value_tracks_true"), expected_length, TensorProto_DataType_INT64, false); + + // The set membership values and the splits must have the same type as the input. + auto* membership_values = ctx.getAttribute("membership_values"); + if (nullptr != membership_values && membership_values->t().data_type() != input_type) { fail_shape_inference( - "Only one of the attributes 'target_weights', 'target_weights_as_tensor' should be specified."); + "Attribute 'membership_values' must have same type as input. Input type is ", + input_type, + " and attribute type is ", + membership_values->t().data_type()); } - if (nullptr != base_values && nullptr != base_values_as_tensor) { + AssertAttributeProtoTypeAndLength( + ctx.getAttribute("nodes_splits"), expected_length, static_cast(input_type), true); + + // Validate all leaf_* attributes that are set have the same length and are 1D. + auto* leaf_targetids = ctx.getAttribute("leaf_targetids"); + auto* leaf_weights = ctx.getAttribute("leaf_weights"); + if (nullptr != leaf_targetids && nullptr != leaf_weights) { + if (leaf_targetids->ints_size() != leaf_weights->t().dims(0)) { + fail_shape_inference( + "Attribute 'leaf_targetids' must have same length as attribute 'leaf_weights'. 'leaf_targetids' " + "length is ", + leaf_targetids->ints_size(), + " and 'leaf_weights' length is ", + leaf_weights->t().dims(0)); + } + } else { + fail_shape_inference("Attributes 'leaf_targetids' and 'leaf_weights' must both be set."); + } + + // Validate weights have same type as input. + if (leaf_weights->t().data_type() != input_type) { fail_shape_inference( - "Only one of the attributes 'base_values', 'base_values_as_tensor' should be specified."); + "Attribute 'leaf_weights' must have same type as input. Input type is ", + input_type, + " and attribute type is ", + leaf_weights->t().data_type()); } checkInputRank(ctx, 0, 2); + Dim N, E; unifyInputDim(ctx, 0, 0, N); if (nullptr != ctx.getAttribute("n_targets")) { unifyDim(E, ctx.getAttribute("n_targets")->i()); } - updateOutputElemType(ctx, 0, TensorProto::FLOAT); + updateOutputElemType(ctx, 0, input_type); updateOutputShape(ctx, 0, {N, E}); })); diff --git a/onnx/defs/traditionalml/old.cc b/onnx/defs/traditionalml/old.cc index a465e37f0c2..0bf9819ec4a 100644 --- a/onnx/defs/traditionalml/old.cc +++ b/onnx/defs/traditionalml/old.cc @@ -37,12 +37,14 @@ ONNX_ML_OPERATOR_SET_SCHEMA( .Attr("classes_strings", "A list of labels.", AttributeProto::STRINGS, OPTIONAL_VALUE) .Attr( "default_int64", - "An integer to use when an input string value is not found in the map.
One and only one of the 'default_*' attributes must be defined.", + "An integer to use when an input string value is not found in the map.
One and only one of the " + "'default_*' attributes must be defined.", AttributeProto::INT, static_cast(-1)) .Attr( "default_string", - "A string to use when an input integer value is not found in the map.
One and only one of the 'default_*' attributes must be defined.", + "A string to use when an input integer value is not found in the map.
One and only one of the " + "'default_*' attributes must be defined.", AttributeProto::STRING, std::string("_Unused")) .TypeAndShapeInferenceFunction([](InferenceContext& ctx) { @@ -82,7 +84,8 @@ ONNX_ML_OPERATOR_SET_SCHEMA( .TypeConstraint( "T2", {"tensor(string)", "tensor(int64)"}, - "The output type will be a tensor of strings or integers, depending on which of the classlabels_* attributes is used.") + "The output type will be a tensor of strings or integers, depending on which of the classlabels_* " + "attributes is used.") .Attr("nodes_treeids", "Tree id for each node.", AttributeProto::INTS, OPTIONAL_VALUE) .Attr( "nodes_nodeids", @@ -102,14 +105,17 @@ ONNX_ML_OPERATOR_SET_SCHEMA( OPTIONAL_VALUE) .Attr( "nodes_modes", - "The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf node.
One of 'BRANCH_LEQ', 'BRANCH_LT', 'BRANCH_GTE', 'BRANCH_GT', 'BRANCH_EQ', 'BRANCH_NEQ', 'LEAF'", + "The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf " + "node.
One of 'BRANCH_LEQ', 'BRANCH_LT', 'BRANCH_GTE', 'BRANCH_GT', 'BRANCH_EQ', 'BRANCH_NEQ', 'LEAF'", AttributeProto::STRINGS, OPTIONAL_VALUE) .Attr("nodes_truenodeids", "Child node if expression is true.", AttributeProto::INTS, OPTIONAL_VALUE) .Attr("nodes_falsenodeids", "Child node if expression is false.", AttributeProto::INTS, OPTIONAL_VALUE) .Attr( "nodes_missing_value_tracks_true", - "For each node, define what to do in the presence of a missing value: if a value is missing (NaN), use the 'true' or 'false' branch based on the value in this array.
This attribute may be left undefined, and the default value is false (0) for all nodes.", + "For each node, define what to do in the presence of a missing value: if a value is missing (NaN), use the " + "'true' or 'false' branch based on the value in this array.
This attribute may be left undefined, and " + "the default value is false (0) for all nodes.", AttributeProto::INTS, OPTIONAL_VALUE) .Attr("class_treeids", "The id of the tree that this node is in.", AttributeProto::INTS, OPTIONAL_VALUE) @@ -118,22 +124,26 @@ ONNX_ML_OPERATOR_SET_SCHEMA( .Attr("class_weights", "The weight for the class in class_id.", AttributeProto::FLOATS, OPTIONAL_VALUE) .Attr( "classlabels_strings", - "Class labels if using string labels.
One and only one of the 'classlabels_*' attributes must be defined.", + "Class labels if using string labels.
One and only one of the 'classlabels_*' attributes must be " + "defined.", AttributeProto::STRINGS, OPTIONAL_VALUE) .Attr( "classlabels_int64s", - "Class labels if using integer labels.
One and only one of the 'classlabels_*' attributes must be defined.", + "Class labels if using integer labels.
One and only one of the 'classlabels_*' attributes must be " + "defined.", AttributeProto::INTS, OPTIONAL_VALUE) .Attr( "post_transform", - "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' or 'PROBIT.'", + "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' " + "or 'PROBIT.'", AttributeProto::STRING, std::string("NONE")) .Attr( "base_values", - "Base values for classification, added to final class score; the size must be the same as the classes or can be left unassigned (assumed 0)", + "Base values for classification, added to final class score; the size must be the same as the classes or " + "can be left unassigned (assumed 0)", AttributeProto::FLOATS, OPTIONAL_VALUE) .TypeAndShapeInferenceFunction([](InferenceContext& ctx) { @@ -148,6 +158,173 @@ ONNX_ML_OPERATOR_SET_SCHEMA( } })); +static const char* TreeEnsembleClassifier_ver3_doc = R"DOC( + Tree Ensemble classifier. Returns the top class for each of N inputs.
+ The attributes named 'nodes_X' form a sequence of tuples, associated by + index into the sequences, which must all be of equal length. These tuples + define the nodes.
+ Similarly, all fields prefixed with 'class_' are tuples of votes at the leaves. + A leaf may have multiple votes, where each vote is weighted by + the associated class_weights index.
+ One and only one of classlabels_strings or classlabels_int64s + will be defined. The class_ids are indices into this list. + All fields ending with _as_tensor can be used instead of the + same parameter without the suffix if the element type is double and not float. +)DOC"; + +ONNX_ML_OPERATOR_SET_SCHEMA( + TreeEnsembleClassifier, + 3, + OpSchema() + .SetDoc(TreeEnsembleClassifier_ver3_doc) + .Input(0, "X", "Input of shape [N,F]", "T1") + .Output(0, "Y", "N, Top class for each point", "T2") + .Output(1, "Z", "The class score for each class, for each point, a tensor of shape [N,E].", "tensor(float)") + .TypeConstraint( + "T1", + {"tensor(float)", "tensor(double)", "tensor(int64)", "tensor(int32)"}, + "The input type must be a tensor of a numeric type.") + .TypeConstraint( + "T2", + {"tensor(string)", "tensor(int64)"}, + "The output type will be a tensor of strings or integers, depending on which of the classlabels_* " + "attributes is used.") + .Attr("nodes_treeids", "Tree id for each node.", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr( + "nodes_nodeids", + "Node id for each node. Ids may restart at zero for each tree, but it not required to.", + AttributeProto::INTS, + OPTIONAL_VALUE) + .Attr("nodes_featureids", "Feature id for each node.", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr( + "nodes_values", + "Thresholds to do the splitting on for each node.", + AttributeProto::FLOATS, + OPTIONAL_VALUE) + .Attr( + "nodes_values_as_tensor", + "Thresholds to do the splitting on for each node.", + AttributeProto::TENSOR, + OPTIONAL_VALUE) + .Attr( + "nodes_hitrates", + "Popularity of each node, used for performance and may be omitted.", + AttributeProto::FLOATS, + OPTIONAL_VALUE) + .Attr( + "nodes_hitrates_as_tensor", + "Popularity of each node, used for performance and may be omitted.", + AttributeProto::TENSOR, + OPTIONAL_VALUE) + .Attr( + "nodes_modes", + "The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf " + "node.
One of 'BRANCH_LEQ', 'BRANCH_LT', 'BRANCH_GTE', 'BRANCH_GT', 'BRANCH_EQ', 'BRANCH_NEQ', 'LEAF'", + AttributeProto::STRINGS, + OPTIONAL_VALUE) + .Attr("nodes_truenodeids", "Child node if expression is true.", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr("nodes_falsenodeids", "Child node if expression is false.", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr( + "nodes_missing_value_tracks_true", + "For each node, define what to do in the presence of a missing value: if a value is missing (NaN), use the " + "'true' or 'false' branch based on the value in this array.
This attribute may be left undefined, and " + "the default value is false (0) for all nodes.", + AttributeProto::INTS, + OPTIONAL_VALUE) + .Attr("class_treeids", "The id of the tree that this node is in.", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr("class_nodeids", "node id that this weight is for.", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr("class_ids", "The index of the class list that each weight is for.", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr("class_weights", "The weight for the class in class_id.", AttributeProto::FLOATS, OPTIONAL_VALUE) + .Attr( + "class_weights_as_tensor", + "The weight for the class in class_id.", + AttributeProto::TENSOR, + OPTIONAL_VALUE) + .Attr( + "classlabels_strings", + "Class labels if using string labels.
One and only one of the 'classlabels_*' attributes must be " + "defined.", + AttributeProto::STRINGS, + OPTIONAL_VALUE) + .Attr( + "classlabels_int64s", + "Class labels if using integer labels.
One and only one of the 'classlabels_*' attributes must be " + "defined.", + AttributeProto::INTS, + OPTIONAL_VALUE) + .Attr( + "post_transform", + "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' " + "or 'PROBIT.'", + AttributeProto::STRING, + std::string("NONE")) + .Attr( + "base_values", + "Base values for classification, added to final class score; the size must be the same as the classes or " + "can be left unassigned (assumed 0)", + AttributeProto::FLOATS, + OPTIONAL_VALUE) + .Attr( + "base_values_as_tensor", + "Base values for classification, added to final class score; the size must be the same as the classes or " + "can be left unassigned (assumed 0)", + AttributeProto::TENSOR, + OPTIONAL_VALUE) + .TypeAndShapeInferenceFunction([](InferenceContext& ctx) { + auto* nodes_values = ctx.getAttribute("nodes_values"); + auto* nodes_values_as_tensor = ctx.getAttribute("nodes_values_as_tensor"); + auto* nodes_hitrates = ctx.getAttribute("nodes_hitrates"); + auto* nodes_hitrates_as_tensor = ctx.getAttribute("nodes_hitrates_as_tensor"); + auto* class_weights = ctx.getAttribute("class_weights"); + auto* class_weights_as_tensor = ctx.getAttribute("class_weights_as_tensor"); + auto* base_values = ctx.getAttribute("base_values"); + auto* base_values_as_tensor = ctx.getAttribute("base_values_as_tensor"); + + if (nullptr != nodes_values && nullptr != nodes_values_as_tensor) { + fail_shape_inference( + "Only one of the attributes 'nodes_values', 'nodes_values_as_tensor' should be specified."); + } + if (nullptr != nodes_hitrates && nullptr != nodes_hitrates_as_tensor) { + fail_shape_inference( + "Only one of the attributes 'nodes_hitrates', 'nodes_hitrates_as_tensor' should be specified."); + } + if (nullptr != class_weights && nullptr != class_weights_as_tensor) { + fail_shape_inference( + "Only one of the attributes 'class_weights', 'class_weights_as_tensor' should be specified."); + } + if (nullptr != base_values && nullptr != base_values_as_tensor) { + fail_shape_inference( + "Only one of the attributes 'base_values', 'base_values_as_tensor' should be specified."); + } + + std::vector classlabels_strings; + auto result = getRepeatedAttribute(ctx, "classlabels_strings", classlabels_strings); + bool using_strings = (result && !classlabels_strings.empty()); + if (using_strings) { + updateOutputElemType(ctx, 0, TensorProto::STRING); + } else { + updateOutputElemType(ctx, 0, TensorProto::INT64); + } + updateOutputElemType(ctx, 1, TensorProto::FLOAT); + + checkInputRank(ctx, 0, 2); + Dim N, E; + unifyInputDim(ctx, 0, 0, N); + + if (using_strings) { + unifyDim(E, classlabels_strings.size()); + } else { + std::vector classlabels_int64s; + result = getRepeatedAttribute(ctx, "classlabels_int64s", classlabels_int64s); + if (!result || classlabels_int64s.empty()) { + fail_shape_inference("Non of classlabels_int64s or classlabels_strings is set."); + } + unifyDim(E, classlabels_int64s.size()); + } + updateOutputShape(ctx, 0, {N}); + updateOutputShape(ctx, 1, {N, E}); + })); + static const char* TreeEnsembleRegressor_ver1_doc = R"DOC( Tree Ensemble regressor. Returns the regressed values for each input in N.
All args with nodes_ are fields of a tuple of tree nodes, and @@ -191,14 +368,17 @@ ONNX_ML_OPERATOR_SET_SCHEMA( OPTIONAL_VALUE) .Attr( "nodes_modes", - "The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf node.
One of 'BRANCH_LEQ', 'BRANCH_LT', 'BRANCH_GTE', 'BRANCH_GT', 'BRANCH_EQ', 'BRANCH_NEQ', 'LEAF'", + "The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf " + "node.
One of 'BRANCH_LEQ', 'BRANCH_LT', 'BRANCH_GTE', 'BRANCH_GT', 'BRANCH_EQ', 'BRANCH_NEQ', 'LEAF'", AttributeProto::STRINGS, OPTIONAL_VALUE) .Attr("nodes_truenodeids", "Child node if expression is true", AttributeProto::INTS, OPTIONAL_VALUE) .Attr("nodes_falsenodeids", "Child node if expression is false", AttributeProto::INTS, OPTIONAL_VALUE) .Attr( "nodes_missing_value_tracks_true", - "For each node, define what to do in the presence of a NaN: use the 'true' (if the attribute value is 1) or 'false' (if the attribute value is 0) branch based on the value in this array.
This attribute may be left undefined and the default value is false (0) for all nodes.", + "For each node, define what to do in the presence of a NaN: use the 'true' (if the attribute value is 1) " + "or 'false' (if the attribute value is 0) branch based on the value in this array.
This attribute may " + "be left undefined and the default value is false (0) for all nodes.", AttributeProto::INTS, OPTIONAL_VALUE) .Attr("target_treeids", "The id of the tree that each node is in.", AttributeProto::INTS, OPTIONAL_VALUE) @@ -208,7 +388,8 @@ ONNX_ML_OPERATOR_SET_SCHEMA( .Attr("n_targets", "The total number of targets.", AttributeProto::INT, OPTIONAL_VALUE) .Attr( "post_transform", - "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' or 'PROBIT'", + "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' " + "or 'PROBIT'", AttributeProto::STRING, std::string("NONE")) .Attr( @@ -218,10 +399,145 @@ ONNX_ML_OPERATOR_SET_SCHEMA( std::string("SUM")) .Attr( "base_values", - "Base values for classification, added to final class score; the size must be the same as the classes or can be left unassigned (assumed 0)", + "Base values for classification, added to final class score; the size must be the same as the classes or " + "can be left unassigned (assumed 0)", AttributeProto::FLOATS, OPTIONAL_VALUE)); +static const char* TreeEnsembleRegressor_ver3_doc = R"DOC( + Tree Ensemble regressor. Returns the regressed values for each input in N.
+ All args with nodes_ are fields of a tuple of tree nodes, and + it is assumed they are the same length, and an index i will decode the + tuple across these inputs. Each node id can appear only once + for each tree id.
+ All fields prefixed with target_ are tuples of votes at the leaves.
+ A leaf may have multiple votes, where each vote is weighted by + the associated target_weights index.
+ All fields ending with _as_tensor can be used instead of the + same parameter without the suffix if the element type is double and not float. + All trees must have their node ids start at 0 and increment by 1.
+ Mode enum is BRANCH_LEQ, BRANCH_LT, BRANCH_GTE, BRANCH_GT, BRANCH_EQ, BRANCH_NEQ, LEAF +)DOC"; + +ONNX_ML_OPERATOR_SET_SCHEMA( + TreeEnsembleRegressor, + 3, + OpSchema() + .SetDoc(TreeEnsembleRegressor_ver3_doc) + .Input(0, "X", "Input of shape [N,F]", "T") + .Output(0, "Y", "N classes", "tensor(float)") + .TypeConstraint( + "T", + {"tensor(float)", "tensor(double)", "tensor(int64)", "tensor(int32)"}, + "The input type must be a tensor of a numeric type.") + .Attr("nodes_treeids", "Tree id for each node.", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr( + "nodes_nodeids", + "Node id for each node. Node ids must restart at zero for each tree and increase sequentially.", + AttributeProto::INTS, + OPTIONAL_VALUE) + .Attr("nodes_featureids", "Feature id for each node.", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr( + "nodes_values", + "Thresholds to do the splitting on for each node.", + AttributeProto::FLOATS, + OPTIONAL_VALUE) + .Attr( + "nodes_values_as_tensor", + "Thresholds to do the splitting on for each node.", + AttributeProto::TENSOR, + OPTIONAL_VALUE) + .Attr( + "nodes_hitrates", + "Popularity of each node, used for performance and may be omitted.", + AttributeProto::FLOATS, + OPTIONAL_VALUE) + .Attr( + "nodes_hitrates_as_tensor", + "Popularity of each node, used for performance and may be omitted.", + AttributeProto::TENSOR, + OPTIONAL_VALUE) + .Attr( + "nodes_modes", + "The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf " + "node.
One of 'BRANCH_LEQ', 'BRANCH_LT', 'BRANCH_GTE', 'BRANCH_GT', 'BRANCH_EQ', 'BRANCH_NEQ', 'LEAF'", + AttributeProto::STRINGS, + OPTIONAL_VALUE) + .Attr("nodes_truenodeids", "Child node if expression is true", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr("nodes_falsenodeids", "Child node if expression is false", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr( + "nodes_missing_value_tracks_true", + "For each node, define what to do in the presence of a NaN: use the 'true' (if the attribute value is 1) " + "or 'false' (if the attribute value is 0) branch based on the value in this array.
This attribute may " + "be left undefined and the default value is false (0) for all nodes.", + AttributeProto::INTS, + OPTIONAL_VALUE) + .Attr("target_treeids", "The id of the tree that each node is in.", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr("target_nodeids", "The node id of each weight", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr("target_ids", "The index of the target that each weight is for", AttributeProto::INTS, OPTIONAL_VALUE) + .Attr("target_weights", "The weight for each target", AttributeProto::FLOATS, OPTIONAL_VALUE) + .Attr("target_weights_as_tensor", "The weight for each target", AttributeProto::TENSOR, OPTIONAL_VALUE) + .Attr("n_targets", "The total number of targets.", AttributeProto::INT, OPTIONAL_VALUE) + .Attr( + "post_transform", + "Indicates the transform to apply to the score.
One of 'NONE,' 'SOFTMAX,' 'LOGISTIC,' 'SOFTMAX_ZERO,' " + "or 'PROBIT'", + AttributeProto::STRING, + std::string("NONE")) + .Attr( + "aggregate_function", + "Defines how to aggregate leaf values within a target.
One of 'AVERAGE,' 'SUM,' 'MIN,' 'MAX.'", + AttributeProto::STRING, + std::string("SUM")) + .Attr( + "base_values", + "Base values for regression, added to final prediction after applying aggregate_function; the size must be " + "the same as the classes or can be left unassigned (assumed 0)", + AttributeProto::FLOATS, + OPTIONAL_VALUE) + .Attr( + "base_values_as_tensor", + "Base values for regression, added to final prediction after applying aggregate_function; the size must be " + "the same as the classes or can be left unassigned (assumed 0)", + AttributeProto::TENSOR, + OPTIONAL_VALUE) + .TypeAndShapeInferenceFunction([](InferenceContext& ctx) { + auto* nodes_values = ctx.getAttribute("nodes_values"); + auto* nodes_values_as_tensor = ctx.getAttribute("nodes_values_as_tensor"); + auto* nodes_hitrates = ctx.getAttribute("nodes_hitrates"); + auto* nodes_hitrates_as_tensor = ctx.getAttribute("nodes_hitrates_as_tensor"); + auto* target_weights = ctx.getAttribute("target_weights"); + auto* target_weights_as_tensor = ctx.getAttribute("target_weights_as_tensor"); + auto* base_values = ctx.getAttribute("base_values"); + auto* base_values_as_tensor = ctx.getAttribute("base_values_as_tensor"); + + if (nullptr != nodes_values && nullptr != nodes_values_as_tensor) { + fail_shape_inference( + "Only one of the attributes 'nodes_values', 'nodes_values_as_tensor' should be specified."); + } + if (nullptr != nodes_hitrates && nullptr != nodes_hitrates_as_tensor) { + fail_shape_inference( + "Only one of the attributes 'nodes_hitrates', 'nodes_hitrates_as_tensor' should be specified."); + } + if (nullptr != target_weights && nullptr != target_weights_as_tensor) { + fail_shape_inference( + "Only one of the attributes 'target_weights', 'target_weights_as_tensor' should be specified."); + } + if (nullptr != base_values && nullptr != base_values_as_tensor) { + fail_shape_inference( + "Only one of the attributes 'base_values', 'base_values_as_tensor' should be specified."); + } + + checkInputRank(ctx, 0, 2); + Dim N, E; + unifyInputDim(ctx, 0, 0, N); + if (nullptr != ctx.getAttribute("n_targets")) { + unifyDim(E, ctx.getAttribute("n_targets")->i()); + } + updateOutputElemType(ctx, 0, TensorProto::FLOAT); + updateOutputShape(ctx, 0, {N, E}); + })); + static const char* LabelEncoder_ver2_doc = R"DOC( Maps each element in the input tensor to another value.
The mapping is determined by the two parallel attributes, 'keys_*' and @@ -337,6 +653,5 @@ ONNX_ML_OPERATOR_SET_SCHEMA( // Input and output shapes are the same. propagateShapeFromInputToOutput(ctx, 0, 0); })); - } // namespace ONNX_NAMESPACE #endif diff --git a/onnx/defs/traditionalml/utils.h b/onnx/defs/traditionalml/utils.h new file mode 100644 index 00000000000..c0aeeab2b1d --- /dev/null +++ b/onnx/defs/traditionalml/utils.h @@ -0,0 +1,27 @@ +#include "onnx/defs/schema.h" +#include "onnx/defs/shape_inference.h" + +namespace ONNX_NAMESPACE { + +void AssertAttributeProtoTypeAndLength( + const AttributeProto* attr_proto, + int expected_length, + TensorProto_DataType expected_type, + bool required) { + if (nullptr == attr_proto) { + if (required) { + fail_shape_inference("Unspecified required attribute."); + } + return; + } + const auto& [type, length] = getAttributeProtoElemTypeAndLength(attr_proto); + if (type != expected_type) { + fail_shape_inference( + "Attribute '", attr_proto->name(), "' must have type ", TensorProto_DataType_Name(expected_type), "."); + } + if (length != expected_length) { + fail_shape_inference("Attribute '", attr_proto->name(), "' must have ", expected_length, " elements."); + } +} + +} // namespace ONNX_NAMESPACE diff --git a/onnx/helper.py b/onnx/helper.py index 0e12dbc2207..37b67fb1bbc 100644 --- a/onnx/helper.py +++ b/onnx/helper.py @@ -75,6 +75,7 @@ ("1.14.0", 9, 19, 3, 1), ("1.14.1", 9, 19, 3, 1), ("1.15.0", 9, 20, 4, 1), + ("1.16.0", 9, 20, 5, 1), ] VersionMapType = Dict[Tuple[str, int], int] diff --git a/onnx/reference/ops/aionnxml/_op_list.py b/onnx/reference/ops/aionnxml/_op_list.py index 71cfe144aa6..ee8af30d8f9 100644 --- a/onnx/reference/ops/aionnxml/_op_list.py +++ b/onnx/reference/ops/aionnxml/_op_list.py @@ -25,6 +25,7 @@ from onnx.reference.ops.aionnxml.op_scaler import Scaler from onnx.reference.ops.aionnxml.op_svm_classifier import SVMClassifier from onnx.reference.ops.aionnxml.op_svm_regressor import SVMRegressor +from onnx.reference.ops.aionnxml.op_tree_ensemble import TreeEnsemble from onnx.reference.ops.aionnxml.op_tree_ensemble_classifier import ( TreeEnsembleClassifier, ) diff --git a/onnx/reference/ops/aionnxml/op_tree_ensemble.py b/onnx/reference/ops/aionnxml/op_tree_ensemble.py new file mode 100644 index 00000000000..4c37ccb59a5 --- /dev/null +++ b/onnx/reference/ops/aionnxml/op_tree_ensemble.py @@ -0,0 +1,253 @@ +from __future__ import annotations + +from enum import IntEnum +from typing import Callable + +import numpy as np + +from onnx.reference.ops.aionnxml._op_run_aionnxml import OpRunAiOnnxMl + + +class AggregationFunction(IntEnum): + AVERAGE = 0 + SUM = 1 + MIN = 2 + MAX = 3 + + +class PostTransform(IntEnum): + NONE = 0 + SOFTMAX = 1 + LOGISTIC = 2 + SOFTMAX_ZERO = 3 + PROBIT = 4 + + +class Mode(IntEnum): + LEQ = 0 + LT = 1 + GTE = 2 + GT = 3 + EQ = 4 + NEQ = 5 + MEMBER = 6 + + +class Leaf: + def __init__(self, weight: float, target_id: int) -> None: + self.weight = weight + self.target_id = target_id + + # Produce the weight and target index + def predict(self, x: np.ndarray) -> np.ndarray: + return np.array([self.weight, self.target_id]) + + def _print(self, prefix: list, indent: int = 0) -> None: + prefix.append( + " " * indent + f"Leaf WEIGHT: {self.weight}, TARGET: {self.target_id}\n" + ) + + def __repr__(self) -> str: + prefix = [] + self._print(prefix) + return "".join(prefix) + + +class Node: + compare: Callable[[float, float | set[float]], bool] + true_branch: Node | Leaf + false_branch: Node | Leaf + feature: int + + def __init__( + self, + mode: Mode, + value: float | set[float], + feature: int, + missing_tracks_true: bool, + ) -> None: + if mode == Mode.LEQ: + self.compare = lambda x: x[feature].item() <= value or ( + missing_tracks_true and np.isnan(x[feature].item()) + ) + elif mode == Mode.LT: + self.compare = lambda x: x[feature].item() < value or ( + missing_tracks_true and np.isnan(x[feature].item()) + ) + elif mode == Mode.GTE: + self.compare = lambda x: x[feature].item() >= value or ( + missing_tracks_true and np.isnan(x[feature].item()) + ) + elif mode == Mode.GT: + self.compare = lambda x: x[feature].item() > value or ( + missing_tracks_true and np.isnan(x[feature].item()) + ) + elif mode == Mode.EQ: + self.compare = lambda x: x[feature].item() == value or ( + missing_tracks_true and np.isnan(x[feature].item()) + ) + elif mode == Mode.NEQ: + self.compare = lambda x: x[feature].item() != value or ( + missing_tracks_true and np.isnan(x[feature].item()) + ) + elif mode == Mode.MEMBER: + self.compare = lambda x: x[feature].item() in value or ( + missing_tracks_true and np.isnan(x[feature].item()) + ) + self.mode = mode + self.value = value + self.feature = feature + + def predict(self, x: np.ndarray) -> float: + if self.compare(x): + return self.true_branch.predict(x) + else: + return self.false_branch.predict(x) + + def _print(self, prefix: list, indent: int = 0) -> None: + prefix.append( + " " * indent + + f"Node CMP: {self.mode}, SPLIT: {self.value}, FEATURE: {self.feature}\n" + ) + self.true_branch._print(prefix, indent + 1) + self.false_branch._print(prefix, indent + 1) + + def __repr__(self) -> str: + prefix = [] + self._print(prefix) + return "".join(prefix) + + +class TreeEnsemble(OpRunAiOnnxMl): + def _run( + self, + X, + nodes_splits, + nodes_featureids, + nodes_modes, + nodes_truenodeids, + nodes_falsenodeids, + nodes_trueleafs, + nodes_falseleafs, + leaf_targetids, + leaf_weights, + tree_roots, + post_transform=PostTransform.NONE, + aggregate_function=AggregationFunction.SUM, + nodes_hitrates=None, + nodes_missing_value_tracks_true=None, + membership_values=None, + n_targets=None, + ): + if membership_values is None: + # assert that no set membership ever appears + if any(mode == Mode.MEMBER for mode in nodes_modes): + raise ValueError( + "Cannot have set membership node without specifying set members" + ) + elif np.isnan(membership_values).sum() != sum( + int(mode == Mode.MEMBER) for mode in nodes_modes + ): + raise ValueError( + "Must specify membership values for all set membership nodes" + ) + + # Build each tree in the ensemble. Note that the tree structure is implicitly defined by following the true and false indices in + # `nodes_truenodeids` and `nodes_falsenodeids` to the leaves of each tree. + set_membership_iter = ( + iter(membership_values) if membership_values is not None else None + ) + + def build_node(current_node_index, is_leaf) -> Node | Leaf: + if is_leaf: + return Leaf( + leaf_weights[current_node_index], leaf_targetids[current_node_index] + ) + + if nodes_modes[current_node_index] == Mode.MEMBER: + # parse next sequence of set members + set_members = set() + while (set_member := next(set_membership_iter)) and not np.isnan( + set_member + ): + set_members.add(set_member) + node = Node( + nodes_modes[current_node_index], + set_members, + nodes_featureids[current_node_index], + nodes_missing_value_tracks_true[current_node_index] + if nodes_missing_value_tracks_true is not None + else False, + ) + else: + node = Node( + nodes_modes[current_node_index], + nodes_splits[current_node_index], + nodes_featureids[current_node_index], + nodes_missing_value_tracks_true[current_node_index] + if nodes_missing_value_tracks_true is not None + else False, + ) + + # recurse true and false branches + node.true_branch = build_node( + nodes_truenodeids[current_node_index], + nodes_trueleafs[current_node_index], + ) + node.false_branch = build_node( + nodes_falsenodeids[current_node_index], + nodes_falseleafs[current_node_index], + ) + return node + + trees = [] + for root_index in tree_roots: + # degenerate case (tree == leaf) + is_leaf = ( + nodes_trueleafs[root_index] + and nodes_falseleafs[root_index] + and nodes_truenodeids[root_index] == nodes_falsenodeids[root_index] + ) + trees.append(build_node(root_index, is_leaf)) + + # predict each sample through tree + raw_values = [ + np.apply_along_axis(tree.predict, axis=1, arr=X) for tree in trees + ] + weights, target_ids = zip(*[np.split(x, 2, axis=1) for x in raw_values]) + weights = np.concatenate(weights, axis=1) + target_ids = np.concatenate(target_ids, axis=1).astype(np.int64) + if aggregate_function in ( + AggregationFunction.SUM, + AggregationFunction.AVERAGE, + ): + result = np.zeros((len(X), n_targets), dtype=X.dtype) + elif aggregate_function == AggregationFunction.MIN: + result = np.full((len(X), n_targets), np.finfo(X.dtype).max) + elif aggregate_function == AggregationFunction.MAX: + result = np.full((len(X), n_targets), np.finfo(X.dtype).min) + else: + raise NotImplementedError( + f"aggregate_transform={aggregate_function!r} not supported yet." + ) + for batch_num, (w, t) in enumerate(zip(weights, target_ids)): + weight = w.reshape(-1) + target_id = t.reshape(-1) + if aggregate_function == AggregationFunction.SUM: + for value, tid in zip(weight, target_id): + result[batch_num, tid] += value + elif aggregate_function == AggregationFunction.AVERAGE: + for value, tid in zip(weight, target_id): + result[batch_num, tid] += value / len(trees) + elif aggregate_function == AggregationFunction.MIN: + for value, tid in zip(weight, target_id): + result[batch_num, tid] = min(result[batch_num, tid], value) + elif aggregate_function == AggregationFunction.MAX: + for value, tid in zip(weight, target_id): + result[batch_num, tid] = max(result[batch_num, tid], value) + else: + raise NotImplementedError( + f"aggregate_transform={aggregate_function!r} not supported yet." + ) + + return (result,) diff --git a/onnx/test/reference_evaluator_ml_test.py b/onnx/test/reference_evaluator_ml_test.py index 15ccc42fc76..2facca276c1 100644 --- a/onnx/test/reference_evaluator_ml_test.py +++ b/onnx/test/reference_evaluator_ml_test.py @@ -4,6 +4,7 @@ # type: ignore +import itertools import unittest from functools import wraps from os import getenv @@ -12,9 +13,10 @@ from numpy.testing import assert_allclose # type: ignore from parameterized import parameterized +import onnx from onnx import ONNX_ML, TensorProto, TypeProto, ValueInfoProto from onnx.checker import check_model -from onnx.defs import onnx_opset_version +from onnx.defs import onnx_ml_opset_version, onnx_opset_version from onnx.helper import ( make_graph, make_model_gen_version, @@ -24,6 +26,11 @@ make_tensor_value_info, ) from onnx.reference import ReferenceEvaluator +from onnx.reference.ops.aionnxml.op_tree_ensemble import ( + AggregationFunction, + Mode, + PostTransform, +) # TODO (https://github.com/microsoft/onnxruntime/issues/14932): Get max supported version from onnxruntime directly # For now, bump the version in CIs whenever there is a new onnxruntime release @@ -36,7 +43,7 @@ ) TARGET_OPSET = onnx_opset_version() - 2 -TARGET_OPSET_ML = 4 +TARGET_OPSET_ML = onnx_ml_opset_version() OPSETS = [make_opsetid("", TARGET_OPSET), make_opsetid("ai.onnx.ml", TARGET_OPSET_ML)] @@ -756,10 +763,81 @@ def test_linear_classifier_unary(self): assert_allclose(expected[1], got[1], atol=1e-6) assert_allclose(expected[0], got[0]) + @staticmethod + def _get_test_tree_ensemble_opset_latest( + aggregate_function, + rule=Mode.LEQ, + unique_targets=False, + input_type=TensorProto.FLOAT, + ): + X = make_tensor_value_info("X", input_type, [None, None]) + Y = make_tensor_value_info("Y", input_type, [None, None]) + if unique_targets: + weights = [ + 1.0, + 10.0, + 100.0, + 1000.0, + 10000.0, + 100000.0, + ] + else: + weights = [ + 0.07692307978868484, + 0.5, + 0.5, + 0.0, + 0.2857142984867096, + 0.5, + ] + node = make_node( + "TreeEnsemble", + ["X"], + ["Y"], + domain="ai.onnx.ml", + n_targets=1, + aggregate_function=aggregate_function, + membership_values=None, + nodes_missing_value_tracks_true=None, + nodes_hitrates=None, + post_transform=0, + tree_roots=[0, 2], + nodes_splits=make_tensor( + "node_splits", + input_type, + (4,), + [ + 0.26645058393478394, + 0.6214364767074585, + -0.5592705607414246, + -0.7208403944969177, + ], + ), + nodes_featureids=[0, 2, 0, 0], + nodes_modes=make_tensor( + "nodes_modes", + TensorProto.UINT8, + (4,), + [rule] * 4, + ), + nodes_truenodeids=[1, 0, 3, 4], + nodes_trueleafs=[0, 1, 1, 1], + nodes_falsenodeids=[2, 1, 3, 5], + nodes_falseleafs=[1, 1, 0, 1], + leaf_targetids=[0, 0, 0, 0, 0, 0], + leaf_weights=make_tensor( + "leaf_weights", input_type, (len(weights),), weights + ), + ) + graph = make_graph([node], "ml", [X], [Y]) + model = make_model_gen_version(graph, opset_imports=OPSETS) + return model + @staticmethod def _get_test_tree_ensemble_regressor( aggregate_function, rule="BRANCH_LEQ", unique_targets=False, base_values=None ): + opsets = [make_opsetid("", TARGET_OPSET), make_opsetid("ai.onnx.ml", 3)] X = make_tensor_value_info("X", TensorProto.FLOAT, [None, None]) Y = make_tensor_value_info("Y", TensorProto.FLOAT, [None, None]) if unique_targets: @@ -826,68 +904,127 @@ def _get_test_tree_ensemble_regressor( target_weights=targets, ) graph = make_graph([node1], "ml", [X], [Y]) - onx = make_model_gen_version(graph, opset_imports=OPSETS) + onx = make_model_gen_version(graph, opset_imports=opsets) check_model(onx) return onx @parameterized.expand( - [ - (f"{agg}_{base_values}", base_values, agg) - for base_values in (None, [1.0]) - for agg in ("SUM", "AVERAGE", "MIN", "MAX") - ] + tuple( + itertools.chain.from_iterable( + ( + ( + AggregationFunction.SUM if opset5 else "SUM", + np.array( + [[0.576923], [0.576923], [0.576923]], dtype=np.float32 + ), + opset5, + ), + ( + AggregationFunction.AVERAGE if opset5 else "AVERAGE", + np.array( + [[0.288462], [0.288462], [0.288462]], dtype=np.float32 + ), + opset5, + ), + ( + AggregationFunction.MIN if opset5 else "MIN", + np.array( + [[0.076923], [0.076923], [0.076923]], dtype=np.float32 + ), + opset5, + ), + ( + AggregationFunction.MAX if opset5 else "MAX", + np.array([[0.5], [0.5], [0.5]], dtype=np.float32), + opset5, + ), + ) + for opset5 in [True, False] + ) + ) ) @unittest.skipIf(not ONNX_ML, reason="onnx not compiled with ai.onnx.ml") - def test_tree_ensemble_regressor(self, name, base_values, agg): - self.assertTrue(ONNX_ML) - del name # variable only used to print test name + def test_tree_ensemble_regressor_aggregation_functions( + self, aggregate_function, expected_result, opset5 + ): x = np.arange(9).reshape((-1, 3)).astype(np.float32) / 10 - 0.5 - expected_agg = { - "SUM": np.array([[0.576923], [0.576923], [0.576923]], dtype=np.float32), - "AVERAGE": np.array([[0.288462], [0.288462], [0.288462]], dtype=np.float32), - "MIN": np.array([[0.076923], [0.076923], [0.076923]], dtype=np.float32), - "MAX": np.array([[0.5], [0.5], [0.5]], dtype=np.float32), - } - - expected = expected_agg[agg] - if base_values is not None: - expected += base_values[0] - with self.subTest(aggregate_function=agg): - onx = self._get_test_tree_ensemble_regressor(agg, base_values=base_values) - self._check_ort(onx, {"X": x}, equal=True) - sess = ReferenceEvaluator(onx) - got = sess.run(None, {"X": x}) - assert_allclose(expected, got[0], atol=1e-6) + model_factory = ( + self._get_test_tree_ensemble_opset_latest + if opset5 + else self._get_test_tree_ensemble_regressor + ) + model_proto = model_factory( + aggregate_function, + ) + sess = ReferenceEvaluator(model_proto) + (actual,) = sess.run(None, {"X": x}) + assert_allclose(expected_result, actual, atol=1e-6) + @parameterized.expand( + tuple( + itertools.chain.from_iterable( + ( + ( + Mode.LEQ if opset5 else "BRANCH_LEQ", + np.array( + [[0.576923], [0.576923], [0.576923]], dtype=np.float32 + ), + opset5, + ), + ( + Mode.GT if opset5 else "BRANCH_GT", + np.array([[0.5], [0.5], [0.5]], dtype=np.float32), + opset5, + ), + ( + Mode.LT if opset5 else "BRANCH_LT", + np.array( + [[0.576923], [0.576923], [0.576923]], dtype=np.float32 + ), + opset5, + ), + ( + Mode.GTE if opset5 else "BRANCH_GTE", + np.array([[0.5], [0.5], [0.5]], dtype=np.float32), + opset5, + ), + ( + Mode.EQ if opset5 else "BRANCH_EQ", + np.array([[1.0], [1.0], [1.0]], dtype=np.float32), + opset5, + ), + ( + Mode.NEQ if opset5 else "BRANCH_NEQ", + np.array( + [[0.076923], [0.076923], [0.076923]], dtype=np.float32 + ), + opset5, + ), + ) + for opset5 in [True, False] + ) + ) + ) @unittest.skipIf(not ONNX_ML, reason="onnx not compiled with ai.onnx.ml") - def test_tree_ensemble_regressor_rule(self): + def test_tree_ensemble_regressor_rule(self, rule, expected, opset5): x = np.arange(9).reshape((-1, 3)).astype(np.float32) / 10 - 0.5 - expected_agg = { - "BRANCH_LEQ": np.array( - [[0.576923], [0.576923], [0.576923]], dtype=np.float32 - ), - "BRANCH_GT": np.array([[0.5], [0.5], [0.5]], dtype=np.float32), - "BRANCH_LT": np.array( - [[0.576923], [0.576923], [0.576923]], dtype=np.float32 - ), - "BRANCH_GTE": np.array([[0.5], [0.5], [0.5]], dtype=np.float32), - "BRANCH_EQ": np.array([[1.0], [1.0], [1.0]], dtype=np.float32), - "BRANCH_NEQ": np.array( - [[0.076923], [0.076923], [0.076923]], dtype=np.float32 - ), - } - for rule, expected in expected_agg.items(): - with self.subTest(rule=rule): - onx = self._get_test_tree_ensemble_regressor("SUM", rule) - self._check_ort(onx, {"X": x}, equal=True) - sess = ReferenceEvaluator(onx) - got = sess.run(None, {"X": x}) - assert_allclose(expected, got[0], atol=1e-6) + model_factory = ( + self._get_test_tree_ensemble_opset_latest + if opset5 + else self._get_test_tree_ensemble_regressor + ) + aggregate_function = AggregationFunction.SUM if opset5 else "SUM" + + model_proto = model_factory(aggregate_function, rule) + sess = ReferenceEvaluator(model_proto) + (actual,) = sess.run(None, {"X": x}) + assert_allclose(expected, actual, atol=1e-6) @unittest.skipIf(not ONNX_ML, reason="onnx not compiled with ai.onnx.ml") - def test_tree_ensemble_regressor_2_targets(self): + def test_tree_ensemble_regressor_2_targets_opset3(self): X = make_tensor_value_info("X", TensorProto.FLOAT, [None, None]) Y = make_tensor_value_info("Y", TensorProto.FLOAT, [None, None]) + opsets = [make_opsetid("", TARGET_OPSET), make_opsetid("ai.onnx.ml", 3)] node1 = make_node( "TreeEnsembleRegressor", ["X"], @@ -972,7 +1109,7 @@ def test_tree_ensemble_regressor_2_targets(self): ], ) graph = make_graph([node1], "ml", [X], [Y]) - onx = make_model_gen_version(graph, opset_imports=OPSETS) + onx = make_model_gen_version(graph, opset_imports=opsets) check_model(onx) x = np.arange(9).reshape((-1, 3)).astype(np.float32) / 10 - 0.5 expected = np.array( @@ -984,7 +1121,7 @@ def test_tree_ensemble_regressor_2_targets(self): assert_allclose(expected, got[0], atol=1e-6) @unittest.skipIf(not ONNX_ML, reason="onnx not compiled with ai.onnx.ml") - def test_tree_ensemble_regressor_missing(self): + def test_tree_ensemble_regressor_missing_opset3(self): x = np.arange(9).reshape((-1, 3)).astype(np.float32) / 10 - 0.5 x[2, 0] = 5 x[1, :] = np.nan @@ -996,6 +1133,165 @@ def test_tree_ensemble_regressor_missing(self): assert_allclose(expected, got[0], atol=1e-6) self.assertIn("op_type=TreeEnsembleRegressor", str(sess.rt_nodes_[0])) + @unittest.skipIf(not ONNX_ML, reason="onnx not compiled with ai.onnx.ml") + @parameterized.expand( + [(input_type,) for input_type in [TensorProto.FLOAT, TensorProto.DOUBLE]] + ) + def test_tree_ensemble_missing_opset5(self, input_type): + model = self._get_test_tree_ensemble_opset_latest( + AggregationFunction.SUM, Mode.LEQ, True, input_type + ) + np_dtype = onnx.helper.tensor_dtype_to_np_dtype(input_type) + x = np.arange(9).reshape((-1, 3)).astype(np_dtype) / 10 - 0.5 + x[2, 0] = 5 + x[1, :] = np.nan + expected = np.array([[100001.0], [100100.0], [100100.0]], dtype=np_dtype) + session = ReferenceEvaluator(model) + (actual,) = session.run(None, {"X": x}) + assert_allclose(expected, actual, atol=1e-6) + + @unittest.skipIf(not ONNX_ML, reason="onnx not compiled with ai.onnx.ml") + def test_tree_ensemble_regressor_missing_opset5_float16(self): + model = self._get_test_tree_ensemble_opset_latest( + AggregationFunction.SUM, Mode.LEQ, False, TensorProto.FLOAT16 + ) + np_dtype = np.float16 + x = np.arange(9).reshape((-1, 3)).astype(np_dtype) / 10 - 0.5 + x[2, 0] = 5 + x[1, :] = np.nan + expected = np.array([[0.577], [1.0], [1.0]], dtype=np_dtype) + session = ReferenceEvaluator(model) + (actual,) = session.run(None, {"X": x}) + assert_allclose(expected, actual, atol=1e-6) + + @unittest.skipIf(not ONNX_ML, reason="onnx not compiled with ai.onnx.ml") + def test_single_tree_ensemble(self): + X = make_tensor_value_info("X", TensorProto.DOUBLE, [None, None]) + Y = make_tensor_value_info("Y", TensorProto.DOUBLE, [None, None]) + node = make_node( + "TreeEnsemble", + ["X"], + ["Y"], + domain="ai.onnx.ml", + n_targets=2, + membership_values=None, + nodes_missing_value_tracks_true=None, + nodes_hitrates=None, + aggregate_function=1, + post_transform=PostTransform.NONE, + tree_roots=[0], + nodes_modes=make_tensor( + "nodes_modes", + TensorProto.UINT8, + (3,), + [Mode.LEQ] * 3, + ), + nodes_featureids=[0, 0, 0], + nodes_splits=make_tensor( + "nodes_splits", + TensorProto.DOUBLE, + (3,), + np.array([3.14, 1.2, 4.2], dtype=np.float64), + ), + nodes_truenodeids=[1, 0, 1], + nodes_trueleafs=[0, 1, 1], + nodes_falsenodeids=[2, 2, 3], + nodes_falseleafs=[0, 1, 1], + leaf_targetids=[0, 1, 0, 1], + leaf_weights=make_tensor( + "leaf_weights", + TensorProto.DOUBLE, + (4,), + np.array([5.23, 12.12, -12.23, 7.21], dtype=np.float64), + ), + ) + graph = make_graph([node], "ml", [X], [Y]) + model = make_model_gen_version( + graph, + opset_imports=[ + make_opsetid("", TARGET_OPSET), + make_opsetid("ai.onnx.ml", 5), + ], + ) + check_model(model) + session = ReferenceEvaluator(model) + (output,) = session.run( + None, + { + "X": np.array([1.2, 3.4, -0.12, 1.66, 4.14, 1.77], np.float64).reshape( + 3, 2 + ) + }, + ) + np.testing.assert_equal( + output, np.array([[5.23, 0], [5.23, 0], [0, 12.12]], dtype=np.float64) + ) + + @unittest.skipIf(not ONNX_ML, reason="onnx not compiled with ai.onnx.ml") + def test_tree_ensemble_regressor_set_membership_opset5(self): + X = make_tensor_value_info("X", TensorProto.FLOAT, [None, None]) + Y = make_tensor_value_info("Y", TensorProto.FLOAT, [None, None]) + node = make_node( + "TreeEnsemble", + ["X"], + ["Y"], + domain="ai.onnx.ml", + n_targets=4, + aggregate_function=AggregationFunction.SUM, + membership_values=make_tensor( + "membership_values", + TensorProto.FLOAT, + (8,), + [1.2, 3.7, 8, 9, np.nan, 12, 7, np.nan], + ), + nodes_missing_value_tracks_true=None, + nodes_hitrates=None, + post_transform=PostTransform.NONE, + tree_roots=[0], + nodes_modes=make_tensor( + "nodes_modes", + TensorProto.UINT8, + (3,), + [Mode.LEQ, Mode.MEMBER, Mode.MEMBER], + ), + nodes_featureids=[0, 0, 0], + nodes_splits=make_tensor( + "nodes_splits", + TensorProto.FLOAT, + (3,), + np.array([11, 232344.0, np.nan], dtype=np.float32), + ), + nodes_trueleafs=[0, 1, 1], + nodes_truenodeids=[1, 0, 1], + nodes_falseleafs=[1, 0, 1], + nodes_falsenodeids=[2, 2, 3], + leaf_targetids=[0, 1, 2, 3], + leaf_weights=make_tensor( + "leaf_weights", TensorProto.FLOAT, (4,), [1, 10, 1000, 100] + ), + ) + graph = make_graph([node], "ml", [X], [Y]) + model = make_model_gen_version( + graph, + opset_imports=OPSETS, + ) + check_model(model) + session = ReferenceEvaluator(model) + X = np.array([1.2, 3.4, -0.12, np.nan, 12, 7], np.float32).reshape(-1, 1) + expected = np.array( + [ + [1, 0, 0, 0], + [0, 0, 0, 100], + [0, 0, 0, 100], + [0, 0, 1000, 0], + [0, 0, 1000, 0], + [0, 10, 0, 0], + ], + dtype=np.float32, + ) + (output,) = session.run(None, {"X": X}) + np.testing.assert_equal(output, expected) + @staticmethod def _get_test_svm_regressor(kernel_type, kernel_params): X = make_tensor_value_info("X", TensorProto.FLOAT, [None, None]) @@ -1138,7 +1434,13 @@ def _get_test_tree_ensemble_classifier_binary(post_transform): post_transform=post_transform, ) graph = make_graph([node1], "ml", [X], [In, Y]) - onx = make_model_gen_version(graph, opset_imports=OPSETS) + onx = make_model_gen_version( + graph, + opset_imports=[ + make_opsetid("", TARGET_OPSET), + make_opsetid("ai.onnx.ml", 3), + ], + ) check_model(onx) return onx @@ -1260,7 +1562,13 @@ def _get_test_tree_ensemble_classifier_multi(post_transform): post_transform=post_transform, ) graph = make_graph([node1], "ml", [X], [In, Y]) - onx = make_model_gen_version(graph, opset_imports=OPSETS) + onx = make_model_gen_version( + graph, + opset_imports=[ + make_opsetid("", TARGET_OPSET), + make_opsetid("ai.onnx.ml", 3), + ], + ) check_model(onx) return onx diff --git a/onnx/test/shape_inference_test.py b/onnx/test/shape_inference_test.py index cd10964e212..c59f287ed44 100644 --- a/onnx/test/shape_inference_test.py +++ b/onnx/test/shape_inference_test.py @@ -9463,6 +9463,139 @@ def test_tree_ensemble_regressor(self) -> None: ], ) + @parameterized.expand([TensorProto.FLOAT, TensorProto.DOUBLE, TensorProto.FLOAT16]) + @unittest.skipUnless(ONNX_ML, "ONNX_ML required to test ai.onnx.ml operators") + def test_tree_ensemble(self, dtype) -> None: + interior_nodes = 5 + leaves = 9 + tree = make_node( + "TreeEnsemble", + ["x"], + ["y"], + domain=ONNX_ML_DOMAIN, + n_targets=5, + nodes_featureids=[0] * interior_nodes, + nodes_splits=make_tensor( + "nodes_splits", + dtype, + (interior_nodes,), + list(range(interior_nodes)), + ), + nodes_modes=make_tensor( + "nodes_modes", + TensorProto.UINT8, + (interior_nodes,), + [0] * interior_nodes, + ), + nodes_truenodeids=[0] * interior_nodes, + nodes_falsenodeids=[0] * interior_nodes, + nodes_trueleafs=[0] * interior_nodes, + nodes_falseleafs=[0] * interior_nodes, + membership_values=make_tensor( + "membership_values", + dtype, + (7,), + [0.0, 0.1, 0.2, np.nan, 0.4, 0.5, 1.0], + ), + leaf_targetids=[0] * leaves, + leaf_weights=make_tensor("leaf_weights", dtype, (leaves,), [1] * leaves), + tree_roots=[0], + ) + + graph = self._make_graph( + [("x", dtype, ("Batch Size", "Features"))], + [tree], + [], + ) + + self._assert_inferred( + graph, + [make_tensor_value_info("y", dtype, ("Batch Size", 5))], + opset_imports=[ + make_opsetid(ONNX_ML_DOMAIN, 5), + make_opsetid(ONNX_DOMAIN, 11), + ], + ) + + @parameterized.expand( + [ + { + "nodes_truenodeids": [0] * 6, + "leaf_weights": make_tensor( + "leaf_weights", TensorProto.DOUBLE, (9,), [1] * 9 + ), + "nodes_splits": make_tensor( + "nodes_splits", TensorProto.DOUBLE, (5,), [1] * 5 + ), + }, + { + "nodes_truenodeids": [0] * 5, + "leaf_weights": make_tensor( + "leaf_weights", TensorProto.FLOAT, (9,), [1] * 9 + ), + "nodes_splits": make_tensor( + "nodes_splits", TensorProto.DOUBLE, (5,), [1] * 5 + ), + }, + { + "nodes_truenodeids": [0] * 5, + "leaf_weights": make_tensor( + "leaf_weights", TensorProto.DOUBLE, (18,), [1] * 18 + ), + "nodes_splits": make_tensor( + "nodes_splits", TensorProto.DOUBLE, (5,), [1] * 5 + ), + }, + { + "nodes_truenodeids": [0] * 5, + "leaf_weights": make_tensor( + "leaf_weights", TensorProto.DOUBLE, (9,), [1] * 9 + ), + "nodes_splits": make_tensor( + "nodes_splits", TensorProto.FLOAT, (5,), [1] * 5 + ), + }, + ] + ) + @unittest.skipUnless(ONNX_ML, "ONNX_ML required to test ai.onnx.ml operators") + def test_tree_ensemble_fails_if_invalid_attributes( + self, + nodes_truenodeids, + leaf_weights, + nodes_splits, + ) -> None: + interior_nodes = 5 + leaves = 9 + tree = make_node( + "TreeEnsemble", + ["x"], + ["y"], + domain=ONNX_ML_DOMAIN, + n_targets=5, + nodes_featureids=[0] * interior_nodes, + nodes_splits=nodes_splits, + nodes_modes=make_tensor( + "nodes_modes", + TensorProto.UINT8, + (interior_nodes,), + [0] * interior_nodes, + ), + nodes_truenodeids=nodes_truenodeids, + nodes_falsenodeids=[0] * interior_nodes, + nodes_trueleafs=[0] * interior_nodes, + nodes_falseleafs=[0] * interior_nodes, + leaf_targetids=[0] * leaves, + leaf_weights=leaf_weights, + tree_roots=[0], + ) + + graph = self._make_graph( + [("x", TensorProto.DOUBLE, ("Batch Size", "Features"))], + [tree], + [], + ) + self.assertRaises(onnx.shape_inference.InferenceError, self._inferred, graph) + @unittest.skipUnless(ONNX_ML, "ONNX_ML required to test ai.onnx.ml operators") def test_tree_ensemble_classifier(self) -> None: tree = make_node( diff --git a/onnx/test/test_backend_onnxruntime.py b/onnx/test/test_backend_onnxruntime.py index 9591dd19d41..43b5a8e95ca 100644 --- a/onnx/test/test_backend_onnxruntime.py +++ b/onnx/test/test_backend_onnxruntime.py @@ -316,6 +316,7 @@ def run_node(cls, node, inputs, device=None, outputs_info=None, **kwargs): "|qlinearmatmul_3D_uint8_float16" # new/updated test cases with opset and/or IR version not supported by onnxruntime 1.17 "|qlinearmatmul_2D_uint8_float32" # new/updated test cases with opset and/or IR version not supported by onnxruntime 1.17 "|qlinearmatmul_3D_uint8_float32" # new/updated test cases with opset and/or IR version not supported by onnxruntime 1.17 + "|tree_ensemble" # tree_ensemble not yet implemented in ort ")" )