Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add the clamp() algorithm #348

Closed
wants to merge 3 commits into from
Closed

Add the clamp() algorithm #348

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5973cd9
Add the clamp() algorithm
zolkis Feb 15, 2023
d68bd87
Add the 'copy MLOperand' and 'create MLActivation' steps
zolkis May 25, 2023
dc4d3d6
clamp(): improve platform related steps
zolkis Jun 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
122 changes: 106 additions & 16 deletions index.bs
View file
Open in desktop
Expand Up @@ -754,6 +754,16 @@ To <dfn>create MLOperand</dfn> given |builder| and |desc|, run the following ste
1. Return |operand|.
</div>

To <dfn>copy MLOperand</dfn> given |operand|, run the following steps:
<div class=algorithm-steps>
1. If |operand| is not an instance of {{MLOperand}}, then throw a "{{TypeError}}" and stop.
1. Let |result| be a new [=object=].
1. Set |result|.{{MLOperand/[[builder]]}} to |operand|.{{MLOperand/[[builder]]}}.
1. Set |result|.{{MLOperand/[[descriptor]]}} to |operand|.{{MLOperand/[[descriptor]]}}.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do deep copy of the descriptor? Would the change of original operand's descriptor impact the copied one?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All internal slots should be immutable, i.e. only changeable with API calls, so assignments to internal slots should involve deep copies by default. I will check this.

Copy link
Collaborator Author

@zolkis zolkis Jun 2, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ECMAScript doesn't really shed light on this. Internal slots are supposed to be created and then not modified unless specified differently.

Now we seem to have 2 options:

  1. change every line that sets an internal slot to replace the argument with "a copy of the argument"
  2. or, create steps to "set the internal slot [[name]] to <arg>" steps, where we say "set [[name]] to a copy of <arg>".

@anssiko @dontcallmedom any input here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@huningxin
Since this is a general issue for the spec, could we create an issue and fix it outside this PR, in all affected places?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine to fix it in a separate issue. Please create that issue and add link into the spec. Thanks.

1. If |operand|.{{MLOperand/[[name]]}} [=map/exists=], then set |result|.{{MLOperand/[[name]]}} to |operand|.{{MLOperand/[[name]]}}.
1. Return |result|.
</div>

To <dfn>check dimensions</dfn> given |dimensions| and |type|, run the following steps:
<div class=algorithm-steps>
1. If |dimensions| is not an array of positive numbers, return `false`;
Expand All @@ -769,16 +779,53 @@ Objects implementing the {{MLActivation}} interface represent activation functio

<script type=idl>
[SecureContext, Exposed=(Window, DedicatedWorker)]
interface MLActivation {};
interface MLActivation {
};
</script>

<div class="internal-slots">
{{MLActivation}} has the following internal slots:
<dl dfn-type=attribute dfn-for="MLActivation">
: <dfn>\[[name]]</dfn> of type [=string=]
::
The {{MLActivation}}'s name.
: <dfn>\[[builder]]</dfn> of type {{MLGraphBuilder}}
::
The graph builder object this {{MLActivation}} belongs to.
: <dfn>\[[options]]</dfn> of type [=object=]
::
A dictionary containing {{MLActivation}} options.
: <dfn>\[[operator]]</dfn> of type [=object=]
::
Reference to {{MLActivation}}'s corresponding [=implementation-defined=] platform operator object.
</dl>
</div>

<div class="note">
These activations function types are used to create other operations. One such use of this interface is for when an activation function is fused into another operation such as [[#api-mlgraphbuilder-conv2d]] or [[#api-mlgraphbuilder-batchnorm]] during a graph construction session. Such fused activation functions can provide a significant performance improvement when supported natively by the underlying implementation. This is intended as an optimization opportunity for implementers.
</div>

#### Creating {{MLActivation}} #### {#api-mlactivation-create}
<div class="note">
The implementation of the {{MLActivation}} interface can simply be a struct that holds a string type of the activation function along with other properties needed. The actual creation of the activation function e.g. a [[#api-mlgraphbuilder-sigmoid]] or [[#api-mlgraphbuilder-relu]] can then be deferred until when the rest of the graph is ready to connect with it such as during the construction of [[#api-mlgraphbuilder-conv2d]] for example.
</div>
The {{MLActivation}} objects (including the ones passed as input to methods) are created by the methods of {{MLGraphBuilder}} and are identified by their name. The |options| dictionary is defined by those methods. The actual creation of the activation function e.g. a [[#api-mlgraphbuilder-sigmoid]] or [[#api-mlgraphbuilder-relu]] can then be deferred until when the rest of the graph is ready to connect with it such as during the construction of [[#api-mlgraphbuilder-conv2d]] for example.
</div>

<details open>
<summary>
To <dfn>create MLActivation</dfn> given |builder|, |name| and |options|, run the following steps:
</summary>
<div class=algorithm-steps>
1. If |builder| is not an instance of {{MLGraphBuilder}}, throw a "{{TypeError}}" and abort these steps.
1. If |name| is `undefined` or `null`, throw a "{{TypeError}}" and abort these steps.
1. Let |activation| be a new [=object=].
1. Set |activation|.{{MLActivation/[[builder]]}} to |builder|.
1. Set |activation|.{{MLActivation/[[name]]}} to |name|.
1. If |options| is an [=object=], set |activation|.{{MLActivation/[[options]]}} to |options|.
1. Make a request to the underlying platform to bind the [=implementation-defined=] platform operator for |name| to |activation|.{{MLActivation/[[operator]]}}.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we leave the platform operator creation to each build method? Because I suppose the creation steps would be different, e.g. handling of options.

Copy link
Collaborator Author

@zolkis zolkis Jun 1, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. Let's start with that assumption. We can always cleanly factor out common things when the opportunity comes.

1. If that fails, throw a "{{TypeError}}" and abort these steps.
1. Return |activation|.
</div>
</details>

## The MLContext interface ## {#api-mlcontext}
The {{MLContext}} interface represents a global state of neural network compute workload and execution processes. Each {{MLContext}} object has associated [=context type=], [=device type=] and [=power preference=].
Expand Down Expand Up @@ -1371,22 +1418,12 @@ dictionary MLClampOptions {
};

partial interface MLGraphBuilder {
MLOperand clamp(MLOperand x, optional MLClampOptions options = {});
MLOperand clamp(MLOperand operand, optional MLClampOptions options = {});
MLActivation clamp(optional MLClampOptions options = {});
};
</script>
<div algorithm=clamp>
**Arguments:**
- *x*: an {{MLOperand}}. The input tensor.
- *options*: an optional {{MLClampOptions}}. The optional parameters of the operation.
- *minValue*: a {{float}} scalar. Specifies the minimum value of the range. When it is not specified, the clamping is not performed on the lower limit of the range.
- *maxValue*: a {{float}} scalar. Specifies the maximum value of the range. When it is not specified, the clamping is not performed on the upper limit of the range.

**Returns:**
- an {{MLOperand}}. The output tensor of the same shape as *x*.
- an {{MLActivation}}. The activation function representing the clamp operation.

<div class="note">
<div class="note">
The behavior of this operation can be generically emulated from the usage of
other operations as follow. However, user agents typically have a more
efficient implementation for it, therefore its usage is encouraged from the
Expand All @@ -1408,7 +1445,60 @@ partial interface MLGraphBuilder {
}
}
</pre>
</div>
</div>

To <dfn>check clamp options</dfn> given |options|, run the following steps:
1. If |options| is not an object that [=implements=] {{MLClampOptions}}, then return `false`.
1. If |options|.{{MLClampOptions/minValue}} and |options|.{{MLClampOptions/maxValue}} are not a [=numeric type=], then then return `false`.
1. If |options|.{{MLClampOptions/minValue}} is greater than |options|.{{MLClampOptions/maxValue}}, then return `false`.
1. Return `true`.

#### The {{MLGraphBuilder/clamp(operand, options)}} method #### {#api-mlgraphbuilder-clamp-operand-options}
<div class="note">
**Arguments:**
- *operand*: an {{MLOperand}}. The input tensor.
- *options*: an optional {{MLClampOptions}}. The optional parameters of the operation.
- *minValue*: a {{float}} scalar. Specifies the minimum value of the range. When it is not specified, the clamping is not performed on the lower limit of the range.
- *maxValue*: a {{float}} scalar. Specifies the maximum value of the range. When it is not specified, the clamping is not performed on the upper limit of the range.
**Returns:**
- an {{MLOperand}}. The output tensor of the same shape as *operand*.
</div>
<div algorithm=clamp>
The {{MLGraphBuilder/clamp(operand, options)}} method steps are:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

operand is commonly used, e.g. for MLOperand or platform operand etc. Should the variable name be more specific, like input or x?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, so that was the reason for using x. Well, input is also a loaded name :).
Since it is an MLOperand, I think using operand is just fine.

1. Let |operand| be the first argument.
1. Let |options| be the second argument.
1. If running the <a>check clamp options</a> steps with |options| returns `false`, then throw a "{{TypeError}}" {{DOMException}} and abort these steps.
1. Let |result| be the result of invoking the <a>copy MLOperand</a> steps given |operand|.
1. If that throws an error, re-throw the error and abort these steps.
huningxin marked this conversation as resolved.
Show resolved Hide resolved
1. If any of the following sub-steps fail, throw an "{{OperationError}}" {{DOMException}} and stop.
1. Make a request to the underlying platform to create an [=implementation-defined=] platform operand |operandImpl| given |result|.{{MLOperand/[[descriptor]]}}.
1. Store a reference to |operandImpl| in |result|.{{MLOperand/[[operand]]}}.
1. Make a request to the underlying platform to create an [=implementation-defined=] platform operator |operatorImpl| for clamp with |options|.{{MLClampOptions/minValue}} and |options|.{{MLClampOptions/minValue}}.
1. Register the |operand|.{{MLOperand/[[operand]]}} as an input to |operatorImpl|.
1. Register the |result|.{{MLOperand/[[operand]]}} as output to |operatorImpl|.
1. Store a reference to |operatorImpl| in |result|.{{MLOperand/[[operator]]}}.
1. Return |result|.
</div>

#### The {{MLGraphBuilder/clamp(options)}} method #### {#api-mlgraphbuilder-clamp-options}
<div class="note">
**Arguments:**
- *options*: an optional {{MLClampOptions}}. The optional parameters of the operation.
- *minValue*: a {{float}} scalar. Specifies the minimum value of the range. When it is not specified, the clamping is not performed on the lower limit of the range.
- *maxValue*: a {{float}} scalar. Specifies the maximum value of the range. When it is not specified, the clamping is not performed on the upper limit of the range.
**Returns:**
- an {{MLActivation}}. The operator representing the clamp operation.
</div>
<div algorithm=clamp-options>
The {{MLGraphBuilder/clamp(options)}} method steps are:
1. Let |options| be the first argument.
1. If running the <a>check clamp options</a> steps with |options| returns `false`, then throw a "{{TypeError}}" {{DOMException}} and abort these steps.
1. Let |op| be the result of invoking the <a>create MLActivation</a> steps with `"clamp"` and |options|.
1. If that throws an error, re-throw the error and abort these steps.
1. If any of the following sub-steps fail, throw an "{{OperationError}}" {{DOMException}} and stop.
1. Make a request to the underlying platform to connect |op| with the [=implementation-defined=] platform operator for clamp |operatorImpl|.
1. Store a reference to |operatorImpl| in |op|.{{MLActivation/[[operator]]}}.
1. Return |op|.
</div>

### The concat() method ### {#api-mlgraphbuilder-concat}
Expand Down