Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ONNX] [#8838] QLinearLeakyRelu contrib op #9063

Merged
merged 4 commits into from
Oct 2, 2021
Merged

[ONNX] [#8838] QLinearLeakyRelu contrib op #9063

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
917dab7
[ONNX] QLinearLeakyRelu contrib op
gayatripk1 Sep 21, 2021
c6c5b76
Add comment
gayatripk1 Sep 22, 2021
6b6f042
jostle ci
gayatripk1 Sep 29, 2021
559bc32
jostle ci
gayatripk1 Oct 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions python/tvm/relay/frontend/onnx.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3430,6 +3430,28 @@ def _impl_v10(cls, inputs, attr, params):
return _qnn.op.quantize(out, y_scale, y_zero_point, out_dtype=dtype)


class QLinearLeakyRelu(OnnxOpConverter):
"""Operator converter for QLinearLeakyRelu from Microsoft onnxruntime contrib opset."""

@classmethod
def _impl_v10(cls, inputs, attr, params):

a_scale = get_scalar(inputs[1], params)
a_zero_point = get_scalar(inputs[2], params, "int32")
y_scale = fold_constant(get_scalar(inputs[3], params))
y_zero_point = get_scalar(inputs[4], params, "int32")
alpha = float(attr.get("alpha", 1.0))

dtype = infer_type(inputs[0]).checked_type.dtype

# Onnxruntime doesn't actually do this op in integer, they dequantize to fp32
# and then requantize afer (according to documentation below)
# https://github.com/microsoft/onnxruntime/blob/master/docs/ContribOperators.md#com.microsoft.QLinearLeakyRelu
a = _qnn.op.dequantize(inputs[0], a_scale, a_zero_point)
out = _op.nn.leaky_relu(a, alpha)
return _qnn.op.quantize(out, y_scale, y_zero_point, out_dtype=dtype)
Comment on lines +3450 to +3452
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're dequantinzing, can you provide a link to ORT doing the same thing? if this is an ORT op and they're running it in int8, I'd prefer to import it in int8 here?f

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the ORT op definition from docs

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If alpha is a power of two e.g. 1 / 16 or 1 / 8, then we can implement this via bit shift operations assuming zero can be represented in the quantization scheme. Not making a suggestion or anything, just thought it was an interesting point.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be an interesting point to try out. Here, we wanted to reuse the existing function.



class QLinearSigmoid(OnnxOpConverter):
"""Operator converter for QLinearSigmoid from Microsoft onnxruntime contrib opset."""

Expand Down Expand Up @@ -4114,6 +4136,7 @@ def _get_convert_map(opset):
"ConvInteger": ConvInteger.get_converter(opset),
"QLinearAveragePool": QLinearAveragePool.get_converter(opset),
"QLinearGlobalAveragePool": QLinearGlobalAveragePool.get_converter(opset),
"QLinearLeakyRelu": QLinearLeakyRelu.get_converter(opset),
# Random number generation.
"RandomUniform": RandomUniform.get_converter(opset),
# Loss functions / training
Expand Down
21 changes: 21 additions & 0 deletions tests/python/frontend/onnx/test_forward.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5557,6 +5557,27 @@ def verify_qlinearmul(a_shape, b_shape, c_shape):
verify_qlinearmul([5, 1, 7], [2, 7], [5, 2, 7])


@tvm.testing.parametrize_targets
def test_qlinearleakyrelu(target, dev):
def verify_qlinearleakyrelu(inshape, kwargs):

in_array = np.random.random(inshape).astype("float32")
node = helper.make_node("LeakyRelu", ["X"], ["Y"], **kwargs)

graph = helper.make_graph(
[node],
"qlinearRelu_test",
inputs=[helper.make_tensor_value_info("X", TensorProto.FLOAT, list(in_array.shape))],
outputs=[helper.make_tensor_value_info("Y", TensorProto.FLOAT, list(in_array.shape))],
)
model = helper.make_model(graph, producer_name="qlinearRelu_test")
quantize_and_verify_with_ort(model, ["X"], [in_array.shape], target, dev)

verify_qlinearleakyrelu([2, 4, 5, 6], {"alpha": 0.25})
verify_qlinearleakyrelu([6, 5, 6, 7], {"alpha": 0.35})
verify_qlinearleakyrelu([5, 1, 4, 6], {"alpha": 0.65})


@tvm.testing.parametrize_targets
def test_qlinearsigmoid(target, dev):
def verify_qlinearsigmoid(a_shape):
Expand Down