[ONNX] Handle Gelu approximate attribute from Opset 20 by Mr-Neutr0n · Pull Request #18773 · apache/tvm

Mr-Neutr0n · 2026-02-13T09:33:35Z

Description

The ONNX frontend Gelu converter was hardcoded to always emit R.nn.gelu, ignoring the approximate attribute introduced in ONNX Opset 20. When a model uses approximate="tanh", TVM would silently produce incorrect results by applying the exact erf-based Gelu instead of the tanh approximation.

Changes

Added _impl_v20 to the Gelu converter class that checks the approximate attribute:

approximate="tanh" → routes to relax.op.nn.gelu_tanh
approximate="none" (default) → routes to relax.op.nn.gelu

The existing _impl_v1 is left untouched for backward compatibility with the Microsoft onnxruntime contrib opset. The version dispatch mechanism will automatically select _impl_v20 for Opset 20+ models.

Testing

The change is a straightforward attribute check that routes to an already-existing and tested operator (R.nn.gelu_tanh).
No new dependencies introduced.

The ONNX frontend Gelu converter was hardcoded to always emit R.nn.gelu, ignoring the `approximate` attribute introduced in ONNX Opset 20. When a model specifies approximate="tanh", TVM would produce incorrect results by using the exact erf-based Gelu instead of the tanh approximation. Add _impl_v20 that checks the `approximate` attribute and routes to R.nn.gelu_tanh when approximate="tanh", falling back to the standard R.nn.gelu otherwise. Fixes apache#18750 Signed-off-by: Mr-Neutr0n <64578610+Mr-Neutr0n@users.noreply.github.com>

gemini-code-assist · 2026-02-13T09:33:48Z

Summary of Changes

Hello @Mr-Neutr0n, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical bug in the ONNX frontend where the Gelu operator's approximate attribute, introduced in Opset 20, was not being respected. By implementing version-specific handling, TVM can now correctly interpret and apply the tanh approximation for Gelu when specified, preventing silent miscomputations and ensuring accurate model conversion from ONNX to Relax.

Highlights

ONNX Gelu Approximation Handling: The ONNX frontend for the Gelu operator now correctly handles the approximate attribute introduced in ONNX Opset 20, resolving an issue where approximate="tanh" was ignored.
New _impl_v20 for Gelu Converter: A new _impl_v20 method has been added to the Gelu converter class. This method checks the approximate attribute and routes to relax.op.nn.gelu_tanh when approximate="tanh" and relax.op.nn.gelu otherwise.
Backward Compatibility Maintained: The existing _impl_v1 method for the Gelu converter remains unchanged, ensuring backward compatibility for models using the Microsoft onnxruntime contrib opset.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

python/tvm/relax/frontend/onnx/onnx_frontend.py
- Added _impl_v20 method to the Gelu converter to handle the approximate attribute from ONNX Opset 20.
- Updated the Gelu class docstring to reflect support for both Microsoft onnxruntime contrib opset and ONNX Opset 20+.

Activity

No specific activity has been recorded for this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request correctly adds support for the approximate attribute in the ONNX Gelu operator for Opset 20. The implementation correctly dispatches to relax.op.nn.gelu_tanh when approximate="tanh".

I have one suggestion to make the implementation more robust by explicitly handling invalid values for the approximate attribute.

Additionally, while the change is straightforward, adding a dedicated test case to tests/python/relax/test_frontend_onnx.py would be beneficial to prevent future regressions. This test should cover both approximate="tanh" and approximate="none" for Opset 20.

gemini-code-assist · 2026-02-13T09:35:28Z

python/tvm/relax/frontend/onnx/onnx_frontend.py

+        approximate = attr.get("approximate", "none")
+        if approximate == "tanh":
+            return relax.op.nn.gelu_tanh(inputs[0])
+        return relax.op.nn.gelu(inputs[0])


While the current implementation correctly handles the 'tanh' approximation, it silently falls back to the default 'erf' implementation for any other value of approximate besides 'tanh'. According to the ONNX specification for Gelu, the only valid values for approximate are 'tanh' and 'none'.

To make the implementation more robust and explicit, it would be better to raise an error for any unsupported values. This aligns with how other operators with string attributes are handled in this file (e.g., BitShift).

Suggested change

approximate = attr.get("approximate", "none")

if approximate == "tanh":

return relax.op.nn.gelu_tanh(inputs[0])

return relax.op.nn.gelu(inputs[0])

approximate = attr.get("approximate", "none")

if approximate == "tanh":

return relax.op.nn.gelu_tanh(inputs[0])

elif approximate == "none":

return relax.op.nn.gelu(inputs[0])

else:

raise ValueError(f"Unsupported approximate mode for Gelu: {approximate}")

mshr-h

Please include testcase in

tvm/tests/python/relax/test_frontend_onnx.py

Line 922 in 84d05e9

def test_gelu():

Add test_gelu_approximate to verify that the Gelu converter correctly handles the approximate="tanh" and approximate="none" attributes introduced in ONNX Opset 20. Signed-off-by: Mr-Neutr0n <64578610+Mr-Neutr0n@users.noreply.github.com>

Mr-Neutr0n · 2026-02-13T13:53:43Z

Thanks for the review @mshr-h! I've added test_gelu_approximate in tests/python/relax/test_frontend_onnx.py covering both approximate="tanh" and approximate="none" for Opset 20.

mshr-h

Please fix the CI error. @Mr-Neutr0n

Mr-Neutr0n mentioned this pull request Feb 13, 2026

[Bug][Frontend][ONNX] Gelu operator misses support for 'approximate="tanh"' attribute #18750

Open

gemini-code-assist bot reviewed Feb 13, 2026

View reviewed changes

mshr-h requested changes Feb 13, 2026

View reviewed changes

This comment was marked as duplicate.

Sign in to view

mshr-h requested changes Feb 13, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ONNX] Handle Gelu approximate attribute from Opset 20#18773

[ONNX] Handle Gelu approximate attribute from Opset 20#18773
Mr-Neutr0n wants to merge 2 commits intoapache:mainfrom
Mr-Neutr0n:fix-onnx-gelu-approximate

Mr-Neutr0n commented Feb 13, 2026

Uh oh!

gemini-code-assist bot commented Feb 13, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Feb 13, 2026

Uh oh!

mshr-h left a comment •

edited

Loading

Uh oh!

Mr-Neutr0n commented Feb 13, 2026

Uh oh!

This comment was marked as duplicate.

Uh oh!

mshr-h left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Mr-Neutr0n commented Feb 13, 2026

Description

Changes

Testing

Uh oh!

gemini-code-assist bot commented Feb 13, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

mshr-h left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Mr-Neutr0n commented Feb 13, 2026

Uh oh!

This comment was marked as duplicate.

Uh oh!

mshr-h left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mshr-h left a comment •

edited

Loading