[BugFix][TOPI] Fix the integer overflow problem of the scatter_nd op. #8415
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I've implemented an alternative fix in #8419 |
@tkonolige Thank you for review this PR. Just curious, is there any specific reason why you created another PR? I understand the alternative fix you proposed maybe better than the existing one, in someway probably. But why bother create a new separate PR? We can discuss and improve the code here, I think that's exactly what code review is about, right? |
|
@zhuwenxi Sorry, I definitely shouldn't have opened a new PR. I just got a little hasty and did the fix myself. Feel free to copy the code from that PR into this one. I'll end up closing the other one. |
|
@tkonolige That's OK, it happens. |
|
You hit a flaky test, I found a fix here: #8431 |
mbrookhart
approved these changes
Jul 9, 2021
added 2 commits
July 12, 2021 11:26
1. Existing scatter_nd cuda implementation has a very large bound, which could overflow int32 range when input tensor shape is large enough; 2. The overflow could cause the if statement always evaluate to true, thus conducts invalid memory accesses; 3. We fix this problem in this commit by reducing the bound, the original large bound is not only unnecessary, but also degrading the performance; With this fix, scatter_op's performance improves 100x on some cases.
zhuwenxi
force-pushed
the
bugfix/wenxizhu/fix-scatter-integer-overflow
branch
from
842243b
to
cd30a24
Compare
|
@mbrookhart I've fixed the UT failure. |
|
Thanks @zhuwenxi @tkonolige |
ylc
pushed a commit
to ylc/tvm
that referenced
this pull request
Sep 29, 2021
…apache#8415) * Fix the integer overflow problem of the scatter_nd op. * Fix scatter_nd's crash problem: 1. Existing scatter_nd cuda implementation has a very large bound, which could overflow int32 range when input tensor shape is large enough; 2. The overflow could cause the if statement always evaluate to true, thus conducts invalid memory accesses; 3. We fix this problem in this commit by reducing the bound, the original large bound is not only unnecessary, but also degrading the performance; With this fix, scatter_op's performance improves 100x on some cases. Co-authored-by: wenxizhu <wenxizhu@tencent.com>
zxy844288792
pushed a commit
to zxy844288792/tvm
that referenced
this pull request
Mar 4, 2022
…apache#8415) * Fix the integer overflow problem of the scatter_nd op. * Fix scatter_nd's crash problem: 1. Existing scatter_nd cuda implementation has a very large bound, which could overflow int32 range when input tensor shape is large enough; 2. The overflow could cause the if statement always evaluate to true, thus conducts invalid memory accesses; 3. We fix this problem in this commit by reducing the bound, the original large bound is not only unnecessary, but also degrading the performance; With this fix, scatter_op's performance improves 100x on some cases. Co-authored-by: wenxizhu <wenxizhu@tencent.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem Statement
scatter_ndcrashes on cuda backend, when input data shape is slightly larger than usual.Code to reproduce
import tvm import numpy as np import tvm.relay as relay dev = tvm.cuda() target = tvm.target.Target("cuda") # input data: data_np = np.zeros((32, 128, 128, 256)).astype(np.float32) indices_np = np.random.uniform(1,5,(32, 600, 3)).astype(np.int64) updates_np = np.random.rand(32, 600, 256).astype(np.float32) # Construct relay input nodes: data = relay.var("data", shape=data_np.shape, dtype=str(data_np.dtype)) indices = relay.var("indices", shape=indices_np.shape, dtype=str(indices_np.dtype)) updates = relay.var("updates", shape=updates_np.shape, dtype=str(updates_np.dtype)) # Compute indices: indices_dim = len(indices_np.shape) axes = list(range(indices_dim)) indices_t = relay.transpose(indices, axes[-1:] + axes[:-1]) # Construct relay scatter_nd op: out = relay.op.scatter_nd(data, indices_t, updates, "update") func = relay.Function([data, indices, updates], out) # Execute scatter_nd: intrp = relay.create_executor("debug", device=dev, target=target) op_res = intrp.evaluate(func)(data_np, indices_np, updates_np)Error Message
Root Cause
We can see the problem more clearly from the cuda code generated. The TIR implementation of scatter_nd would cause a int32 overflow when "i" is large, thus the if statement is always evaluate to true, and conducts a invalid memory access.