Skip to content

[Unity][CUTLASS] Require the residual input to have the same shape as input#14657

Merged
vinx13 merged 1 commit intoapache:unityfrom
masahi:cutlass-residual-fix
Apr 19, 2023
Merged

[Unity][CUTLASS] Require the residual input to have the same shape as input#14657
vinx13 merged 1 commit intoapache:unityfrom
masahi:cutlass-residual-fix

Conversation

@masahi
Copy link
Member

@masahi masahi commented Apr 18, 2023

In SD UNet, I met a pattern conv2d -> bias_add -> residual_add where the input shape is (2, 16, 16, 1280) and the residual input shape is (2, 1, 1, 1280). Such general broadcast add for the residual block is not supported by cutlass.

This was the cause of nan output when using the cutlass BYOC with an improved AMP pass (that allows bias add to be done in fp16, PR later).

@vinx13 @yelite @spectrometerHBH

@tvm-bot
Copy link
Collaborator

tvm-bot commented Apr 18, 2023

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

Generated by tvm-bot

@vinx13 vinx13 merged commit f7835a6 into apache:unity Apr 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants