Packing for Gelu backwards #306
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JaneIllario
force-pushed
the
gelu_backwards
branch
from
68c16ab
to
71da2d2
Compare
|
This PR can't compare the previous kernel and the new one, and also isn't there a compile bug? x128 typedef doesn't exist. |
|
Sorry, I must've deleted the definition for x128 while cleaning up my branch. Pushed the correction now. |
JaneIllario
force-pushed
the
gelu_backwards
branch
from
2771d7e
to
09b313a
Compare
|
I updated the gelu_backward.cu to match the most recent changes in master for the other .cu files -- updating the other pr now |
ngc92
reviewed
May 1, 2024
ngc92
reviewed
May 1, 2024
ngc92
reviewed
May 1, 2024
JaneIllario
force-pushed
the
gelu_backwards
branch
from
95c59ea
to
3de3c53
Compare
ngc92
reviewed
May 2, 2024
| } | ||
|
|
||
| void gelu_backward2(floatX* dinp, const floatX* inp, const floatX* dout, int N, const int block_size) { | ||
| const int grid_size = ceil_div(N, block_size * x128::size); |
There was a problem hiding this comment.
maybe add an assert(N % x128::size == 0) here? documents the assumption and we may get a better error message in case we call the kernel wrongly later
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update gelu backwards kernel to do packing into 128 bits, and create gelu brackward cuda file
Previous kernel:
block_size 32 | time 0.1498 ms | bandwidth 503.99 GB/s
block_size 64 | time 0.0760 ms | bandwidth 993.32 GB/s
block_size 128 | time 0.0490 ms | bandwidth 1540.78 GB/s
block_size 256 | time 0.0487 ms | bandwidth 1548.88 GB/s
block_size 512 | time 0.0487 ms | bandwidth 1548.88 GB/s
block_size 1024 | time 0.0497 ms | bandwidth 1518.38 GB/s
total average iteration time: 39.030942 ms
New Kernel
block_size 32 | time 0.0328 ms | bandwidth 1535.18 GB/s
block_size 64 | time 0.0319 ms | bandwidth 1575.59 GB/s
block_size 128 | time 0.0333 ms | bandwidth 1509.35 GB/s
block_size 256 | time 0.0337 ms | bandwidth 1491.94 GB/s
block_size 512 | time 0.0340 ms | bandwidth 1478.92 GB/s
block_size 1024 | time 0.0352 ms | bandwidth 1430.92 GB/s
total average iteration time: 38.145030 ms