Added packing for gelu forwards kernel #301
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements packing for the Gelu forwards kernel using the example provided. The kernel dev file was also updated to show the impact of changing the data types for floatX.
Before changes:
total average iteration time: 38.480425 ms
After changes:
total average iteration time: 37.817789 ms
Before:
block_size 32 | time 0.1502 ms | bandwidth 335.17 GB/s
block_size 64 | time 0.0761 ms | bandwidth 661.65 GB/s
block_size 128 | time 0.0404 ms | bandwidth 1246.72 GB/s
block_size 256 | time 0.0376 ms | bandwidth 1337.40 GB/s
block_size 512 | time 0.0389 ms | bandwidth 1294.26 GB/s
block_size 1024 | time 0.0407 ms | bandwidth 1235.78 GB/s
After:
block_size 32 | time 0.0225 ms | bandwidth 2232.26 GB/s
block_size 64 | time 0.0198 ms | bandwidth 2547.13 GB/s
block_size 128 | time 0.0198 ms | bandwidth 2544.23 GB/s
block_size 256 | time 0.0201 ms | bandwidth 2508.65 GB/s
block_size 512 | time 0.0207 ms | bandwidth 2433.87 GB/s
block_size 1024 | time 0.0218 ms | bandwidth 2308.04 GB/s