Add IFRT C-API bindings #178
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Closed
10 tasks
Contributor
Benchmark Results
Benchmark PlotsA plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Reactant.jl Benchmarks
| Benchmark suite | Current: 8bbe74b | Previous: 45ae14f | Ratio |
|---|---|---|---|
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :after_enzyme) |
1261356777 ns |
1287700343 ns |
0.98 |
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Reactant |
1269242739 ns |
1271515659 ns |
1.00 |
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :before_enzyme) |
1178720422 ns |
1253394269 ns |
0.94 |
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :only_enzyme) |
3021590584 ns |
3106663633 ns |
0.97 |
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Lux |
202043185 ns |
217499591 ns |
0.93 |
ViT base (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :after_enzyme) |
5280120268 ns |
6749076193 ns |
0.78 |
ViT base (256 x 256 x 3 x 32)/forward/CPU/Reactant |
5435033235 ns |
5078740247 ns |
1.07 |
ViT base (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :before_enzyme) |
5220761744 ns |
5013817961 ns |
1.04 |
ViT base (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :only_enzyme) |
7350408625 ns |
7197691815 ns |
1.02 |
ViT base (256 x 256 x 3 x 32)/forward/CPU/Lux |
32082362874 ns |
35464964244 ns |
0.90 |
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :after_enzyme) |
1273442646 ns |
1257317145 ns |
1.01 |
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Reactant |
1259668677 ns |
1424374803 ns |
0.88 |
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :before_enzyme) |
1382478970 ns |
1350049098 ns |
1.02 |
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :only_enzyme) |
3078972719 ns |
3052800629 ns |
1.01 |
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Lux |
8892386 ns |
8862682 ns |
1.00 |
ViT small (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :after_enzyme) |
1559714397 ns |
1572590140 ns |
0.99 |
ViT small (256 x 256 x 3 x 4)/forward/CPU/Reactant |
1547215661 ns |
1559474266 ns |
0.99 |
ViT small (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :before_enzyme) |
1534197630 ns |
1557501067 ns |
0.99 |
ViT small (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :only_enzyme) |
3290194465 ns |
3290628669 ns |
1.00 |
ViT small (256 x 256 x 3 x 4)/forward/CPU/Lux |
3004636060 ns |
2876354148 ns |
1.04 |
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :after_enzyme) |
1239941477 ns |
1231219515 ns |
1.01 |
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Reactant |
1348128765 ns |
1441159242 ns |
0.94 |
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :before_enzyme) |
1360430845.5 ns |
1282010253 ns |
1.06 |
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :only_enzyme) |
3061759665 ns |
3051584957 ns |
1.00 |
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Lux |
22715871.5 ns |
22776746 ns |
1.00 |
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :after_enzyme) |
2140449540 ns |
2154505585 ns |
0.99 |
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Reactant |
2162802449 ns |
2139776302 ns |
1.01 |
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :before_enzyme) |
2192061646 ns |
2123332313 ns |
1.03 |
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :only_enzyme) |
3951839063 ns |
3879039560 ns |
1.02 |
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Lux |
5864564773 ns |
5729200009 ns |
1.02 |
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :after_enzyme) |
1205684124.5 ns |
1259798635 ns |
0.96 |
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Reactant |
1361293530.5 ns |
1262851193 ns |
1.08 |
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :before_enzyme) |
1258000955.5 ns |
1266665882 ns |
0.99 |
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :only_enzyme) |
3018719976 ns |
3319553871 ns |
0.91 |
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Lux |
7130550 ns |
7445203.5 ns |
0.96 |
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :after_enzyme) |
1497472070 ns |
1424258021 ns |
1.05 |
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Reactant |
1437570293 ns |
1421721118 ns |
1.01 |
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :before_enzyme) |
1436486513 ns |
1420742881 ns |
1.01 |
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :only_enzyme) |
3209038549 ns |
3162578762 ns |
1.01 |
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Lux |
1247478583 ns |
2138106366 ns |
0.58 |
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :after_enzyme) |
1253502094.5 ns |
1297050944 ns |
0.97 |
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Reactant |
1209056759.5 ns |
1403907055 ns |
0.86 |
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :before_enzyme) |
1217892910 ns |
1269229731 ns |
0.96 |
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :only_enzyme) |
3022827150 ns |
3063143344 ns |
0.99 |
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Lux |
12327361.5 ns |
12347497 ns |
1.00 |
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :after_enzyme) |
1744121564 ns |
1721006513 ns |
1.01 |
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Reactant |
1735401365 ns |
1711405549 ns |
1.01 |
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :before_enzyme) |
1733120776 ns |
1704835369 ns |
1.02 |
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :only_enzyme) |
3558534960 ns |
3443971150 ns |
1.03 |
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Lux |
3207629038 ns |
3110298785 ns |
1.03 |
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :after_enzyme) |
1179647754 ns |
1266729302 ns |
0.93 |
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Reactant |
1219054760 ns |
1308873395 ns |
0.93 |
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :before_enzyme) |
1175591863.5 ns |
1275958493 ns |
0.92 |
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :only_enzyme) |
2790920121 ns |
3081413477 ns |
0.91 |
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Lux |
27427973 ns |
27435162 ns |
1.00 |
ViT small (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :after_enzyme) |
2219937951 ns |
2169947879 ns |
1.02 |
ViT small (256 x 256 x 3 x 16)/forward/CPU/Reactant |
2246920348 ns |
2163945294 ns |
1.04 |
ViT small (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :before_enzyme) |
2201111273 ns |
2151891950 ns |
1.02 |
ViT small (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :only_enzyme) |
4021981613 ns |
3946269320 ns |
1.02 |
ViT small (256 x 256 x 3 x 16)/forward/CPU/Lux |
7405080628.5 ns |
6287057122 ns |
1.18 |
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :after_enzyme) |
1444979179 ns |
1260705673 ns |
1.15 |
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Reactant |
1228694889 ns |
1369717954 ns |
0.90 |
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :before_enzyme) |
1237910656 ns |
1281076652 ns |
0.97 |
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :only_enzyme) |
3005959850 ns |
3130042297 ns |
0.96 |
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Lux |
53077508 ns |
53036705.5 ns |
1.00 |
ViT small (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :after_enzyme) |
3042874784 ns |
3050356994 ns |
1.00 |
ViT small (256 x 256 x 3 x 32)/forward/CPU/Reactant |
3037476985 ns |
3082997102 ns |
0.99 |
ViT small (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :before_enzyme) |
3055162917 ns |
2965563203 ns |
1.03 |
ViT small (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :only_enzyme) |
4992480792 ns |
4841087626 ns |
1.03 |
ViT small (256 x 256 x 3 x 32)/forward/CPU/Lux |
8258150245 ns |
8484129480 ns |
0.97 |
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :after_enzyme) |
1268979030 ns |
1260921375 ns |
1.01 |
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Reactant |
1263972151 ns |
1253872568 ns |
1.01 |
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :before_enzyme) |
1288602384 ns |
1479498539 ns |
0.87 |
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :only_enzyme) |
2824720527 ns |
3113671601 ns |
0.91 |
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Lux |
71424467 ns |
71338519.5 ns |
1.00 |
ViT base (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :after_enzyme) |
3182230685 ns |
3125511597 ns |
1.02 |
ViT base (256 x 256 x 3 x 16)/forward/CPU/Reactant |
3248216282 ns |
3098530069 ns |
1.05 |
ViT base (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :before_enzyme) |
3255490728 ns |
3115589553 ns |
1.04 |
ViT base (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :only_enzyme) |
5267807294 ns |
5036626230 ns |
1.05 |
ViT base (256 x 256 x 3 x 16)/forward/CPU/Lux |
16603406947 ns |
11289651474 ns |
1.47 |
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :after_enzyme) |
1321419610 ns |
1339569725 ns |
0.99 |
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Reactant |
1236302169 ns |
1259019883 ns |
0.98 |
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :before_enzyme) |
1278941160 ns |
1254828379 ns |
1.02 |
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :only_enzyme) |
2870930704 ns |
2975337456 ns |
0.96 |
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Lux |
20770263.5 ns |
20758936 ns |
1.00 |
ViT base (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :after_enzyme) |
1906488091 ns |
1859519475 ns |
1.03 |
ViT base (256 x 256 x 3 x 4)/forward/CPU/Reactant |
2112347994 ns |
1869845638 ns |
1.13 |
ViT base (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :before_enzyme) |
2063244549 ns |
1850101657 ns |
1.12 |
ViT base (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :only_enzyme) |
3789248228 ns |
3593739548 ns |
1.05 |
ViT base (256 x 256 x 3 x 4)/forward/CPU/Lux |
4399685443 ns |
3325189113.5 ns |
1.32 |
This comment was automatically generated by workflow using github-action-benchmark.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #178 +/- ##
==========================================
- Coverage 26.37% 23.00% -3.38%
==========================================
Files 32 46 +14
Lines 4751 8056 +3305
==========================================
+ Hits 1253 1853 +600
- Misses 3498 6203 +2705 ☔ View full report in Codecov by Sentry. |
mofeing
force-pushed
the
mofeing/issue176
branch
from
8ac9faa to
abfe541
Compare
This was referenced Oct 21, 2024
mofeing
force-pushed
the
mofeing/issue176
branch
from
2304615 to
cbc4787
Compare
wsmoses
reviewed
Dec 1, 2024
wsmoses
approved these changes
Dec 1, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.