Added scaffolding for Oryon arch as in Snapdragon X Elite #5537
+107
−7
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Using NeoverseN1 Kernels for now with cache info taken from official specs.
Collaborator
|
Thanks - do you get markedly better performance with this change, compared to the default approach in 0.3.30 of autodetecting this cpu as a regular NEOVERSEN1 ? I would prefer to avoid the code and library size explosion from adding any and all arm64 design variant, so unless the exact model-specific cost tables make a serious difference to the compiler output I'd like to avoid mere duplication. |
Contributor
Author
|
I need to do some benchmarking, so I'll report back on that. I have to imagine the significant difference in cache layout here is going to do something. |
Contributor
Author
|
Extremely unscientific runthrough: Stock Rblas.dll: OpenBLAS 3.30.0.dev: OpenBLAS NeoverseN1 kernel w/ oryon cache sizes: |
Contributor
Author
|
Gonna be completely honest here-I can't quite tell. Looks like there's some sizes for which it performs better and some for which it is worse. Any recs for drilling down a bit deeper? edit: just saw the openblas_loops setting, bear with |
Contributor
Author
|
3.30.0dev Oryon-modded cache size |
Contributor
Author
|
I think there's definitely something here, judging by the decent improvement at certain matrix sizes, but this is not it judging by the degraded performance at other matrix sizes. May be worth having it as a full clone of neoverse n1 (ie-removing the cache changes i made here) pending further investigation. |
Contributor
Author
|
.....I had an idea. This is an 8-wide chip, neoverse is 5-wide. I wonder what happens if i run the VORTEX target (which is 7-wide and should be otherwise compatible. Because I get the feeling the optimization here isn't so much in the cache definitons as much as its in the kernels. |
Contributor
Author
|
Scratch that, it would do nothing, as there's no difference. |
Collaborator
|
Yes, right now VORTEX is also just ARMV8 with a bunch of NEOVERSEN1 kernels on top. Without dedicated kernels, I think the easiest fix would be to put the proper L1 and L2 cache sizes in cpuid_arm64.c when we're on Windows, to guide the block sizes for GEMM etc. |
Contributor
Author
|
Yeah-and even if there is optimization here (and there almost certanily is) I don't even know that the cache sizes are an improvement. |
Collaborator
|
Probably needs larger loops to get more stable benchmark results. I do have an Oryon system on loan from Qualcomm, it's just that I'm away from it at the moment but I'll try to run some experiments myself when I have more time for OpenBLAS again - hopefully soon |
Contributor
Author
Oh, I can absolutely just run them on my laptop. How large are we talking? |
Collaborator
|
I'd guess a hundred instead of ten should help |
Contributor
Author
Will report back. With bonus ArmPL for comparison. |
Contributor
Author
|
So, it turns out the issue was mostly that running BLAS on 12 cores well exceeds the heat capacity of my laptop. Fixed that one. Anyway: Seems that there's a thousand to a few thousand megaflops difference in favor of the cache-tuned build at all sizes, which is more what I would have expected. Funnily enough, ArmPL seems to be on par with the n1 build and similarly behind the tuned build. Guess that does make sense, they did optimize for their own cores. Do we know if QC has an optimized implementation? N1 Oryon ArmPL for comparison |
Collaborator
|
Hmm. I'm still not that convinced - looks like there is still a lot of noise in the data, and where it looks like there is an improvement from using the correct cache sizes, it is around 2 percent at most ? |
Contributor
Author
By noise do you mean the fluctuating MFlops as size increases? That's actually fairly reproducible. And yes, around 2%. I think the bottleneck here isn't so much cache locality as much as it is the difference in execution pipeline size (5-wide vs 8-wide). edit: looking at the block diagrams it appears the correct way of looking at it is 2 NEON/FP units on the N1 and 4 on oryon |
Contributor
|
Hi @theAeon Out of curiosity, are you going to add/modifying/optimize any kernels for this arch in future? |
Contributor
Author
|
unfortunately this is not exactly my strong suit, so while I will take a look i am...not expecting to, no. |
Collaborator
|
I can add a small hack to the cpu detection code to put the correct cache sizes in the config file, as that bit of performance gain it is low-hanging (if fairly small) fruit. But frankly I expect the upcoming X2 Elite cpu with its SVE+SME capability to be a markedly more attractive platform for any kind of numerical workload, and it should be quite adequately covered by the ARMV9SME target already. |
Contributor
Author
|
That sounds like the way to go. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
All I know is that this builds and works fine with clangarm64 on my laptop. Unsure about performance improvement, but certainly no performance regression.
I am not an assembly wizard, so this still uses the neoverse kernels. I imagine there is much optimization to be had. Feel free to edit if I missed a spot.
https://www.hwcooling.net/en/oryon-arm-core-in-snapdragon-x-cpus-architecture-analysis/ for cache reference