Display human readable text for AArch64 vendors #13825

alalazo · 2019-11-21T20:46:41Z

Refers to #13780

Also fixes a couple of typos in GCC flags.

alalazo · 2019-11-21T20:48:24Z

Current output:

$ spack arch --known-targets 
Generic architectures (families)
    aarch64  arm  ppc  ppc64  ppc64le  ppcle  sparc  sparc64  x86  x86_64

GenuineIntel - x86
    i686  pentium2  pentium3  pentium4  prescott

GenuineIntel - x86_64
    nocona  core2  nehalem  westmere  sandybridge  ivybridge  haswell  broadwell  skylake  mic_knl  skylake_avx512  cannonlake  cascadelake  icelake

AuthenticAMD - x86_64
    k10  bulldozer  zen  piledriver  zen2  steamroller  excavator

IBM - ppc64
    power7  power8  power9

IBM - ppc64le
    power8le  power9le

Cavium - aarch64
    thunderx2

Fujitsu - aarch64
    a64fx

@tkameyama Can you give a look at this PR when you have some time?

t-karatsu · 2019-11-22T01:05:25Z

Sorry for the delay in submitting the pull request due to some matters under consideration... Thank you @alalazo. And I will Correct the typo that was pointed out.

lib/spack/llnl/util/cpu/microarchitectures.json

alalazo · 2019-11-22T05:57:18Z

Sorry for the delay in submitting the pull request due to some matters under consideration...

No worries, I had some time so happy to help keep things in order. Thanks for the review!

tkameyama

Maybe the original source of implementer id is Arm Architecture Reference Manual Armv8, for Armv8-A architecture profile https://developer.arm.com/docs/ddi0487/latest/arm-architecture-reference-manual-armv8-for-armv8-a-architecture-profile D13.2.87 MIDR_EL1, Main ID Register (With version ea, page 3213)

tldahlgren · 2019-11-22T17:01:45Z

Maybe the original source of implementer id is Arm Architecture Reference Manual Armv8, for Armv8-A architecture profile https://developer.arm.com/docs/ddi0487/latest/arm-architecture-reference-manual-armv8-for-armv8-a-architecture-profile D13.2.87 MIDR_EL1, Main ID Register (With version ea, page 3213)

FWIW. The lists don't match though. That table has two entries not included here (i.e., 0xC0=Ampere, 0x4D=Motorola (or Freescale)) while the list here includes entries not in that table (i.e., 0x48=HiSilicon, 0x66=Faraday).

Another partial list can be found for gcc at https://github.com/gcc-mirror/gcc/blob/master/gcc/config/aarch64/aarch64-cores.def, which includes: 0x68=HXT (not here) and 0x48=HiSilicon (here).

The partial list at https://patchwork.kernel.org/patch/10524949/ indicates 0x61=Apple (not here).

alalazo · 2019-11-26T12:35:02Z

@tkameyama @t-karatsu Can you please have a look at the latest updates to clang in 31f6286 and 9ce0b6b and double check the features enabled?

alalazo · 2019-11-26T13:31:31Z

This is ready for a further review.

lib/spack/llnl/util/cpu/detect.py

tldahlgren

Thanks for adding the extra vendor ids.

This LGTM though I have not double-checked any of the flags under the assumption you've been getting that feedback from others.

tgamblin

@alalazo: LGTM except for a minor request -- I think all the mappings should go in the JSON, so that someone can easily make another set of bindings for the archspec library. Basically, if it's knowledge gleaned from somewhere, we should consolidate it in one place: microarchitectures.json

lib/spack/llnl/util/cpu/detect.py

t-karatsu · 2019-11-27T06:00:39Z

lib/spack/llnl/util/cpu/microarchitectures.json

+          {
+            "versions": "9:",
+            "family": "aarch64",
+            "flags": "-march={family} -mcpu=thunderx2t99"
+          }
+        ]


Thanks @alalazo, but these options raise errors following:

[ogura@langtx202 bin]$ ./clang test.c -march=aarch64 -mattr=+v8.1a,+fp-armv8,+neon,+crc,+crypto clang-8: error: unknown argument: '-mattr=+v8.1a,+fp-armv8,+neon,+crc,+crypto' clang-8: error: the clang compiler does not support '-march=aarch64'

So, options for llc and clang seem different. I considered options for clang compiler commands. Currently I will Check whether it works normally.

diff --git a/lib/spack/llnl/util/cpu/microarchitectures.json b/lib/spack/llnl/util/cpu/microarchitectures.json index 3b58899..e37327a 100644 --- a/lib/spack/llnl/util/cpu/microarchitectures.json +++ b/lib/spack/llnl/util/cpu/microarchitectures.json @@ -1196,10 +1196,16 @@ "flags": "-mcpu=thunderx2t99" } ], - "clang": { - "versions": ":", - "flags": "-march=armv8-a -mcpu=generic" - } + "clang": [ + { + "versions": "3.9:", + "flags": "-march=armv8.1-a+crc+crypto" + }, + { + "versions": "5.0:", + "flags": "-mcpu=thunderx2t99" + } + ] } }, "a64fx": { @@ -1246,10 +1252,16 @@ "flags": "-arch=armv8.2a+crc+aes+sh2+fp16+sve -msve-vector-bits=512" } ], - "clang": { - "versions": ":", - "flags": "-march=armv8-a -mcpu=generic" - } + "clang": [ + { + "versions": "3.9:", + "flags": "-march=armv8.2-a+crc+crypto+fp16" + }, + { + "versions": "5.0:", + "flags": "-march=armv8.2-a+crc+crypto+fp16+sve" + } + ] } }, "arm": {

I am sorry if there is a lack of consideration here.

Thanks @t-karatsu. You're completely correct. I think we need to revisit also the flags that are passed to x86-64. I tried to disassemble some vectorizable code and it seems clang does not give the same meaning to flags that are also used by its assembler. I'll try to see if I can find some reference for it and eventually post here what I'll find. @tgamblin FYI

For reference, when I researched this I couldn't find a good source of information. Even clang manuals suggest options that are either not recognized on command line or don't work as described. The best source of information I could find at the time was this SO answer.

@becker33 I think you might also be interested in the discussion above.

Further update: I found this discussion on how clang opt and llc relate to each other (and goes on a 4 stage manual build to use them). There's also this other question that looks exactly what we need, but has no answer.

I'll continue looking, in the meanwhile I posted a message to cfe users - let's see if something comes out from there.

For reference, when I researched this I couldn't find a good source of information.

How about clang preprocessor test code?
https://github.com/llvm/llvm-project/blob/master/clang/test/Preprocessor/aarch64-target-features.c

@tkameyama @t-karatsu I got access today to a thunderx2 machine. Will try out the flags there.

How about clang preprocessor test code?

That should work thanks. The front-end driver seems consistent with GCC in terms of flags which is also what is briefly written here. Will update this PR asap and ping you when done.

lib/spack/llnl/util/cpu/microarchitectures.json

Co-Authored-By: t-karatsu <49965247+t-karatsu@users.noreply.github.com>

alalazo · 2019-11-29T14:37:45Z

@t-karatsu @tkameyama @tgamblin PR ready for another review

t-karatsu · 2019-12-02T06:57:33Z

Thank you grateful @alalazo ! I operated in spack with this code, it worked very good.

[tx202 spack]$ spack arch
linux-centos7-thunderx2
[tx202 spack]$  spack arch --known-targets
Generic architectures (families)
    aarch64  arm  ppc  ppc64  ppc64le  ppcle  sparc  sparc64  x86  x86_64
...
Fujitsu - aarch64
    a64fx
...
Cavium - aarch64
    thunderx2

env in installed OSS:
- gcc@4.8.5
    SPACK_TARGET_ARGS=-march=armv8-a; export SPACK_TARGET_ARGS
- gcc@9.1.0
    SPACK_TARGET_ARGS=-mcpu=thunderx2t99; export SPACK_TARGET_ARGS

But please tell me about one code.

lib/spack/llnl/util/cpu/detect.py

alalazo · 2019-12-11T10:50:31Z

@tgamblin ping

alalazo added platform-support arm microarchitectures labels Nov 21, 2019

alalazo requested review from tgamblin, scheibelp, becker33 and tldahlgren November 21, 2019 20:46

t-karatsu reviewed Nov 22, 2019

View reviewed changes

lib/spack/llnl/util/cpu/microarchitectures.json Outdated Show resolved Hide resolved

t-karatsu reviewed Nov 22, 2019

View reviewed changes

lib/spack/llnl/util/cpu/microarchitectures.json Outdated Show resolved Hide resolved

tkameyama reviewed Nov 22, 2019

View reviewed changes

alalazo added 4 commits November 26, 2019 13:32

Display human readable text for AArch64 vendors

16af198

Added further vendors to translation table

53f4713

Thunderx2: added optimization flags for clang

31f6286

A64FX: added optimization flags for clang, indented file consistently

9ce0b6b

alalazo force-pushed the fixes/vendor_on_arm branch from a7d1df7 to 88b7101 Compare November 26, 2019 12:32

Updated unit tests to fit the new optimization flags

27922fb

alalazo force-pushed the fixes/vendor_on_arm branch from 88b7101 to 27922fb Compare November 26, 2019 12:40

tldahlgren reviewed Nov 26, 2019

View reviewed changes

lib/spack/llnl/util/cpu/detect.py Show resolved Hide resolved

tldahlgren approved these changes Nov 26, 2019

View reviewed changes

tgamblin approved these changes Nov 26, 2019

View reviewed changes

tgamblin requested changes Nov 26, 2019

View reviewed changes

lib/spack/llnl/util/cpu/detect.py Outdated Show resolved Hide resolved

lib/spack/llnl/util/cpu/detect.py Outdated Show resolved Hide resolved

t-karatsu reviewed Nov 27, 2019

View reviewed changes

lib/spack/llnl/util/cpu/microarchitectures.json Outdated Show resolved Hide resolved

t-karatsu reviewed Nov 27, 2019

View reviewed changes

lib/spack/llnl/util/cpu/microarchitectures.json Outdated Show resolved Hide resolved

Fix typo when using -march

a7cd81b

Co-Authored-By: t-karatsu <49965247+t-karatsu@users.noreply.github.com>

alalazo mentioned this pull request Nov 27, 2019

Fixed x86-64 optimization flags for clang #13913

Merged

alalazo added 5 commits November 29, 2019 13:37

Fixed Clang flags for AArch64 chips

99d9505

Moved vendor information into microarchitectures.json

16f0c59

Fixup cpu test

141e2fd

Moved Darwin specific information into microarchitectures.json

1bc682c

Removed unneeded entry in JSON file

b3a80fb

t-karatsu reviewed Dec 2, 2019

View reviewed changes

lib/spack/llnl/util/cpu/detect.py Outdated Show resolved Hide resolved

Removed leftover line from llnl.util.cpu.detect

dde6df4

tkameyama reviewed Dec 2, 2019

View reviewed changes

lib/spack/llnl/util/cpu/detect.py Outdated Show resolved Hide resolved

tkameyama reviewed Dec 2, 2019

View reviewed changes

lib/spack/llnl/util/cpu/detect.py Outdated Show resolved Hide resolved

Fixed dictionary lookup in llnl.util.cpu.detect

c7e83b5

tgamblin added this to Todo in Spack v0.14.0 release via automation Dec 3, 2019

Renamed 'miscellaneous' to 'conversions' in the JSON file

3cf72fc

tgamblin approved these changes Dec 16, 2019

View reviewed changes

Spack v0.14.0 release automation moved this from Todo to In review Dec 16, 2019

tgamblin merged commit 5eca4f1 into spack:develop Dec 17, 2019

Spack v0.14.0 release automation moved this from In review to Done Dec 17, 2019

alalazo deleted the fixes/vendor_on_arm branch December 17, 2019 08:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Display human readable text for AArch64 vendors #13825

Display human readable text for AArch64 vendors #13825

alalazo commented Nov 21, 2019

alalazo commented Nov 21, 2019

t-karatsu commented Nov 22, 2019 •

edited

alalazo commented Nov 22, 2019

tkameyama left a comment

tldahlgren commented Nov 22, 2019 •

edited

alalazo commented Nov 26, 2019

alalazo commented Nov 26, 2019

tldahlgren left a comment

tgamblin left a comment

t-karatsu Nov 27, 2019 •

edited

alalazo Nov 27, 2019

alalazo Nov 27, 2019

alalazo Nov 27, 2019

alalazo Nov 27, 2019

tkameyama Nov 28, 2019

alalazo Nov 28, 2019

alalazo commented Nov 29, 2019

t-karatsu commented Dec 2, 2019 •

edited

alalazo commented Dec 11, 2019

Display human readable text for AArch64 vendors #13825

Display human readable text for AArch64 vendors #13825

Conversation

alalazo commented Nov 21, 2019

alalazo commented Nov 21, 2019

t-karatsu commented Nov 22, 2019 • edited

alalazo commented Nov 22, 2019

tkameyama left a comment

Choose a reason for hiding this comment

tldahlgren commented Nov 22, 2019 • edited

alalazo commented Nov 26, 2019

alalazo commented Nov 26, 2019

tldahlgren left a comment

Choose a reason for hiding this comment

tgamblin left a comment

Choose a reason for hiding this comment

t-karatsu Nov 27, 2019 • edited

Choose a reason for hiding this comment

alalazo Nov 27, 2019

Choose a reason for hiding this comment

alalazo Nov 27, 2019

Choose a reason for hiding this comment

alalazo Nov 27, 2019

Choose a reason for hiding this comment

alalazo Nov 27, 2019

Choose a reason for hiding this comment

tkameyama Nov 28, 2019

Choose a reason for hiding this comment

alalazo Nov 28, 2019

Choose a reason for hiding this comment

alalazo commented Nov 29, 2019

t-karatsu commented Dec 2, 2019 • edited

alalazo commented Dec 11, 2019

t-karatsu commented Nov 22, 2019 •

edited

tldahlgren commented Nov 22, 2019 •

edited

t-karatsu Nov 27, 2019 •

edited

t-karatsu commented Dec 2, 2019 •

edited