OpenBLAS-0.3.7, Compile problems with PGI 19.4

[qianfoling]

CPU: AMD EPYC 7601
OS: CentOS 7.6
Compiler：PGI 19.4

Problem 1

[root@n212 OpenBLAS-0.3.7]# make CC=pgcc FC=pgf90
pgcc-Error-Unknown switch: -march=native
make: *** [getarch] Error 1
Makefile.system:216: Makefile.conf: No such file or directory
make: *** No rule to make target `Makefile.conf'. Stop.
[root@n212 OpenBLAS-0.3.7]#

Problem 2

First, modify Makefile.system 148 line:
146 ifeq ($(ARCH), x86_64)
147 ifneq ($(C_COMPILER), PGI)
148 #GETARCH_FLAGS += -march=native
149 endif
150 endif

And,
[root@n212 OpenBLAS-0.3.7]# make CC=pgcc FC=pgf90 1>make.log 2>make.err
[root@n212 OpenBLAS-0.3.7]# tail -n 3 make.log
pgcc -c -O2 -DMAX_STACK_ALLOC=2048 -tp p7-64 -DF_INTERFACE_PGI -fPIC -DNO_AVX512 -DSMP_SERVER -DNO_WARMUP -DMAX_CPU_NUMBER=32 -DMAX_PARALLEL_NUMBER=1 -DVERSION="0.3.7" -DASMNAME=dtrmm_outucopy -DASMFNAME=dtrmm_outucopy_ -DNAME=dtrmm_outucopy_ -DCNAME=dtrmm_outucopy -DCHAR_NAME="dtrmm_outucopy_" -DCHAR_CNAME="dtrmm_outucopy" -DNO_AFFINITY -I.. -DDOUBLE -UCOMPLEX -DDOUBLE -UCOMPLEX -DOUTER -ULOWER -DUNIT generic/trmm_utcopy_8.c -o dtrmm_outucopy.o
pgcc -c -O2 -DMAX_STACK_ALLOC=2048 -tp p7-64 -DF_INTERFACE_PGI -fPIC -DNO_AVX512 -DSMP_SERVER -DNO_WARMUP -DMAX_CPU_NUMBER=32 -DMAX_PARALLEL_NUMBER=1 -DVERSION="0.3.7" -DASMNAME=dtrmm_outncopy -DASMFNAME=dtrmm_outncopy_ -DNAME=dtrmm_outncopy_ -DCNAME=dtrmm_outncopy -DCHAR_NAME="dtrmm_outncopy_" -DCHAR_CNAME="dtrmm_outncopy" -DNO_AFFINITY -I.. -DDOUBLE -UCOMPLEX -DDOUBLE -UCOMPLEX -DOUTER -ULOWER -UUNIT generic/trmm_utcopy_8.c -o dtrmm_outncopy.o
make[1]: Leaving directory `/root/src/OpenBLAS-0.3.7/kernel'
[root@n212 OpenBLAS-0.3.7]#
[root@n212 OpenBLAS-0.3.7]# cat -n make.err
...
PREPRO-I-0222-Redundant definition for symbol extension (../kernel/x86_64/dgemm_kernel_4x8_haswell.S: -1)
PREPRO-I-0222-Redundant definition for symbol extension (../kernel/x86_64/gemm_beta.S: -1)
:2:13: error: invalid operand for instruction
addq $128, %dl
^~~
:5:15: error: invalid base+index expression
vmulpd -128(%dl), %xmm0, %xmm4
^
:6:15: error: invalid base+index expression
vmulpd -112(%dl), %xmm0, %xmm5
^
:7:15: error: invalid base+index expression
vmulpd -96(%dl), %xmm0, %xmm6
^
:8:15: error: invalid base+index expression
vmulpd -80(%dl), %xmm0, %xmm7
^
:9:15: error: invalid base+index expression
vmulpd -64(%dl), %xmm0, %xmm8
^
...
[root@n212 OpenBLAS-0.3.7]#

[martin-frbg]

Ah, sorry - I see now that the line 147 ifneq ($(C_COMPILER), PGI) can not work (C_COMPILER is only set by c_check, which is run at the same - later - time as getarch). Will probably need to check for HOSTCC matching "pgcc" instead.
Could you please check if the assembler errors go away when you change the "-tp p7-64" in Makefile.system around line 700 to "-tp haswell" ?

[martin-frbg]

Actually it is probably best to simply change these -tp options to the same simple -m64 (or -m32 as appropriate) as used on most other compilers, leaving the PGI compiler to use its own defaults for the cpu-specific options. At least the "nollvm" version of PGI 19.4 certainly understands the -m64, the previous options must have been necessary for some much earlier version, ten or more years ago.
Unfortunately I cannot get rid of the assembler errors when I use the default llvm-based compiler,
looks like it has serious problems with inline assembly. When I add -Mnollvm to the compile options, it builds correctly but cannot run the openblas_utest binary due to a segmentation fault
in the startup code of libpgpmp.so. When I point CC and FC at the binaries in the linux-x86_64-nollvm/bin/ branch directly, everything works as normal.

[qianfoling]

Ah, sorry - I see now that the line 147 ifneq ($(C_COMPILER), PGI) can not work (C_COMPILER is only set by c_check, which is run at the same - later - time as getarch). Will probably need to check for HOSTCC matching "pgcc" instead.
Could you please check if the assembler errors go away when you change the "-tp p7-64" in Makefile.system around line 700 to "-tp haswell" ?

I set the env for pgi compiler and used the directory /opt/pgi/linux86-64-nollvm (not /opt/pgi/linux86-64-llvm). And, modify the content of Makefile.system to "-tp p7-64" or "-tp haswell" or "-tp zen", all compiled successfully.

[martin-frbg]

Thanks. I will leave the "-tp p7-64" unchanged then, and just correct the wrong "-march=native".