OpenBLAS-0.3.11 fails to compile on ppc64le

[susilehtola]

Updating the Fedora package of OpenBLAS to version 0.3.11 fails, since the package does not build on ppc64le. I'm getting a bunch of errors like

../kernel/power/sgemm_kernel_power10.c:131:11: error: '__builtin_mma_xvf32gerpp' requires the '-mmma' option
  131 |           __builtin_mma_xvf32gerpp (&acc2, rowB[i], rowA[j+1]); \
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

which means you should probably add this option to the makefiles.

Also, I noticed a bunch of macro redefinitions in the power8 kernels, e.g.

../kernel/power/dtrmm_kernel_16x4_power8.S:85: note: this is the location of the previous definition
   85 | #define STACKSIZE 320
      | 
../kernel/power/dtrsm_kernel_LT_16x4_power8.S:51: warning: "STACKSIZE" redefined
   51 | #define STACKSIZE 520
      | 
../kernel/power/dtrsm_kernel_LT_16x4_power8.S:50: note: this is the location of the previous definition
   50 | #define STACKSIZE 320
      | 

which you may want to address.

[susilehtola]

Full build log
build.log.gz

[martin-frbg]

Thanks, looks like 0.3.11 is going to set a new record for short-livedness. At least this mma error does not appear to be my fault (the redefinitions are, I'd left the old values in when I expanded the stack to save&restore additional registers). Good to know you have access to POWER10 though...

[martin-frbg]

Ah, that is "only" a DYNAMIC_ARCH build on POWER8 ? Guess we may need to add some core-specific compiler options like it is already necessary for AVX512 on x86_64.

[martin-frbg]

The POWER10 support is only included when GCC >10.2 , and unfortunately I cannot test my current PR on Fedora32 in the OpenPOWER cloud as even with gcc 10.2.1 and the latest binutils (2.34-5.fc32) I hit an assembler error invalid switch -mpower10
@RajalakshmiSR do we need to (test for) a specific as or binutils version for POWER10 as well ?

[martin-frbg]

Answering my own question it appears binutils 2.35 would be required

[martin-frbg]

Hmm, do not get the -mmma error on Debian11 ppc64le (which has gcc10.2 & binutils 2.35)

[martin-frbg]

Suspect something in the way your patches modify the build options overrides or prevents the automatic addition of -mcpu=power10 -mtune=power10 -mvsx here https://github.com/xianyi/OpenBLAS/blob/5e8ddc9001d705473e106cf7ff285d170b7e8944/Makefile.power#L13

[RajalakshmiSR]

checking

[RajalakshmiSR]

We need gcc10.2 or gcc11 and binutils 2.35 to compile power10 files.

[martin-frbg]

May need to add a version test for as then (not that I like the idea, but that is what anyone on fedora32 would probably encounter right now). Main problem seems to be that his build options are somehow removing/preventing the addition of the -mcpu=power10 -mvsx, but perhaps that line in Makefile.power just needs to be override COMMON_OPT +=... for good measure.

[RajalakshmiSR]

So are you suggesting to check for binutils version before we add POWER10 in the DYNAMIC_ARCH list in Makefile.system?

[martin-frbg]

Yes, thinking about that at least, as we cannot be sure that gcc and binutils are updated together.

[RajalakshmiSR]

I will create a PR for this.

[RajalakshmiSR]

@susilehtola Is it possible to apply this patch on your fedora system and compile/check if it helps.

diff --git a/Makefile.system b/Makefile.system
index 461f7370..44a9bbf5 100644
--- a/Makefile.system
+++ b/Makefile.system
@@ -648,12 +648,17 @@ DYNAMIC_CORE += POWER9
else
$(info, OpenBLAS: Your gcc version is too old to build the POWER9 kernels.)
endif
+LDVERSIONGTEQ35 := $(shell expr ld --version | head -1 | cut -f2 -d "." | cut -f1 -d "-" >= 35)
ifeq ($(GCCVERSIONGTEQ11), 1)
+ifeq ($(LDVERSIONGTEQ35), 1)
DYNAMIC_CORE += POWER10
+endif
else ifeq ($(GCCVERSIONGTEQ10), 1)
ifeq ($(GCCMINORVERSIONGTEQ2), 1)
+ifeq ($(LDVERSIONGTEQ35), 1)
DYNAMIC_CORE += POWER10
endif
+endif
else
$(info, OpenBLAS: Your gcc version is too old to build the POWER10 kernels.)
endif