Compile for iOS ARMV8

[ForLiShuang]

I successfully compile Openblas following the guide in https://github.com/xianyi/OpenBLAS/wiki/How-to-build-OpenBLAS-for-iPhone-iOS-(ARMv8),

But when my program use Openblas, it always show the error:

Undefined symbols for architecture arm64:
"_sgemm_kernel", referenced from:
_sgemm_nn in libopenblas_armv8p-r0.2.20.dev.a(sgemm_nn.o)
_sgemm_nt in libopenblas_armv8p-r0.2.20.dev.a(sgemm_nt.o)
_inner_thread in libopenblas_armv8p-r0.2.20.dev.a(sgemm_thread_nn.o)
_inner_thread in libopenblas_armv8p-r0.2.20.dev.a(sgemm_thread_nt.o)
_inner_thread in libopenblas_armv8p-r0.2.20.dev.a(sgemm_thread_tn.o)
_inner_thread in libopenblas_armv8p-r0.2.20.dev.a(sgemm_thread_tt.o)
_sgemm_tn in libopenblas_armv8p-r0.2.20.dev.a(sgemm_tn.o)
...
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Anyone can give me some solution?

Thanks

[martin-frbg]

Can you run "nm" on the library to see which function symbols it has, in particular if there is any form of _sgemm_kernel, sgemm_kernel, sgemm_kernel_ or the like present ? Also does the build log
show anything unusual ?
Attempts to get this working on iOS have been tracked in #569 (from which the wiki entry was created), but unfortunately none of the current developers seems to use iOS or iPhone.

[brada4]

run "make clean" before next build.
Error sort of says symbol is in place, just that not for arm64... (but check with nm/ar)

[ForLiShuang]

@martin-frbg
log.txt
thanks, I use nm to print its info into log.txt, please check. I think it contains sgemm_kernel

[ForLiShuang]

@brada4 thanks, I think I use correctly command to compile openblas for ARMV8, suggested by https://github.com/xianyi/OpenBLAS/wiki/How-to-build-OpenBLAS-for-iPhone-iOS-(ARMv8),

I used
log.txt
nm to print its info into log.txt, please check. I think it contains sgemm_kernel

[martin-frbg]

_sgemm_kernel is there, but only with the "U" label (basically "called from here") but nowhere with the "T" for "here it is" (compare to "_dgemm_kernel", which has many "U" instances as well plus the important single "T"). Actually my original reason for the log request was to check if all underscores are present in the right places, now it looks as if the entire function got left out from the library.

[martin-frbg]

Strange also that your nm log contains detailed output of what looks to be local jump address labels within the assembler file for the sgemm kernel specifically (and labeled as "libopenblas.a(sgemm_kernel.o)" while all other paragraphs start with the plain name of the object file). Just not sure what if anything this is trying to tell me...
Can you redo the make run, and redirect its output to a file so that we can see if sgemm_kernel.o is somehow treated differently by the linker, or if building it triggered any compiler warnings ?

[martin-frbg]

From ashwinyes' comment in the older issue I mentioned above, #569 (comment) it seems the sgemm problem may be expected. The difference from dgemm_kernel etc. is apparently that those are implemented in C, and for some reason the xcode compiler fails to pick up the symbol names from the assembly file. (Though I do not understand why numerous other functions that are likewise implemented as assembly files, samax, damax, etc. appear to be compiled correctly).
You could try to replace the line SGEMMKERNEL= sgemm_kernel_4x4.S in kernel/arm64/KERNEL.ARMV8 with SGEMMKERNEL=../generic/gemmkernel_2x2.c (like it is used for DGEMMKERNEL a few lines below) to see if this fixes the build.

[ForLiShuang]

@martin-frbg thanks the compile seems correct, I need to test its speed

[martin-frbg]

I wonder if there is some combination of system data or environment variables that would allow the build system to recognize "cross-compiling on iOS for ARMv8" and make the required choices automatically ?
(Something like if defined(OS_DARWIN) && defined(CROSS) in KERNEL.ARMV8 for SGEMMKERNEL might work, and we would also need to learn what exactly goes wrong with the perl function for extracting the XCode path for $cross_suffix in c_check, instead of the current "does not work well, edit this file manually")

[martin-frbg]

So did you find time to test the library and see if you get decent speed from it ?

[bitianchi]

@ForLiShuang @martin-frbg
When I compiled openblas in IOS, I met following errors.

:4:21: error: unexpected token in argument list
beq KERNEL_F1_NEXT_@
^
:1:1: note: while in macro instantiation
KERNEL_F1
^
../kernel/arm64/nrm2.S:150:2: note: while in macro instantiation
KERNEL_F8
^
:7:27: error: unexpected token in argument list
bge KERNEL_F1_SCALE_GE_X_@
^
:1:1: note: while in macro instantiation
KERNEL_F1
^
../kernel/arm64/nrm2.S:150:2: note: while in macro instantiation
KERNEL_F8
^
:13:19: error: unexpected token in argument list
b KERNEL_F1_NEXT_@
^
:1:1: note: while in macro instantiation
KERNEL_F1
^
../kernel/arm64/nrm2.S:150:2: note: while in macro instantiation
KERNEL_F8
^
....

Is there any solutions?

[bitianchi]

I follow 569 and comment out lines in kernel/arm64/KERNEL.ARMV8(SNRM2KERNEL = nrm2.S DNRM2KERNEL = nrm2.S CNRM2KERNEL = znrm2.S ZNRM2KERNEL = znrm2.S), another error is showing up.

make[1]: *** No rule to make target ../kernel/arm64/nrm2.c', needed by snrm2_k.o'. Stop.

[martin-frbg]

I was just about to suggest this - what error is showing up now ? (Is it about SGEMM as discussed here, or another ? As I do not have IOS and nobody has reported back from a successful build, I do not yet know what #ifdef to put in KERNEL.ARMV8 to make these changes automatically when building on that platform)

[martin-frbg]

Did you do make clean ? If yes, it could be that in kernel/arm64/KERNEL the lines for NRM2 functions need to be changed to ../arm/nrm2.c (and ../arm/znrm2.c)

[bitianchi]

@martin-frbg
Thanks for your reply, here is the log

compile.log

[martin-frbg]

Thanks. That seems to be the original log before you commented the NRM2 entries ? But I guess the NRM2 entries in kernel/arm64/KERNEL do need to be changed to point to ../arm. Can you upload your Makefile.conf please ? (That should give me an idea what combination of settings to check for automatically)

[bitianchi]

It seems that if I replace "SNRM2KERNEL= nrm2.S" with "SNRM2KERNEL = ../arm/nrm2.c", I will get libopenblas.a

[martin-frbg]

Great. Did you also change the entry for SGEMMKERNEL ? (#1531 (comment))

[bitianchi]

I didn't change it. Is it necessary?

[martin-frbg]

Seems without that change you end up with missing symbols for sgemm when you try to link with the library (see start of this issue)

[bitianchi]

OK, I will try it. Thanks

[martin-frbg]

I have also prepared a patch that will hopefully make manual editing unnecessary in the future.

[L1onKing]

So did you find time to test the library and see if you get decent speed from it ?

Hello! I am also interested to use OpenBLAS for iOS and I must say that your tip works and after that change the error with sgemm_kernel was gone:

You could try to replace the line SGEMMKERNEL= sgemm_kernel_4x4.S in kernel/arm64/KERNEL.ARMV8 with SGEMMKERNEL=../generic/gemmkernel_2x2.c (like it is used for DGEMMKERNEL a few lines below) to see if this fixes the build.

But so far I can't really tell if there's an advantage in speed. Before I was using the BLAS version which is provided by Accelerate.framework of Apple, and with currently built OpenBLAS I have same results.

I have built OpenBLAS from release-0.2.21. Can that be an issue? Is there maybe a branch which fits better for iOS system for some reason?

[martin-frbg]

@L1onKing you could try 0.3.7 or the current develop branch, but if you are using SGEMM in your code the fundamental issue remains that XCode somehow cannot compile some of the optimized assembly code (the .S files) in OpenBLAS so they have to be replaced with (probably slower) C code.

[L1onKing]

@martin-frbg ok, I will try it a bit later, now I just have more important tasks to do. But anyway, my story with OpenBLAS on iOS is far from over and I will post any updates related to that here as soon as I have them.

[martin-frbg]

Thanks. Unfortunately I do not have IOS or any experience with xcode. As far as I know the Travis CI service has added IOS as a build option, but I think we would need to overcome the "search for where your code is installed and replace things in the c_check script" bit in the build instructions first. (Also not sure if we would need code signing and/or an Apple developer account for it to work)

[L1onKing]

I just compared performances of OpenBLAS I built with BLAS provided by Apple, and my binary is 30% slower in average.

Also, I'm pretty much a rookie when it comes to building static libraries so I assume I might miss some optimization step. For example, I understand that static library can be either Debug or Release. How to define if my static library is a fast release version, and not slow debug version?

[iiicp]

Even if you compile successfully, you may still call Accelerate.framework of Apple