Android prebuilt no arm64-v8a support

[mars3142]

In the external folder are not prebuilts with 64bit support like arm64-v8a. Is there a way to get 64 bit support?

[WenhaiLin]

https://github.com/WenhaiLin/cocos2d-x-lite-external/tree/master/android

[minggo]

@mars3142 did you meet any issue or just want to improve performance?

[mars3142]

I found it, because I used the ndk-bundle from the Android SDK. Which seems to use arm64-v8a as default. And in my Application.mk I can't set

APP_ABI := all

only

APP_ABI := armeabi armeabi-v7a x86

is possible. And because more 64 bit phones are coming, I was just wondering.

Another problem was, that I had to downgrade my NDK toolchain to r10, because r11 isn't working. But this seems to be an Google bug.

@WenhaiLin And why isn't this in the official release included?

[minggo]

@mars3142 thanks for the information. I agree with you that cocos2d-x should support arm-64-v8a now. @zilongshanren any idea of it?

[zilongshanren]

@minggo
The current directory structure is not good for have both 32-bit and 64-bit support, we have to do some refactoring of the external deps just as we do in v4.0.

I think it's very important to include arm64 not just for iOS but also for Android.

[minggo]

@ricardoquesada i think we need to provide arm64 for Android, can you modify the compile script to build out arm64 libs?

[ricardoquesada]

@minggo sure. I'll do that.

[ricardoquesada]

@minggo the scripts already support that... I think I added that a few months ago while compiling cocos2d-x v2.1.5 for arm64 (or perhaps someone else added it).

You should know that the API level should be changed from 9 to 21.
How to compile it:

$ ./build.sh -p=android --arch=arm64

but API level should be changed to 21. I'll make sure that cocos2d-x compiles with arm64 as well

[ricardoquesada]

@minggo should we add arm64-v8a in APP_ABI in all our Application.mk files?
it will take longer to compile, but it will be a good tests.
Also, what is your idea? to bundle all the 64bit libraries in the 3rd party libs ?

[minggo]

@ricardoquesada I think we don't have to add arm64-v8a in APP_ABI, i think it is better to let developers determine it. But we should provide 64bit libraries in 3rd party libs, then developers can build for arm64-v8a if they want. Indeed, the default APP_ABI is arm64-v8a in NDK r12b, and i added ABI: armeabi to fix it using NDK 12b.

[ricardoquesada]

@minggo

sure. done... sort of:

• luajit in 64bit, does it work?
• what version of SpiderMonkey are we using? ( https://github.com/cocos2d/Spidermonkey/ )

Default branch is 33, but there is a 34 branch.
I'll merge my changes now, and once lua and js are fixed, I'll merge those two changes as well.

[minggo]

@ricardo

As i know, luajit doesn't support arm64, and we use lua for arm64. About SpiderMonkey, @panda told me that, we can not update it to new version, or the binding codes may break compatibility.

[minggo]

@pandamicro

[ricardoquesada]

@pandamicro which branch are we using in cocos2d-x v3.12? v34 or v33?
And any idea how to compile it from Android arm64?

@minggo I think LuaJIT does support arm64 now... but I could be wrong.
Anyway, how did we solve it for iOS and Mac 64-bits? Are we using Lua instead of LuaJIT?
If so, what scripts should I change so that Android ARM64 uses Lua?
Thanks!

[minggo]

@ricardoquesada yep, we use lua source code for iOS arm64. It is supported for a long time. I think what we should do just for Android 64 bit.

[ricardoquesada]

@minggo thanks. Any idea what should I do? what scripts should I modify? thanks.

[minggo]

@ricardoquesada now when i run build.sh -p=android --libs=websocket, it doesn't generate lib for armeabi-v8a, can we modify it to generate armeabi-v8a automatically without doing any other operation?

[pandamicro]

@pandamicro which branch are we using in cocos2d-x v3.12? v34 or v33?
And any idea how to compile it from Android arm64?

It's v33, before we made a total abstract API layer from Spidermonkey, I don't think we should upgrade it, because the API changes will affect user binding code, AnySDK support, SDKBox support, etc...

[ricardoquesada]

@minggo you might be using an slightly older version... trying "git pulling" again.
I've just tried it and I got:

(v3)$ find .
.
./include
./include/libwebsockets.h
./prebuilt
./prebuilt/arm64-v8a
./prebuilt/arm64-v8a/libwebsockets.a
./prebuilt/armeabi
./prebuilt/armeabi/libwebsockets.a
./prebuilt/armeabi-v7a
./prebuilt/armeabi-v7a/libwebsockets.a
./prebuilt/x86
./prebuilt/x86/libwebsockets.a

@pandamicro Ok. So I'll use branch v33.

[minggo]

@ricardoquesada yep, now it builds out arm64-va8. Sorry about that, i didn't know you modify it.

[ricardoquesada]

JavaScript ARM64:
sources: Spidermonkey android arm64: cocos2d/Spidermonkey#3
binary: Spidermonkey android arm64: cocos2d/cocos2d-x-3rd-party-libs-bin#238

Lua ARM64:
cocos2d/cocos2d-x-3rd-party-libs-bin#239

@minggo Everything seems to be working fine:

• cpp tests
• js tests
• lua tests... except that the Lua encrypted bytecodes can't be read by Lua... not sure why: lua-empty-test works Ok, but lua-tests fails because of the encrypted bytecodes.

Any idea where should I look? Who is our Lua guy? I need his help here.

[ricardoquesada]

PR merged ( #16090 )
So, android arm64 is like 90% complete... what's missing is this Lua encrypted bytecode thing.

[minggo]

thanks @ricardoquesada , are you sure luajit supports arm64, if so, then we don't have to use lua source code for iOS arm64.

[ricardoquesada]

@minggo no, I'm not a 100% sure about it.
Apparently in LuaJIT 2.1 (still in beta) supports it. More info here:
https://collaborate.linaro.org/display/TCWGPUB/The+State+of+LuaJit+for+ARM+64+-+July+2016

[minggo]

@ricardoquesada what's the status of this issue?

[ricardoquesada]

@minggo everything has been merged... but I couldn't make it work on Android 64 + Lua + Lua encrypted bytecodes.
The Lua-empty-tests works Ok since it doesn't use encrypted bytecodes. But Lua-tests doesn't work because of that.
It would be great if someone who understands how our Lua bindings work could take a look at it. Otherwise I'll try to fix it myself, but it might take me more time.

[minggo]

@ricardoquesada as i remember the byte code of luajit64 is not compatible with luajit32. Which means we should provide two versions of lua byte codes and load correct byte codes in different architecture.

We should discuss how to fix it and not bring two much inconvenience.

[ricardoquesada]

@minggo a ha! that explains it! what's your proposal for this?
I guess users will have to build two different bytecodes for their games then. is that correct?
what are we doing in iOS regarding LuaJIT in 32-bit and 64-bit?

[minggo]

Currently we use luajit for 32-bit and lua for 64-bit because luajit doesn't support 64-bit before. If we want to use luajit both for 32-bit and 64-bit, then i think there are two solutions:

1. use source code instead of byte code. We currently uses this solution, because byte code of lua an luajit is not compatible. So we just need to replace lua to luajit for 64-bit.
2. use different byte code for different architecture. Then we should provide tools to generate byte codes, and we should have a mechanism to load correct byte code. As i remember, generating 64-bit byte code is not easy, should use real 64-bit machine to generate.

So i think we can use solution one. Just replace lua with luajit for 64-bit and use lua source code.

[fuyifan]

actually, it should not be the source code but the compiled one without byte code. In cocos console, it would be something like "cocos luacompile -s ./lua -d ./src -e -k XXX -b XXX --disable-compile". Source code is not safe for publish coz your app will be easy hacked and the related data can be read and revised. The only problem is that the compiled code will be much larger than the bytecode.

[minggo]

actually, it should not be the source code but the compiled one without byte code

Compile it without byte code, what did you mean?

[fuyifan]

According to forum : http://www.cocoachina.com/bbs/read.php?tid=335178, I use this method for publish in iOS.

[minggo]

@fuyifan as thread you pasted said, it just encrypt the source codes, not compile to byte code.

[minggo]

@ricardoquesada now the testing engineer reported that, it will be black screen when building and running lua-tests on 64-bit Android devcices with cocos run -p android --app-abi arm64-v8a.

[ricardoquesada]

@minggo yes, that was what I was saying about the encrypted bytecodes. Because it works ok on lua-empty-tests.

I'm not aware of the internals of our Lua implementation... I also don't know how the encryption works... but I can take a look at it if you want.

[minggo]

@ricardoquesada i don't know it uses bytecodes in lua-tests, i will take a look too.

[minggo]

@ricardoquesada have you already modify cocos2d-x-3rd-party-libs-src to built luajit for arm64? And have you updated it to 2.1.0-beta2? If so, could you please send a PR? Then i will fix the error.

[minggo]

Now android uses lua for 64bit, and it can not load the bytecodes generated by luajit, that's the reason lua-tests failed. I think we can just ignore the test case and create a new issue in v3.14 that:

• use luajit for arm64 on android
• provide tools that can generate bytecodes for luajit32 and luajit64 and add corresponding test case

[ricardoquesada]

@minggo ok. makes sense.

just to know, what are we doing for iOS 64bit? Are we using LuaJIT 2.1.0-beta2? Whatever we are doing with iOS 64-bit, we should do for Android.

[minggo]

@ricardoquesada i think we used luajit 2.1.0-alpha for iOS 64-bit. I created #16365 to trace it.

[ricardoquesada]

@minggo ok. good to know, thanks.