Permalink
Browse files

build: add suport for x86 architecture

Modified android-configure script to support also x86 arch.
Currently added support only for ia32 target arch.
Also, compile openssl without asm, since using the asm sources will make
node fail to run on Android, because it adds text relocations.

Signed-off-by: Robert Chiras <robert.chiras@intel.com>
PR-URL: #5544
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
  • Loading branch information...
robertchiras authored and bnoordhuis committed Mar 3, 2016
1 parent c9c387f commit 271201fea935cdf85336736e87c06104ce185f61
Showing with 47 additions and 8 deletions.
  1. +39 −8 android-configure
  2. +8 −0 common.gypi
View
@@ -1,18 +1,49 @@
#!/bin/bash
if [ -z "$2" ]; then
ARCH=arm
else
ARCH="$2"
fi
CC_VER="4.9"
case $ARCH in
arm)
DEST_CPU="$ARCH"
SUFFIX="$ARCH-linux-androideabi"
TOOLCHAIN_NAME="$SUFFIX"
;;
x86)
DEST_CPU="ia32"
SUFFIX="i686-linux-android"
TOOLCHAIN_NAME="$ARCH"
;;
x86_64)
DEST_CPU="ia32"
SUFFIX="$ARCH-linux-android"
TOOLCHAIN_NAME="$ARCH"
;;
*)
echo "Unsupported architecture provided: $ARCH"
exit 1
;;
esac
export TOOLCHAIN=$PWD/android-toolchain
mkdir -p $TOOLCHAIN
$1/build/tools/make-standalone-toolchain.sh \
--toolchain=arm-linux-androideabi-4.9 \
--arch=arm \
--toolchain=$TOOLCHAIN_NAME-$CC_VER \
--arch=$ARCH \
--install-dir=$TOOLCHAIN \
--platform=android-21
export PATH=$TOOLCHAIN/bin:$PATH
export AR=$TOOLCHAIN/bin/arm-linux-androideabi-ar
export CC=$TOOLCHAIN/bin/arm-linux-androideabi-gcc
export CXX=$TOOLCHAIN/bin/arm-linux-androideabi-g++
export LINK=$TOOLCHAIN/bin/arm-linux-androideabi-g++
export AR=$TOOLCHAIN/bin/$SUFFIX-ar
export CC=$TOOLCHAIN/bin/$SUFFIX-gcc
export CXX=$TOOLCHAIN/bin/$SUFFIX-g++
export LINK=$TOOLCHAIN/bin/$SUFFIX-g++
./configure \
--dest-cpu=arm \
--dest-os=android
--dest-cpu=$DEST_CPU \
--dest-os=android \
--without-snapshot \
--openssl-no-asm
View
@@ -68,6 +68,10 @@
'cflags': [ '-gxcoff' ],
'ldflags': [ '-Wl,-bbigtoc' ],
}],
['OS == "android"', {
'cflags': [ '-fPIE' ],
'ldflags': [ '-fPIE', '-pie' ]
}]
],
'msvs_settings': {
'VCCLCompilerTool': {
@@ -101,6 +105,10 @@
['OS!="mac" and OS!="win"', {
'cflags': [ '-fno-omit-frame-pointer' ],
}],
['OS == "android"', {
'cflags': [ '-fPIE' ],
'ldflags': [ '-fPIE', '-pie' ]
}]
],
'msvs_settings': {
'VCCLCompilerTool': {

12 comments on commit 271201f

@clausreinke

This comment has been minimized.

Show comment
Hide comment
@clausreinke

clausreinke Jun 28, 2016

Adding -pie to ldflags seems to break node-gyp builds on android (if this is where node-gyp gets its common.gypi). It breaks the final addon linking step, even in the hello world addon example, by counteracting the -rdynamic, leading to undefined references.

clausreinke replied Jun 28, 2016

Adding -pie to ldflags seems to break node-gyp builds on android (if this is where node-gyp gets its common.gypi). It breaks the final addon linking step, even in the hello world addon example, by counteracting the -rdynamic, leading to undefined references.

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Jun 29, 2016

Member

@clausreinke Android support is self-serve. If you want to file a bug fix or a revert, can you open a pull request?

Member

bnoordhuis replied Jun 29, 2016

@clausreinke Android support is self-serve. If you want to file a bug fix or a revert, can you open a pull request?

@clausreinke

This comment has been minimized.

Show comment
Hide comment
@clausreinke

clausreinke Jun 29, 2016

I'm afraid this was my first dive into node-gyp (just trying to get ionic to install, which failed on building node-sass, and then it was "down the rabbit hole" from there). So i have no idea why you put that -pie here, or where it should go instead, so that executables get linked with -pie and addons without.

For what it is worth: removing -pie from ldflags in the common.gypiin my ~/.node-gyp directory fixed my linker issues for node-sass (and the hello world addon). Btw, this is using node via termux, on a non-rooted nexus 7 (2012) tablet, so node on android is available to the "masses"..;-)

But i guess the patch was added for a reason, and if i were to change it without understanding how it all works, something else would break.

clausreinke replied Jun 29, 2016

I'm afraid this was my first dive into node-gyp (just trying to get ionic to install, which failed on building node-sass, and then it was "down the rabbit hole" from there). So i have no idea why you put that -pie here, or where it should go instead, so that executables get linked with -pie and addons without.

For what it is worth: removing -pie from ldflags in the common.gypiin my ~/.node-gyp directory fixed my linker issues for node-sass (and the hello world addon). Btw, this is using node via termux, on a non-rooted nexus 7 (2012) tablet, so node on android is available to the "masses"..;-)

But i guess the patch was added for a reason, and if i were to change it without understanding how it all works, something else would break.

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Jun 29, 2016

Member

@robertchiras Can you comment?

Member

bnoordhuis replied Jun 29, 2016

@robertchiras Can you comment?

@robertchiras

This comment has been minimized.

Show comment
Hide comment
@robertchiras

robertchiras Jun 29, 2016

Contributor

@clausreinke Android requires that executables to be built using -pie, otherwise they will fail to run. This is why, node binary has to be compiled with -pie and linked with -fPIE. This requirement was added in Jelly Bean, as you can see here: https://duo.com/blog/exploit-mitigations-in-android-jelly-bean-4-1
But, I don't understand why do you need common.gypi when building addons? I thought that all you need, when building addons is the binding.gyp file (which you have to provide). Next, node-gyp configure will generate the config.gypi file with is then used by node-gyp build.
I had no issues in building node addons for Android using both npm or node-gyp before.

Contributor

robertchiras replied Jun 29, 2016

@clausreinke Android requires that executables to be built using -pie, otherwise they will fail to run. This is why, node binary has to be compiled with -pie and linked with -fPIE. This requirement was added in Jelly Bean, as you can see here: https://duo.com/blog/exploit-mitigations-in-android-jelly-bean-4-1
But, I don't understand why do you need common.gypi when building addons? I thought that all you need, when building addons is the binding.gyp file (which you have to provide). Next, node-gyp configure will generate the config.gypi file with is then used by node-gyp build.
I had no issues in building node addons for Android using both npm or node-gyp before.

@clausreinke

This comment has been minimized.

Show comment
Hide comment
@clausreinke

clausreinke Jun 29, 2016

@robertchiras ˋ-pieˋ for linking and ˋ-fPIEˋ for compiling, if it is an executable, yes. But the same ˋldflagsˋ are apparently used for linking addons (shared lib), which is causing linking issues with the open references that will be provided when ˋnodeˋ loads the addon.

Is it possible that you have an older ˋ~/.node-gyp/ˋ lying around? In that case, ˋnode-gypˋ might not overwrite its contents with your new, patched version, so building addons on android still works. If it does not exist, or does not have the files for the current node version, ˋnode-gypˋ will download and use a tarball with ˋnodeˋ's ˋincludeˋ dir, including ˋcommon.gypiˋ (see ˋnode-gypˋ, ˋlib/{install.js,config.js}ˋ, ˋdevDirˋ and ˋnodeDirˋ).

clausreinke replied Jun 29, 2016

@robertchiras ˋ-pieˋ for linking and ˋ-fPIEˋ for compiling, if it is an executable, yes. But the same ˋldflagsˋ are apparently used for linking addons (shared lib), which is causing linking issues with the open references that will be provided when ˋnodeˋ loads the addon.

Is it possible that you have an older ˋ~/.node-gyp/ˋ lying around? In that case, ˋnode-gypˋ might not overwrite its contents with your new, patched version, so building addons on android still works. If it does not exist, or does not have the files for the current node version, ˋnode-gypˋ will download and use a tarball with ˋnodeˋ's ˋincludeˋ dir, including ˋcommon.gypiˋ (see ˋnode-gypˋ, ˋlib/{install.js,config.js}ˋ, ˋdevDirˋ and ˋnodeDirˋ).

@robertchiras

This comment has been minimized.

Show comment
Hide comment
@robertchiras

robertchiras Jun 30, 2016

Contributor

@clausreinke I don't know what is this '~/.node-gyp' directory.
Since I am building on my host for my target device (which, in my case is an Edison board), I need to first configure npm for cross-compilation. So, I will run <path_to_node>/android-configure <path_to_ndk> x86 (I use x86 because I am building for an 'x86' target device, but you could use 'arm' if you're building for an arm device). In my case, I also need to specify the arch to npm by running: 'npm config set arch ia32' (or 'npm config set arch arm' for an arm device). These steps are necessary for cross-compilation with both npm or node-gyp (anyway, npm is calling node-gyp configure, node-gyp rebuild and node-gyp install, when you issue an npm install command, so basically these two are doing the same job).

I use the following methods to build a node module:

  1. Using npm. Make sure you have node installed, go to your module directory and run 'npm build .'
  2. Using node-gyp directly: <path_to_node>/node_modules/node-gyp/bin/node-gyp configure . followed by <path_to_node>/node_modules/node-gyp/bin/node-gyp build .
  3. Install node-gyp using npm install -g node-gyp the run node-gyp --nodedir <path_to_node> configure followed by node-gyp --nodedir <path_to_node> build

Basically, all the three methods are the same, since all of them are calling node-gyp configure and node-gyp build.

One note here: these steps worked until a few versions. Now, there are some issues with the cross-compilation of node and node modules, but I have some patches submitted, but not yet accepted (I also have to come back to them since they are kind of old).

Contributor

robertchiras replied Jun 30, 2016

@clausreinke I don't know what is this '~/.node-gyp' directory.
Since I am building on my host for my target device (which, in my case is an Edison board), I need to first configure npm for cross-compilation. So, I will run <path_to_node>/android-configure <path_to_ndk> x86 (I use x86 because I am building for an 'x86' target device, but you could use 'arm' if you're building for an arm device). In my case, I also need to specify the arch to npm by running: 'npm config set arch ia32' (or 'npm config set arch arm' for an arm device). These steps are necessary for cross-compilation with both npm or node-gyp (anyway, npm is calling node-gyp configure, node-gyp rebuild and node-gyp install, when you issue an npm install command, so basically these two are doing the same job).

I use the following methods to build a node module:

  1. Using npm. Make sure you have node installed, go to your module directory and run 'npm build .'
  2. Using node-gyp directly: <path_to_node>/node_modules/node-gyp/bin/node-gyp configure . followed by <path_to_node>/node_modules/node-gyp/bin/node-gyp build .
  3. Install node-gyp using npm install -g node-gyp the run node-gyp --nodedir <path_to_node> configure followed by node-gyp --nodedir <path_to_node> build

Basically, all the three methods are the same, since all of them are calling node-gyp configure and node-gyp build.

One note here: these steps worked until a few versions. Now, there are some issues with the cross-compilation of node and node modules, but I have some patches submitted, but not yet accepted (I also have to come back to them since they are kind of old).

@robertchiras

This comment has been minimized.

Show comment
Hide comment
@robertchiras

robertchiras Jun 30, 2016

Contributor

One more thing: in order to verify the validity of my addon library, I run the following commands:

  • file build/Release/obj.target/addon.node and make sure the correct arch is used (if I don't configure npm for the target's arch, node-gyp will use the host arch and of course, it won't work on my device).
  • nm -D build/Release/obj.target/addon.node and make sure that all the symbols to 'node_module_register' and v8 functions used by my module are present and undefined.
Contributor

robertchiras replied Jun 30, 2016

One more thing: in order to verify the validity of my addon library, I run the following commands:

  • file build/Release/obj.target/addon.node and make sure the correct arch is used (if I don't configure npm for the target's arch, node-gyp will use the host arch and of course, it won't work on my device).
  • nm -D build/Release/obj.target/addon.node and make sure that all the symbols to 'node_module_register' and v8 functions used by my module are present and undefined.
@clausreinke

This comment has been minimized.

Show comment
Hide comment
@clausreinke

clausreinke Jun 30, 2016

~/.node-gyp is where node-gyp installs its and node's common include files (unless you give a --nodedir flag). You can see that in the gist linked to below, or in the sources I mentioned above.

I still don't understand how you can build any addon for android with those ldflags - -pie is meant for linking executables, not for libraries.

I've made a gist with a failure log for the hello world addon, on my android tablet (using termux). The most relevant parts are

gyp verb ensuring nodedir is created /data/data/com.termux/files/home/.node-gyp/6.2.2
gyp verb created nodedir /data/data/com.termux/files/home/.node-gyp

gyp http GET https://nodejs.org/download/release/v6.2.2/node-v6.2.2-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v6.2.2/node-v6.2.2-headers.tar.gz

gyp verb extracted file from tarball include/node/common.gypi

  g++ -shared -rdynamic -fPIE -pie  -Wl,-soname=addon.node -o Release/obj.target/addon.node -Wl,--start-group Release/obj.target/addon/hello.o -Wl,--end-group -llog

/data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/6.1.0/../../../crtbegin_dynamic.o: In function `_start':
crtbrand.c:(.text+0x84): undefined reference to `main'
Release/obj.target/addon/hello.o: In function `demo::Method(v8::FunctionCallbackInfo<v8::Value> const&)':
hello.cc:(.text+0x20): undefined reference to `v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::String::NewStringType, int)'

$ uname -a
Linux localhost 3.1.10-gf5d7b8b #1 SMP PREEMPT Thu Jan 8 04:50:16 UTC 2015 armv7l Android

note the conflicting flags -shared -rdynamic (produce shared library) and -pie (produce position-independent executable). Eliminating -pie from common.gypi lets the build go through.

clausreinke replied Jun 30, 2016

~/.node-gyp is where node-gyp installs its and node's common include files (unless you give a --nodedir flag). You can see that in the gist linked to below, or in the sources I mentioned above.

I still don't understand how you can build any addon for android with those ldflags - -pie is meant for linking executables, not for libraries.

I've made a gist with a failure log for the hello world addon, on my android tablet (using termux). The most relevant parts are

gyp verb ensuring nodedir is created /data/data/com.termux/files/home/.node-gyp/6.2.2
gyp verb created nodedir /data/data/com.termux/files/home/.node-gyp

gyp http GET https://nodejs.org/download/release/v6.2.2/node-v6.2.2-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v6.2.2/node-v6.2.2-headers.tar.gz

gyp verb extracted file from tarball include/node/common.gypi

  g++ -shared -rdynamic -fPIE -pie  -Wl,-soname=addon.node -o Release/obj.target/addon.node -Wl,--start-group Release/obj.target/addon/hello.o -Wl,--end-group -llog

/data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/6.1.0/../../../crtbegin_dynamic.o: In function `_start':
crtbrand.c:(.text+0x84): undefined reference to `main'
Release/obj.target/addon/hello.o: In function `demo::Method(v8::FunctionCallbackInfo<v8::Value> const&)':
hello.cc:(.text+0x20): undefined reference to `v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::String::NewStringType, int)'

$ uname -a
Linux localhost 3.1.10-gf5d7b8b #1 SMP PREEMPT Thu Jan 8 04:50:16 UTC 2015 armv7l Android

note the conflicting flags -shared -rdynamic (produce shared library) and -pie (produce position-independent executable). Eliminating -pie from common.gypi lets the build go through.

@clausreinke

This comment has been minimized.

Show comment
Hide comment
@clausreinke

clausreinke Jul 1, 2016

I had some difficulties figuring out target_condition syntax and _type values, eg an addon strangely does not seem to count as a shared_library..

Does something like this work for you (moving ˋ-pieˋ behind a target_condition)?

['OS == "android"', {
            'cflags': [ '-fPIE' ],
            'ldflags': [ '-fPIE' ],
            'target_conditions': [
              ['_type == "executable"', {
                'ldflags': [ '-pie' ]
              }]
            ]
          }]

If I do that for both configurations, I can still build addons, and you should (might?-) be able to build executables.

clausreinke replied Jul 1, 2016

I had some difficulties figuring out target_condition syntax and _type values, eg an addon strangely does not seem to count as a shared_library..

Does something like this work for you (moving ˋ-pieˋ behind a target_condition)?

['OS == "android"', {
            'cflags': [ '-fPIE' ],
            'ldflags': [ '-fPIE' ],
            'target_conditions': [
              ['_type == "executable"', {
                'ldflags': [ '-pie' ]
              }]
            ]
          }]

If I do that for both configurations, I can still build addons, and you should (might?-) be able to build executables.

@clausreinke

This comment has been minimized.

Show comment
Hide comment
@clausreinke

clausreinke Jul 2, 2016

@robertchiras Sorry, forgot to ping you. Could you please test my suggestion above and submit a fix for your code if it works?

clausreinke replied Jul 2, 2016

@robertchiras Sorry, forgot to ping you. Could you please test my suggestion above and submit a fix for your code if it works?

@akehir

This comment has been minimized.

Show comment
Hide comment
@akehir

akehir May 29, 2017

@robertchiras 👍 for testing @clausreinke 's proposal.

akehir replied May 29, 2017

@robertchiras 👍 for testing @clausreinke 's proposal.

Please sign in to comment.