New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: mkpeephole fails during cross compiling #9707

Closed
mscdex opened this Issue Nov 20, 2016 · 18 comments

Comments

Projects
None yet
9 participants
@mscdex
Contributor

mscdex commented Nov 20, 2016

  • Version: v7.x
  • Platform: linux
  • Subsystem: build

I'm trying to cross-compile node v7.x for a MIPS device running OpenWrt. At first I ran into an issue where mkpeephole was failing because it was trying to do something on the host system for the target but it was using the target compiler (CC and CXX env vars set).

So then I tried setting CC_host=gcc and CXX_host=g++ and now I get this:

In file included from ../deps/v8/src/base/logging.h:8:0,
                 from ../deps/v8/src/base/macros.h:10,
                 from ../deps/v8/src/base/accounting-allocator.h:9,
                 from ../deps/v8/src/base/accounting-allocator.cc:5:
/usr/include/c++/5/cstring:41:28: fatal error: bits/c++config.h: No such file or directory

because it's trying to use the host compiler to compile that accounting-allocator.cc, even though it used the target compiler to build other V8 source files before this error.

With CC_host/CXX_host set and these configure flags: --without-snapshot --without-intl --without-inspector --without-ssl --dest-os=linux --dest-cpu=mips

I get this config.gypi:

{ 'target_defaults': { 'cflags': [],
                       'default_configuration': 'Release',
                       'defines': [],
                       'include_dirs': [],
                       'libraries': []},
  'variables': { 'asan': 0,
                 'debug_devtools': 'node',
                 'force_dynamic_crt': 0,
                 'host_arch': 'x64',
                 'icu_small': 'false',
                 'mips_arch_variant': 'r2',
                 'mips_fpu_mode': 'fp32',
                 'node_byteorder': 'little',
                 'node_enable_d8': 'false',
                 'node_enable_v8_vtunejit': 'false',
                 'node_install_npm': 'true',
                 'node_module_version': 51,
                 'node_no_browser_globals': 'false',
                 'node_prefix': '/usr/local',
                 'node_release_urlbase': '',
                 'node_shared': 'false',
                 'node_shared_cares': 'false',
                 'node_shared_http_parser': 'false',
                 'node_shared_libuv': 'false',
                 'node_shared_openssl': 'false',
                 'node_shared_zlib': 'false',
                 'node_tag': '',
                 'node_use_bundled_v8': 'true',
                 'node_use_dtrace': 'false',
                 'node_use_etw': 'false',
                 'node_use_lttng': 'false',
                 'node_use_openssl': 'false',
                 'node_use_perfctr': 'false',
                 'node_use_v8_platform': 'true',
                 'openssl_fips': '',
                 'openssl_no_asm': 0,
                 'shlib_suffix': 'so.51',
                 'target_arch': 'mips',
                 'uv_parent_path': '/deps/uv/',
                 'uv_use_dtrace': 'false',
                 'v8_can_use_fpu_instructions': 'true',
                 'v8_enable_gdbjit': 0,
                 'v8_enable_i18n_support': 0,
                 'v8_inspector': 'false',
                 'v8_no_strict_aliasing': 1,
                 'v8_optimized_debug': 0,
                 'v8_random_seed': 0,
                 'v8_use_mips_abi_hardfloat': 'true',
                 'v8_use_snapshot': 'false',
                 'want_separate_host_toolset': 0,
                 'want_separate_host_toolset_mkpeephole': 1}}
@mscdex

This comment has been minimized.

Show comment
Hide comment
@mscdex

mscdex Nov 20, 2016

Contributor

I should note that node v4.x compiles without such build issues (not including manual patches needed for V8 and other sources), using the same configure flags (minus the newer flags of course).

Contributor

mscdex commented Nov 20, 2016

I should note that node v4.x compiles without such build issues (not including manual patches needed for V8 and other sources), using the same configure flags (minus the newer flags of course).

@dna2github

This comment has been minimized.

Show comment
Hide comment
@dna2github

dna2github Nov 20, 2016

it is a v8 issue;
maybe currently we can do is merely:
waiting it fails, copy mkpeephole to target machine;
run mkpeephole and copy bytecode-peephole-table.cc back;
then modify v8_base.target.mk to skip mkpeephole executing.
i think you have already done manual patches to pass the issue.

dna2github commented Nov 20, 2016

it is a v8 issue;
maybe currently we can do is merely:
waiting it fails, copy mkpeephole to target machine;
run mkpeephole and copy bytecode-peephole-table.cc back;
then modify v8_base.target.mk to skip mkpeephole executing.
i think you have already done manual patches to pass the issue.

@mscdex

This comment has been minimized.

Show comment
Hide comment
@mscdex

mscdex Nov 20, 2016

Contributor

I'll leave this issue open but unfortunately I've bricked the device I was testing with (with no easy way to un-brick it), so I won't be able to test any fixes...

Contributor

mscdex commented Nov 20, 2016

I'll leave this issue open but unfortunately I've bricked the device I was testing with (with no easy way to un-brick it), so I won't be able to test any fixes...

@artynet

This comment has been minimized.

Show comment
Hide comment
@artynet

artynet Dec 22, 2016

@bnoordhuis please consider also this simple bash script 👍

https://gist.github.com/artynet/bc7b12e91c81dd4d4bed7cf9d47c9a02

to build a cross-compiled binary release of the node sources for Raspberry Pi 2. It relies on a pre-existing native build so to leverage the native mkpeephole tool which of course being cross-compiled during the normal build can't be executed by the host machine.

This binary build should be residing in the very same folder of the new source to cross-compile. The cross-toolchain has been generated with the crosstools-ng suite.

Best, Arturo

artynet commented Dec 22, 2016

@bnoordhuis please consider also this simple bash script 👍

https://gist.github.com/artynet/bc7b12e91c81dd4d4bed7cf9d47c9a02

to build a cross-compiled binary release of the node sources for Raspberry Pi 2. It relies on a pre-existing native build so to leverage the native mkpeephole tool which of course being cross-compiled during the normal build can't be executed by the host machine.

This binary build should be residing in the very same folder of the new source to cross-compile. The cross-toolchain has been generated with the crosstools-ng suite.

Best, Arturo

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Dec 22, 2016

Member

@artynet If you have a script that builds from inside a source tree instead of downloading a tarball, you're welcome to open a pull request. Try to unify it with the android-configure script if possible.

Member

bnoordhuis commented Dec 22, 2016

@artynet If you have a script that builds from inside a source tree instead of downloading a tarball, you're welcome to open a pull request. Try to unify it with the android-configure script if possible.

@artynet

This comment has been minimized.

Show comment
Hide comment
@artynet

artynet Dec 22, 2016

@mscdex please take a look at my new openwrt makefile quoted a couple of posts ago....

artynet commented Dec 22, 2016

@mscdex please take a look at my new openwrt makefile quoted a couple of posts ago....

@mscdex

This comment has been minimized.

Show comment
Hide comment
@mscdex

mscdex Dec 22, 2016

Contributor

@artynet As I mentioned previously, I can't test any changes because the device I was testing with is bricked, so I'm not the best person to ask to review/test any changes.

Contributor

mscdex commented Dec 22, 2016

@artynet As I mentioned previously, I can't test any changes because the device I was testing with is bricked, so I'm not the best person to ask to review/test any changes.

@artynet

This comment has been minimized.

Show comment
Hide comment
@artynet

artynet Dec 22, 2016

@bnoordhuis

This is the quickest fix I come up with :

--- a/android-configure	2016-12-20 20:53:59.000000000 +0100
+++ b/android-configure	2016-12-22 23:25:34.535540854 +0100
@@ -8,6 +8,12 @@
 # modules with npm. Also, don't forget to set the arch in npm config using
 # 'npm config set arch=<arch>'

+if [ -z $3 ]; then
+    echo "insert node version !"
+    return 1
+fi
+
+VERSION=$3

 if [ -z "$2" ]; then
     ARCH=arm
@@ -15,6 +21,14 @@
     ARCH="$2"
 fi

+fixv7 () {
+
+    var=$(readlink -f ../node-v$VERSION-linux)"/out/Release/mkpeephole"
+    echo $var
+    sed "s#<(mkpeephole_exec)#$var#g" -i deps/v8/src/v8.gyp
+
+}
+
 CC_VER="4.9"
 case $ARCH in
     arm)
@@ -38,13 +52,16 @@
         ;;
 esac

+# fixing mkpeephole system call
+fixv7 $3
+
 export TOOLCHAIN=$PWD/android-toolchain
 mkdir -p $TOOLCHAIN
 $1/build/tools/make-standalone-toolchain.sh \
     --toolchain=$TOOLCHAIN_NAME-$CC_VER \
     --arch=$ARCH \
     --install-dir=$TOOLCHAIN \
-    --platform=android-21
+    --platform=android-21 --force
 export PATH=$TOOLCHAIN/bin:$PATH
 export AR=$TOOLCHAIN/bin/$SUFFIX-ar
 export CC=$TOOLCHAIN/bin/$SUFFIX-gcc
@@ -62,5 +79,6 @@
         --dest-cpu=$DEST_CPU \
         --dest-os=android \
         --without-snapshot \
-        --openssl-no-asm
+        --openssl-no-asm \
+        --without-intl
 fi

for node7 v7.3.0 (latest release). We have to find a way to make it universal for the trunk tree. As mentioned before, a native build for the host machine is needed as a prerequisite.

Best, Arturo

artynet commented Dec 22, 2016

@bnoordhuis

This is the quickest fix I come up with :

--- a/android-configure	2016-12-20 20:53:59.000000000 +0100
+++ b/android-configure	2016-12-22 23:25:34.535540854 +0100
@@ -8,6 +8,12 @@
 # modules with npm. Also, don't forget to set the arch in npm config using
 # 'npm config set arch=<arch>'

+if [ -z $3 ]; then
+    echo "insert node version !"
+    return 1
+fi
+
+VERSION=$3

 if [ -z "$2" ]; then
     ARCH=arm
@@ -15,6 +21,14 @@
     ARCH="$2"
 fi

+fixv7 () {
+
+    var=$(readlink -f ../node-v$VERSION-linux)"/out/Release/mkpeephole"
+    echo $var
+    sed "s#<(mkpeephole_exec)#$var#g" -i deps/v8/src/v8.gyp
+
+}
+
 CC_VER="4.9"
 case $ARCH in
     arm)
@@ -38,13 +52,16 @@
         ;;
 esac

+# fixing mkpeephole system call
+fixv7 $3
+
 export TOOLCHAIN=$PWD/android-toolchain
 mkdir -p $TOOLCHAIN
 $1/build/tools/make-standalone-toolchain.sh \
     --toolchain=$TOOLCHAIN_NAME-$CC_VER \
     --arch=$ARCH \
     --install-dir=$TOOLCHAIN \
-    --platform=android-21
+    --platform=android-21 --force
 export PATH=$TOOLCHAIN/bin:$PATH
 export AR=$TOOLCHAIN/bin/$SUFFIX-ar
 export CC=$TOOLCHAIN/bin/$SUFFIX-gcc
@@ -62,5 +79,6 @@
         --dest-cpu=$DEST_CPU \
         --dest-os=android \
         --without-snapshot \
-        --openssl-no-asm
+        --openssl-no-asm \
+        --without-intl
 fi

for node7 v7.3.0 (latest release). We have to find a way to make it universal for the trunk tree. As mentioned before, a native build for the host machine is needed as a prerequisite.

Best, Arturo

@unional

This comment has been minimized.

Show comment
Hide comment
@unional

unional Jun 30, 2017

I'm trying to build node for mips on centos with a different error:

node-8.1.3/out/Release/mkpeephole: cannot execute binary file

Is this related?

unional commented Jun 30, 2017

I'm trying to build node for mips on centos with a different error:

node-8.1.3/out/Release/mkpeephole: cannot execute binary file

Is this related?

@piranna

This comment has been minimized.

Show comment
Hide comment
@piranna

piranna Jun 30, 2017

Contributor
Contributor

piranna commented Jun 30, 2017

@piranna

This comment has been minimized.

Show comment
Hide comment
@piranna

piranna Jul 22, 2017

Contributor

Seems this patch from @nxhack fix it, can some of the maintainers point out if it's in the correct way?

Contributor

piranna commented Jul 22, 2017

Seems this patch from @nxhack fix it, can some of the maintainers point out if it's in the correct way?

@ggkitsas

This comment has been minimized.

Show comment
Hide comment
@ggkitsas

ggkitsas Jul 24, 2017

@piranna: I think this patch is also accompanied with these patches performed over the node's sources (per node version).

ggkitsas commented Jul 24, 2017

@piranna: I think this patch is also accompanied with these patches performed over the node's sources (per node version).

@piranna

This comment has been minimized.

Show comment
Hide comment
@piranna

piranna Jul 24, 2017

Contributor

I'm not sure, but now you've make me doubt... Maybe @nxhack should do some pull-requests for them?

Contributor

piranna commented Jul 24, 2017

I'm not sure, but now you've make me doubt... Maybe @nxhack should do some pull-requests for them?

@nxhack

This comment has been minimized.

Show comment
Hide comment
@nxhack

nxhack Aug 21, 2017

It seems that patch related to 'mkpeephole' becomes unnecessary in node.js 8.3.0 and later releases.

nxhack commented Aug 21, 2017

It seems that patch related to 'mkpeephole' becomes unnecessary in node.js 8.3.0 and later releases.

@piranna

This comment has been minimized.

Show comment
Hide comment
@piranna

piranna Aug 21, 2017

Contributor

It seems that patch related to 'mkpeephole' becomes unnecessary in node.js 8.3.0 and later releases.

What do you mean? Does it works now? I'm still getting build fails...

Contributor

piranna commented Aug 21, 2017

It seems that patch related to 'mkpeephole' becomes unnecessary in node.js 8.3.0 and later releases.

What do you mean? Does it works now? I'm still getting build fails...

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Aug 21, 2017

Member

mkpeephole no longer exists in v8.3.0 and up.

Member

bnoordhuis commented Aug 21, 2017

mkpeephole no longer exists in v8.3.0 and up.

@piranna

This comment has been minimized.

Show comment
Hide comment
@piranna

piranna Aug 21, 2017

Contributor

It's still fetching incorectly the host cpu from the toolchain, but efectively, I can confirm that now cross-compiling works again.

Contributor

piranna commented Aug 21, 2017

It's still fetching incorectly the host cpu from the toolchain, but efectively, I can confirm that now cross-compiling works again.

@targos

This comment has been minimized.

Show comment
Hide comment
@targos

targos Oct 27, 2017

Member

v7.x is not supported anymore and if I'm reading this correctly, the issue does not affect other supported releases. Closing.

Member

targos commented Oct 27, 2017

v7.x is not supported anymore and if I'm reading this correctly, the issue does not affect other supported releases. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment