Make the necessary changes to facilitate ARM cross-compilation #85

Closed
wants to merge 4 commits into from

5 participants

@ignisf
Collaborator

Add ARM cross-compilation support to the libv8 gem. Note the discussion in #79.

@ignisf ignisf was assigned Feb 28, 2013
@ignisf
Collaborator

Hmm... I suspect that using ENV['TARGET'] and ENV['CXX'] anywhere else but in the Rakefile might cause some unexpected issues. I think I'll alter it to with_config 'cxx' and with_config 'target'.

ignisf added some commits Mar 10, 2013
@ignisf ignisf Re-implement architecture detection
Drop support for Rubies before 1.8.6 and use RbConfig::CONFIG['target']
for reliable architecture detection.

Add support for manual target specification.
3cac93f
@ignisf ignisf Use RbConfig::CONFIG['target'] instead of RUBY_PLATFORM 002aa62
@ignisf ignisf Add support for choosing a compiler via the CXX env variable 9a4b513
@ignisf ignisf Add cross-compilation instructions ceb2f11
@cpg

+1 on this one.

I have hit this issue when trying to bundle this gem on an ARM system (natively, not cross compiling). The system is using the official Fedora 18 for ARM. Since this is a little inside our install scripts I cannot easily apply this patch and test it, but eventually I will he able to test.

This is the error:

...
creating Makefile
Using compiler: /usr/bin/g++
In file included from ../src/allocation.h:31:0,
                 from ../src/allocation.cc:28:
../src/globals.h:113:2: error: #error Target architecture ia32 is only supported on ia32 host
gmake[1]: *** [/home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.13/vendor/v8/out/ia32.release/obj.target/preparser_lib/src/allocation.o] Error 1
gmake: *** [ia32.release] Error 2
/home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.13/ext/libv8/location.rb:36:in `block in verify_installation!': libv8 did not install properly, expected binary v8 archive '/home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.13/vendor/v8/out/ia32.release/obj.target/tools/gyp/libv8_base.a'to exist, but it was not found (Libv8::Location::Vendor::ArchiveNotFound)
    from /home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.13/ext/libv8/location.rb:35:in `each'
    from /home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.13/ext/libv8/location.rb:35:in `verify_installation!'
    from /home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.13/ext/libv8/location.rb:26:in `install!'
    from extconf.rb:7:in `<main>'
GYP_GENERATORS=make \
build/gyp/gyp --generator-output="out" build/all.gyp \
              -Ibuild/standalone.gypi --depth=. \
              -Dv8_target_arch=ia32 \
              -S.ia32  -Dv8_can_use_vfp_instructions=true
gmake[1]: Entering directory `/home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.13/vendor/v8/out'
  CXX(target) /home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.13/vendor/v8/out/ia32.release/obj.target/preparser_lib/src/allocation.o
gmake[1]: Leaving directory `/home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.13/vendor/v8/out'
@ignisf
Collaborator

Hello @cpg,

We have support for native arm compilation. However, we have not tested it on Fedora. Would you care to provide us with the values of RbConfig::MAKEFILE_CONFIG['build_cpu'] and RbConfig::CONFIG['target']?

@cpg

=> "armv5tel"

(note, the hardware is the raspberry pi)

@ignisf
Collaborator

@cpg, I just noticed that you attempted to install 3.11.8.13. This one has only x86 and x86_64 support. The next version 3.11.8.14 added support for native arm compilation (tested on the Raspbian with hard floating point). I would be very grateful if you could spare me the installation of Fedora in my very slow emulator and test compiling this branch (arm-crosscompile) as well as the master branch. Though the branch name contains 'crosscompile' it actually introduces new architecture detection as well as crosscompilation functionality.

@cpg

For some (probably historic) reason, our Gemfile contained this:

gem "libv8", "~> 3.11.8.4"

when changed to gem "libv8", "~> 3.11.8.14", we get this:

Installing libv8 (3.11.8.17) .............

Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.

        /usr/bin/ruby extconf.rb
creating Makefile
Compiling v8 for ia32
Using python 2.7.3
Using compiler: /usr/bin/g++
In file included from ../src/allocation.h:31:0,
                 from ../src/allocation.cc:28:
../src/globals.h:113:2: error: #error Target architecture ia32 is only supported on ia32 host
gmake[1]: *** [/home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.17/vendor/v8/out/ia32.release/obj.target/preparser_lib/src/allocation.o] Error 1
gmake: *** [ia32.release] Error 2
/home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.17/ext/libv8/location.rb:36:in `block in verify_installation!': libv8 did not install properly, expected binary v8 archive '/home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.17/vendor/v8/out/ia32.release/obj.target/tools/gyp/libv8_base.a'to exist, but it was not found (Libv8::Location::Vendor::ArchiveNotFound)
    from /home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.17/ext/libv8/location.rb:35:in `each'
    from /home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.17/ext/libv8/location.rb:35:in `verify_installation!'
    from /home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.17/ext/libv8/location.rb:26:in `install!'
    from extconf.rb:7:in `<main>'
GYP_GENERATORS=make \
build/gyp/gyp --generator-output="out" build/all.gyp \
              -Ibuild/standalone.gypi --depth=. \
              -Dv8_target_arch=ia32 \
              -S.ia32  -Dv8_can_use_vfp_instructions=false
gmake[1]: Entering directory `/home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.17/vendor/v8/out'
  CXX(target) /home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.17/vendor/v8/out/ia32.release/obj.target/preparser_lib/src/allocation.o
gmake[1]: Leaving directory `/home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.17/vendor/v8/out'


Gem files will remain installed in /home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.17 for inspection.
Results logged to /home/cpg/.gem/ruby/1.9.1/gems/libv8-3.11.8.17/ext/libv8/gem_make.out

An error occurred while installing libv8 (3.11.8.17), and Bundler cannot continue.
Make sure that `gem install libv8 -v '3.11.8.17'` succeeds before bundling.

As far as testing master, we tried using :git => ... , but it failed with problems with deps with therubyracer.

Could not find gem 'libv8 (~> 3.11.8.12) ruby', which is required by gem 'therubyracer (>= 0) ruby', in any of the
sources.

When using :git => .. in therubyracer, it failed as well for similar reasons. Any tips?

Also, any tips on how to bundle a branch?

FWIW, this is the code we're trying to bundle: https://github.com/amahi/platform

@ignisf
Collaborator

About using this gem directly from the git repo: #89

For now if one was to test the git version, he/she would have to:

git clone git://github.com/cowboyd/libv8.git
cd libv8
git checkout arm-crosscompile
bundle install
bundle exec rake checkout compile

Then bundle exec rake binary to create a binary version of the gem or bundle exec rake build -- for a source version. Both of the packages will be stored in the pkg directory and are installable via gem install pkg/file_name.gem.

P.S. Keep in mind that the compilation process takes a long time on the Pi. That's why we're going for cross-compilation instead :)

@ignisf
Collaborator

Also, does Fedora provide a v8 package? If it does, you can also try this.

@cpg

Fedora 18 does in fact provide a v8 and v8-devel. And using the system v8 worked (as far as bundling -- hope it works when running the app!).

The git version takes a while to compile, but we're going to try with 3.11.8.17 and the system v8.

Thanks!

@cpg

Fedora 18 does in fact provide a v8 and v8-devel. And using the system v8 worked (as far as bundling -- hope it works when running the app!).

The git version takes a while to compile, but we're going to try with 3.11.8.17 and the system v8.

Thanks!

@glennmartinez

I Get the same issue on a Samsung chromebook arm with ubuntu 12.04.

What do i do? i'm on 3.11.8.17?

@ignisf
Collaborator

@glennmartinez did you try using the system libv8 as @cpg did?

@glennmartinez

How do I run the system libv8? is it the instructions by yourself above?

I ran the instructions from above about getting the repo version and it wouldn't compile.

@cpg

In my case, installing the rpms v8 and v8-devel is what i did.

@glennmartinez

Hey System did the job, now rubyrace is failing?

here's the error message, any tips?

Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.

/home/glenn/.rvm/rubies/ruby-1.9.3-p392/bin/ruby extconf.rb 

checking for main() in -lpthread... yes
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.

Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/home/glenn/.rvm/rubies/ruby-1.9.3-p392/bin/ruby
--with-pthreadlib
--without-pthreadlib
--enable-debug
--disable-debug
/home/glenn/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/site_ruby/1.9.1/rubygems/core_ext/kernel_require.rb:110:in require': cannot load such file -- libv8 (LoadError)
from /home/glenn/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/site_ruby/1.9.1/rubygems/core_ext/kernel_require.rb:110:in
rescue in require'
from /home/glenn/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/site_ruby/1.9.1/rubygems/core_ext/kernel_require.rb:35:in require'
from extconf.rb:29:in

'

Gem files will remain installed in /home/glenn/.rvm/gems/ruby-1.9.3-p392/gems/therubyracer-0.11.4 for inspection.
Results logged to /home/glenn/.rvm/gems/ruby-1.9.3-p392/gems/therubyracer-0.11.4/ext/v8/gem_make.out

@glennmartinez

latest error when running with 16.

libv8 (3.11.8.16)
glenn@localhost:~/Workspace$ gem install therubyracer
Building native extensions. This could take a while...
ERROR: Error installing therubyracer:
ERROR: Failed to build gem native extension.

/home/glenn/.rvm/rubies/ruby-1.9.3-p392/bin/ruby extconf.rb

checking for main() in -lpthread... yes
checking for v8.h... yes
creating Makefile

make
compiling handles.cc
compiling value.cc
compiling template.cc
compiling script.cc
compiling stack.cc
compiling trycatch.cc
compiling constants.cc
compiling function.cc
function.cc: In static member function ‘static VALUE rr::Function::GetInferredName(VALUE)’:
function.cc:40:34: error: ‘class v8::Function’ has no member named ‘GetInferredName’
function.cc:41:3: warning: control reaches end of non-void function [-Wreturn-type]
make: *** [function.o] Error 1

@kyrofa

What is the status of this? Obviously it's still in its own branch-- is there a reason this has not been merged to master and pushed out to rubygems? Using the system V8 is not a great workaround for Debian users since Debian is way behind the times with their V8 packages and rubyracer won't compile with it. I'm currently compiling this branch natively on my Mirabox-- no errors yet.

@ignisf
Collaborator

Hi @kyle-f,

I am kind of hesitant to merge it because I do not have enough feedback about it.

@kyrofa

@ignisf,

Ah, okay that makes sense. And actually the compile ended up failing for me a few minutes ago with the "For thumb inter-working we require an architecture which supports blx," so I agree with you. I got that error before when I was compiling node.js from source and just hacked around it by commenting out that error (since my platform supports thumb so there was just a define issue). Let me dig into it a bit more.

@ignisf
Collaborator

Would you please leave information about your system in #98?

@ignisf
Collaborator

Also, did you try compiling the latest master version? It would be awesome to see if and where it fails.

@kyrofa

@ignisf,

I left my info on #98. Also, I did try compiling master, and I got a ton of errors like this:

/usr/bin/ld: failed to merge target specific data of file /home/user/src/gems/libv8/vendor/v8/out/arm.release/obj.target/tools/gyp/libpreparser_lib.a(preparser-api.o)
/usr/bin/ld: error: /home/user/src/gems/libv8/vendor/v8/out/arm.release/obj.target/tools/gyp/libpreparser_lib.a(scanner.o) uses VFP register arguments, /home/user/src/gems/libv8/vendor/v8/out/arm.release/preparser does not

followed by the obvious Libv8::Location::Vendor::ArchiveNotFound error.

Note my cpuinfo:

Processor       : Marvell PJ4Bv7 Processor�� rev 1 (v7l)
BogoMIPS        : 1199.30
Features        : swp half thumb fastmult vfp edsp vfpv3 vfpv3d16 
CPU implementer : 0x56
CPU architecture: 7
CPU variant     : 0x1
CPU part        : 0x581
CPU revision    : 1

Hardware        : Marvell Armada-370
Revision        : 0000
Serial          : 0000000000000000

I know that has something to do with the hard vs. soft float stuff. For what it's worth, I was able to compile the V8 included with node.js by adding -march=armv7l -mthumb-interwork -mfloat-abi=softfp -mfpu=vfpv3 to the CC/CXXFLAGS, adding 'armv7%': 0 to the gyp variables, and by literally commenting out the "For thumb inter-working we require an architecture which supports blx" preprocessor error.

If you've got stuff you would specifically like me to test, please let me know. I'm all yours.

@ignisf
Collaborator

there are a bunch of fixes for arm autodetection in the new upstream v8 version. Will close this and try again in the future

@ignisf ignisf closed this Aug 15, 2013
@badboyjames

Is it fixed or was just closed?

@kyrofa

@badboyjames: Just closed. I have that MiraBox platform and an arm system up in QEMU. I would totally be a test guinea pig for this if someone would let me and tell me what to test!

@ignisf
Collaborator

Let's continue this discussion in an issue that is still open.
#98 (comment)

@ignisf ignisf deleted the arm-crosscompile branch Jun 17, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment