Unable to compile v0.4.0 on armhf architecture

[endemics]

Hi,

I have been unsuccessful so far in my attempts to compile ruby-packer v0.4.0 on a raspberry pi running raspbian (architecture armhf).

I am able to compile ruby from sources on the same setup (using ./configure && make), as well as able to use the same Docker image on x86_64 to build a working rubyc binary.

What goes wrong?

Compilation on armhf always fails at the following:

(...)
linking miniruby
generating encdb.h
encdb.h updated
./tool/ifchange "--timestamp=.rbconfig.time" rbconfig.rb rbconfig.tmp
rbconfig.rb updated
generating enc.mk
./enc/make_encmake.rb:59:in `sort_by': comparison of Array with Array failed (ArgumentError)
    from ./enc/make_encmake.rb:59:in `target_encodings'
    from ./enc/make_encmake.rb:120:in `<main>'
uncommon.mk:707: recipe for target 'enc.mk' failed
make: *** [enc.mk] Error 1
Failed running [{"CI"=>"true", "ENCLOSE_IO_USE_ORIGINAL_RUBY"=>"1", "CFLAGS"=>" -fPIC -O3 -fno-fast-math -ggdb3 -Os -fdata-sections -ffunction-sections -pipe  -I/tmp/rubyc/zlib  -I/tmp/rubyc/openssl/include  -I/tmp/rubyc/gdbm/build/include  -I/tmp/rubyc/yaml/build/include  -I/tmp/rubyc/libffi/build/lib/libffi-3.2.1/include  -I/tmp/rubyc/ncurses/build/include  -I/tmp/rubyc/readline/build/include ", "LDFLAGS"=>" -L/tmp/rubyc/zlib /tmp/rubyc/zlib/libz.a  -L/tmp/rubyc/openssl  -L/tmp/rubyc/gdbm/build/lib  -L/tmp/rubyc/yaml/build/lib  -L/tmp/rubyc/libffi/build/lib  -L/tmp/rubyc/ncurses/build/lib  -L/tmp/rubyc/readline/build/lib ", "ENCLOSE_IO_RUBYC_1ST_PASS"=>"1", "ENCLOSE_IO_RUBYC_2ND_PASS"=>nil}, "make -j4 -j1"]

My analysis so far

I wasn't sure if that was a spurious message/red herring, so I have compared the build logs between x86_64 and armhf.

I didn't find any error message on armhf that wasn't on x86_64 (I can provide the extensive logs if required).

So I came to the conclusion there wasn't something obviously missing or failing at an earlier step in either case, especially since the x86_64 build successfully finishes and produce a working rubyc binary as stated above.

Note that I have tried several versions of ruby-packer but was only successful in building a functional rubyc on x86_64 when using v0.4.0, which is why I am reporting the issue for an old version of the project.

Now, looking more specifically at the issue, I found that the enc.mk file was empty.

I traced it to a problem with the generation of the encs array at L55 of ruby/enc/make_encmake.rb: when running from miniruby on armhf, the Dir.open command returns some truncated file names on the armhf build while it works on the x86_64 builds. This is what leads to a functional enc.mk file or not.

I was able to create a simple test.rb file (see below) to run a comparative test between x86_64 and armhf. Results are in the attached files below:

test_results_armhf.txt
test_results_x86.txt

Since the vanilla ruby build on armhf work, I am guessing that this issue does not happen with it, hence me reporting to the ruby-packer project.

Additionally I was unable to find anything related to miniruby failing on armhf while googling.

My attempts to hard-code the values for the encs, atrans and trans variables in the make_encmake.rb file to try and progress in the build were not really conclusive.

How to reproduce?

While I was originally working on Debian stretch based Docker images, I have tried to reproduce it with a docker image that would be as close as the travisci image as possible so here is what I ended up with:

FROM ubuntu:xenial
RUN apt-get update && \
    apt-get install -y \
        git \
        bison \
        wget \
        squashfs-tools \
        build-essential \
        texinfo \
        software-properties-common
RUN apt-add-repository -y ppa:ubuntu-toolchain-r/ppa && \
    apt-add-repository -y ppa:rael-gc/rvm && \
    apt-get update && \
    apt-get install -y \
        gcc-4.8 \
        g++-4.8 \
        rvm
WORKDIR /tmp
RUN /usr/share/rvm/bin/rvm install --disable-binary ruby-2.4
RUN git clone https://github.com/pmq20/ruby-packer.git
WORKDIR /tmp/ruby-packer
RUN git checkout v0.4.0
#RUN bash -c 'source "/usr/share/rvm/scripts/rvm" && \
#    rvm use ruby-2.4 && \
#    export CPPFLAGS="-P" && \
#    ruby -Ilib bin/rubyc bin/rubyc -o rubyc'

Now, the last RUN line is commented as it fails on armhf, but it runs fine and produce a working image on x86_64.

The way to get a test environment is to build the image, and then run the container then trying to build rubyc using source "/usr/share/rvm/scripts/rvm" && rvm use ruby-2.4 && export CPPFLAGS="-P" && ruby -Ilib bin/rubyc bin/rubyc -o rubyc.

You can then try and run the following test.rb:

# test.rb
puts Dir.open('/tmp/rubyc/ruby-2.4.1-0.4.0/enc/') {|d| d.grep(/.+\.c\z/)}
puts '--'
puts Dir.open('/tmp/rubyc/ruby-2.4.1-0.4.0/enc/') {|d| d.grep(/.*/)}

Where to?

Any help from people more experienced with the project or a better understanding of compilers would be welcome.

If a later commit is known to produce some working a.out from gems I would be more than happy to try it too, as ultimately I am just trying to produce a self-contained binary for a gem on armhf as a one-off :)

[pmq20]

@endemics Thank you very much for the detailed bug report! However, currently only x64 is supported. I am personally not able to fix this as I do not have a armhf machine. Help from other people and Pull Requests are welcome!

Related: pmq20/node-packer#141

[endemics]

@endemics Thank you very much for the detailed bug report! However, currently only x64 is supported. I am personally not able to fix this as I do not have a armhf machine. Help from other people and Pull Requests are welcome!

Thanks for the answer Minqi! Unfortunately I won't be able to help with the patch but sure can help by testing if anyone feel like trying to solve this. Also, if you are interested in trying to do the port yourself to armhf/aarch64 let me know and I can try to arrange to get you a raspberry pi 3.