Can't compile crystal --release

[joshgoebel]

I'm trying to build crystal. I have all the dependencies via homebrew. I've tried compiling both 0.20.5 and HEAD (15032cc). I'm using the x86 build provided in the 0.20.5 release for the bootstrap. I can build the compiler fine, but NOT with --release. It seems to hang forever on Codegen (bc+obj) (when running stats):

Parse:                             00:00:00.0005260 (   0.19MB)
Semantic (top level):              00:00:00.2484970 (  76.07MB)
Semantic (new):                    00:00:00.0033650 (  84.07MB)
Semantic (type declarations):      00:00:00.0356490 (  84.07MB)
Semantic (abstract def check):     00:00:00.0009560 (  84.07MB)
Semantic (cvars initializers):     00:00:00.0084090 (  92.07MB)
Semantic (ivars initializers):     00:00:00.0300400 ( 100.07MB)
Semantic (main):                   00:00:09.0334310 ( 828.50MB)
Semantic (cleanup):                00:00:00.0010840 ( 828.50MB)
Semantic (recursive struct check): 00:00:00.0016440 ( 828.50MB)
Codegen (crystal):                 00:00:04.0398970 ( 965.00MB)
Codegen (bc+obj):      

The binary compiler I'm using PATH="../crystal-0.20.5-1/bin:$PATH" ./bin/crystal -v:

Crystal 0.20.5 [ccf46c0] (2017-01-20)

What I'm running exactly:

CC=clang-3.8 CXX=clang++-3.8 PATH="../crystal-0.20.5-1/bin:$PATH" make crystal release=true stats=true 

Previously I was using the latest LLVM (3.9.1) from homebrew. Going back to 3.8 doesn't seem to have had any noticeable effect.

I get them same hang when trying brew install crystal-lang. brew install crystal-lang --without-release works just fine.

Please advise. Any help would be appreciated.

[joshgoebel]

I can compile a simple hello_world (with a puts statement) with --release (with the 0.20.5 binary).

[joshgoebel]

Ok, this might be a false alert. Someone told me it should take a few minutes. I finally got it working after 30 minutes of waiting. Will try again on LLVM 3.9 next.

[ysbaddaden]

Yes, bc+obj will execute all LLVM optimisations in aggressive mode, but 30 minutes is a lot. What happens if you rebuild Crystal, using the just built compiler? How long does it take?

[ysbaddaden]

Note that the optimisation step is slower and slower on each LLVM release.

[joshgoebel]

Ok, so I'm building 0.20.5 --release WITH 0.20.5 (binary from github releases) with LLVM 3.9.1.

• First compile (using downloaded binary) - 30 minutes, 48mb binary
• Second compile (using just compiled binary) - 6 minutes, 34mb binary
• Third compile (using just compiled binary) - 6 minutes, 34mb binary

Why is the distributed compiler showing such "poor" performance vs the one I built?

Also interesting: The second and 3rd builds are exactly the same size (down to the byte) but they are not matching binaries.

[ysbaddaden]

Maybe LLVM took wrong assumptions when tuning optimizations for your CPU? Or maybe this is how LLVM itself is built (it's statically linked)? I'm not sure which LLVM is used for osx packages or homebrew (built manually? homebrew's one)?

Around 5 or 6 minutes is expected with LLVM 3.9. I have no idea why the difference. I know it happened to me when cross-compiling to another OS (e.g. FreeBSD or OpenBSD). It's often quicker to build without --release build on the platform, then build with --release using the unoptimized build.

[joshgoebel]

I was using the x86 binary provided from github for 0.20.5... this is all on my own outside of homebrew (homebrew is just providing the dependencies).

It's often quicker to build without --release build on the platform, then build with --release using the unoptimized build.

Now that is interesting. I wonder how hard it would be to make homebrew do that.

[ysbaddaden]

You misunderstood me: it's better to use an optimized build in the whole majority of cases! But in some unidentified cases, the initial build may take a very long time, so first building without optimizations then with optimizations may be faster, but only when it takes forever the first time.

[joshgoebel]

Guess I'm having trouble believing my experience is that much of an edge case. :-)

[joshgoebel]

I'll close this now though since the only issue was how slow it was, not that the actual compile was broken.

[asterite]

@yyyc514 Regarding your three separate compiles, what happens if you delete ~/.cache/crystal in between? We cache .o files between successive compilations and try to reuse them, so if you delete the cache chances are that it will take 30 minutes again.

Compiling the compiler with --release used to take 5 minutes a long time ago, then it went up to 10, 15, and now 30 minutes. This is with LLVM 3.5.1 (we are using that in omnibus) so I guess with newer LLVM versions it would be slower. We never really profiled LLVM or spent time understanding how, and if, we can generate better LLVM IR code for faster compilation times, but it would be nice to do so as soon as possible to save time.

In any case, we only compile the compiler in release mode for releases, and waiting a few hours there (because we compile for several platforms) isn't that terrible.

[RX14]

Compiling the compiler cleanly with --release takes only 6 minutes for me:

[rx14@RX-MAIN1 crystal]$ rm -Rf ~/.cache/crystal
[rx14@RX-MAIN1 crystal]$ make clean
Using /usr/bin/llvm-config [version=3.9.1]
rm -rf .build
rm -rf ./doc
rm -rf src/llvm/ext/llvm_ext.o
rm -rf src/ext/sigfault.o src/ext/libcrystal.a
[rx14@RX-MAIN1 crystal]$ make deps
Using /usr/bin/llvm-config [version=3.9.1]
g++ -c  -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc `/usr/bin/llvm-config --cxxflags`
cc -fPIC    -c -o src/ext/sigfault.o src/ext/sigfault.c
ar -rcs src/ext/libcrystal.a src/ext/sigfault.o
[rx14@RX-MAIN1 crystal]$ time make crystal release=true stats=true
Using /usr/bin/llvm-config [version=3.9.1]
./bin/crystal build --release --stats  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Parse:                             00:00:00.0000715 (   0.19MB)
Semantic (top level):              00:00:00.2509239 (  77.89MB)
Semantic (new):                    00:00:00.0019469 (  77.89MB)
Semantic (type declarations):      00:00:00.0300041 (  85.89MB)
Semantic (abstract def check):     00:00:00.0008288 (  85.89MB)
Semantic (cvars initializers):     00:00:00.0079509 (  93.89MB)
Semantic (ivars initializers):     00:00:00.0268399 ( 101.89MB)
Semantic (main):                   00:00:07.7891221 ( 822.32MB)
Semantic (cleanup):                00:00:00.0006955 ( 822.32MB)
Semantic (recursive struct check): 00:00:00.0008452 ( 822.32MB)
Codegen (crystal):                 00:00:03.4397810 ( 958.82MB)
Codegen (bc+obj):                  00:06:04.3448058 ( 958.82MB)
Codegen (linking):                 00:00:00.1886388 ( 958.82MB)

real	6m16.503s
user	6m15.487s
sys	0m0.933s

I don't know what's going on here but I don't think its fair to blame LLVM.

[asterite]

Hey, it's true! On my machine:

Parse:                             00:00:00.0007690 (   0.19MB)
Semantic (top level):              00:00:00.3402780 (  75.46MB)
Semantic (new):                    00:00:00.0031380 (  83.46MB)
Semantic (type declarations):      00:00:00.0493130 (  91.46MB)
Semantic (abstract def check):     00:00:00.0010760 (  91.46MB)
Semantic (cvars initializers):     00:00:00.0137940 (  91.46MB)
Semantic (ivars initializers):     00:00:00.0500500 (  99.46MB)
Semantic (main):                   00:00:10.6461620 ( 812.40MB)
Semantic (cleanup):                00:00:00.0010710 ( 812.40MB)
Semantic (recursive struct check): 00:00:00.0013920 ( 812.40MB)
Codegen (crystal):                 00:00:07.3254350 ( 859.40MB)
Codegen (bc+obj):                  00:07:03.1885600 ( 859.40MB)
Codegen (linking):                 00:00:00.9332070 ( 859.40MB)

real	7m22.868s
user	7m25.764s
sys	0m2.558s

But with the binary extracted from the tar.gz in the release page:

Parse:                             00:00:00.0007320 (   0.19MB)
Semantic (top level):              00:00:00.3520230 (  76.05MB)
Semantic (new):                    00:00:00.0032800 (  84.05MB)
Semantic (type declarations):      00:00:00.0507240 (  84.05MB)
Semantic (abstract def check):     00:00:00.0011250 (  84.05MB)
Semantic (cvars initializers):     00:00:00.0114620 (  92.05MB)
Semantic (ivars initializers):     00:00:00.0497820 ( 100.05MB)
Semantic (main):                   00:00:10.1725670 ( 788.49MB)
Semantic (cleanup):                00:00:00.0012000 ( 788.49MB)
Semantic (recursive struct check): 00:00:00.0014900 ( 796.49MB)
Codegen (crystal):                 00:00:06.2847340 ( 863.99MB)
Codegen (bc+obj):                  00:27:51.1123820 ( 863.99MB)
Codegen (linking):                 00:00:01.1741650 ( 863.99MB)

real	28m9.642s
user	27m59.989s
sys	0m12.751s

That gives a light of hope: I thought that the compile times went higher because the compiler size grew, but I originally expected it would grow linearly. And it seems to be the case, there's something strange with the one we distribute. It might be because it's statically linked against a patched LLVM 3.5.1... maybe it was built with assertions and without optimizations, but I don't know, I think this was used to generate the LLVM binaries we have in s3.

[joshgoebel]

Should I create a new issue for slow compile time with the release binary?

[mosop]

I experienced the similar problem. Three cross-compile builds took all the night...

my build script:

ver=0.20.5
git clone --depth=1 -b $ver https://github.com/crystal-lang/crystal.git
cd crystal
LLVM_CONFIG=llvm-config-3.8 CRYSTAL_CONFIG_VERSION=$ver crystal build -o crystal src/compiler/crystal.cr \
      -D i_know_what_im_doing -D without_openssl -D without_zlib \
      --release \
      --cross-compile --target $target \
      --stats --verbose

The targets are x86_64-alpine-linux-musl, x86_64-apple-darwin and x86_64-unknown-linux-gnu.

Both on Mac yosemite 2015 8GB RAM and Ubuntu 14.04.05 4GB RAM (dockerized).
FYI.