build fails on Raspberry Pi 3 with "LLVM ERROR: out of memory"

[josefsachsconning]

Fresh clone of julia master at bfb1c1b
This is with a 4 GB swap file. My last successful build was at f965fd4 on Feb 14 with a 1 GB swap file.
Does it just no longer work on a 32-bit OS?

Stdlibs total  ─────────1361.346640 seconds
/home/sachs/src/julia-master/base/precompile.jl
Sysimage built. Summary:
Total ─────── 2067.502890 seconds
Base: ─────── 428.275362 seconds 20.7146%
Stdlibs: ──── 1361.346640 seconds 65.845%
Precompile: ─ 277.854516 seconds 13.4391%
LLVM ERROR: out of memory

signal (6): Aborted
in expression starting at no file:0
raise at /build/glibc-6f8a9a/glibc-2.19/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:56
Allocations: 223152989 (Pool: 223124343; Big: 28646); GC: 1410
Aborted
*** This error is usually fixed by running `make clean`. If the error persists, try `make cleanall`. ***
Makefile:203: recipe for target '/home/sachs/src/julia-master/usr/lib/julia/sys-o.a' failed
make[1]: *** [/home/sachs/src/julia-master/usr/lib/julia/sys-o.a] Error 1
Makefile:78: recipe for target 'julia-sysimg-release' failed
make: *** [julia-sysimg-release] Error 2

[vchuravy]

Does it just no longer work on a 32-bit OS?

No it should still work on 32bit OS, but it seems that either the memory pressure is getting to high or we are running into a different problem on armv7-a.

[its-pointless]

As im compiling this on android im also seeing this in arm.

[EMCP]

just tried on a RPI3 running rasbian with 5d4eaca AKA v1.0.0 .. got something else.. going to try make clean and rerun

client.jl
docs/Docs.jl
Base  ───────────210.699773 seconds
Base64  ───────── 36.217664 seconds
CRC32c  ─────────  0.057772 seconds
SHA  ────────────  1.439312 seconds
FileWatching  ───  0.731206 seconds
Unicode  ────────  0.047768 seconds
Mmap  ───────────  0.623913 seconds
Serialization  ── 10.913070 seconds
Libdl  ──────────  0.196170 seconds
Markdown  ─────── 10.546536 seconds
LibGit2  ──────── 38.753154 seconds
Logging  ────────  5.544823 seconds
Sockets  ──────── 17.911183 seconds
Printf  ─────────  0.046482 seconds
Profile  ────────  1.727453 seconds
Dates  ────────── 16.777460 seconds
DelimitedFiles  ─  0.861232 seconds
Random  ─────────  7.316024 seconds
UUIDs  ──────────  0.048344 seconds
Future  ─────────  0.038443 seconds
LinearAlgebra  ──114.642041 seconds
SparseArrays  ─── 31.908507 seconds
SuiteSparse  ────  8.359849 seconds
Distributed  ──── 53.358976 seconds
SharedArrays  ───  1.257814 seconds
Pkg  ────────────288.196439 seconds
Test  ───────────  7.063168 seconds
REPL  ─────────── 10.858876 seconds
Statistics  ─────  3.164944 seconds
Stdlibs total  ──669.827907 seconds
Sysimage built. Summary:
Total ─────── 880.546288 seconds 
Base: ─────── 210.699773 seconds 23.9283%
Stdlibs: ──── 669.827907 seconds 76.0696%
    JULIA usr/lib/julia/sys-o.a
Generating precompile statements... 790 generated in 393.981056 seconds
Killed
*** This error is usually fixed by running `make clean`. If the error persists, try `make cleanall`. ***
Makefile:216: recipe for target '/home/pi/dev/git/julialang/julia/usr/lib/julia/sys-o.a' failed
make[1]: *** [/home/pi/dev/git/julialang/julia/usr/lib/julia/sys-o.a] Error 1
Makefile:78: recipe for target 'julia-sysimg-release' failed
make: *** [julia-sysimg-release] Error 2

[its-pointless]

Are you using system llvm or using patches supplied by julia?

[EMCP]

Update: re-ran make and make clean... even make cleanall , nothing changed.

@its-pointless just ran make inside the project.. I am unsure what the default behavior is.

this was also on a very stock Pi .. so I would doubt it's the patches unless that's the default when running make

update 2: will now try make -C deps distclean-llvm and report back

[KristofferC]

Put an exit(0) on the top of generate_precompile.jl and see if that helps (should reduce precompilation and this might reduce memory to build the sysimg).

[EMCP]

@KristofferC thank you for the tip, however it appeared to not affect the error

deprecated.jl
docs/basedocs.jl
client.jl
docs/Docs.jl
Base  ───────────210.557411 seconds
Base64  ───────── 35.914330 seconds
CRC32c  ─────────  0.057648 seconds
SHA  ────────────  1.428127 seconds
FileWatching  ───  0.723315 seconds
Unicode  ────────  0.047662 seconds
Mmap  ───────────  0.619192 seconds
Serialization  ── 10.794706 seconds
Libdl  ──────────  0.194784 seconds
Markdown  ─────── 10.466102 seconds
LibGit2  ──────── 38.408924 seconds
Logging  ────────  5.507085 seconds
Sockets  ──────── 17.745917 seconds
Printf  ─────────  0.046769 seconds
Profile  ────────  1.706760 seconds
Dates  ────────── 16.655646 seconds
DelimitedFiles  ─  0.855479 seconds
Random  ─────────  7.246835 seconds
UUIDs  ──────────  0.048598 seconds
Future  ─────────  0.038368 seconds
LinearAlgebra  ──113.896223 seconds
SparseArrays  ─── 31.736119 seconds
SuiteSparse  ────  8.299857 seconds
Distributed  ──── 53.086265 seconds
SharedArrays  ───  1.251394 seconds
Pkg  ────────────289.317990 seconds
Test  ───────────  7.035487 seconds
REPL  ─────────── 10.781064 seconds
Statistics  ─────  3.167383 seconds
Stdlibs total  ──668.286308 seconds
Sysimage built. Summary:
Total ─────── 878.862322 seconds 
Base: ─────── 210.557411 seconds 23.958%
Stdlibs: ──── 668.286308 seconds 76.0399%
    JULIA usr/lib/julia/sys-o.a
Generating precompile statements... 790 generated in 391.985117 seconds
Killed
*** This error is usually fixed by running `make clean`. If the error persists, try `make cleanall`. ***
Makefile:216: recipe for target '/home/pi/dev/git/julialang/julia/usr/lib/julia/sys-o.a' failed
make[1]: *** [/home/pi/dev/git/julialang/julia/usr/lib/julia/sys-o.a] Error 1
Makefile:78: recipe for target 'julia-sysimg-release' failed
make: *** [julia-sysimg-release] Error 2
pi@raspberrypi3:~/dev/git/julialang/julia $ 

[EMCP]

just going to go with the binary of v1.0.0 for now https://julialang-s3.julialang.org/bin/linux/armv7l/1.0/julia-1.0.0-linux-armv7l.tar.gz

EDIT: doesn't seem to like the ARMV8 environment

with the binary in path

pi@raspberrypi3:~/ $ julia mytemp.jl 
ERROR: LoadError: UndefVarError: Pkg not defined
Stacktrace:
 [1] top-level scope at none:0
 [2] include at ./boot.jl:317 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1038
 [4] include(::Module, ::String) at ./sysimg.jl:29
 [5] exec_options(::Base.JLOptions) at ./client.jl:229
 [6] _start() at ./client.jl:421
in expression starting at /home/pi/mytemp.jl:1

pi@raspberrypi3:~/ $ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.0.0 (2018-08-08)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> Pkg.add("PortAudio")
ERROR: UndefVarError: Pkg not defined
Stacktrace:
 [1] top-level scope at none:0

julia> Pkg.add("JSON")
ERROR: UndefVarError: Pkg not defined
Stacktrace:
 [1] top-level scope at none:0

then i learned I should manually remove the old ~/.julia/ ... but that had no effect

[KristofferC]

You need to import Pkg first.

[EMCP]

Ah yes, reading #28574 now

should this now be in all my scripts?

import Pkg

edit: Now it seems the package is not able to build due to not seeing JULIA_HOME .. unsure if it's the build or something I did wrong

JuliaAudio/PortAudio.jl#33

I'll move there and stop hi-hacking this build related thread

[ViralBShah]

We also provide precompiled binaries now.

[terasakisatoshi]

Hi, combination of some techniques e.g. cross compilation, Docker and GitHub repository jlcross will save our time.

[ViralBShah]

Also we provide precompiled binaries