Creating large array of mutables crashes REPL

[mauro3]

Creating an array of a mutable type which doesn't fit into memory crashes the REPL (after hanging the computer for some time):

~ >> julia5          
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.5.0-rc3+0 (2016-08-22 23:43 UTC)
 _/ |\__'_|_|_|\__'_|  |  
|__/                   |  x86_64-pc-linux-gnu

julia> type A
       a::Int
       end

julia> [A(3) for i=1:1000000000];
zsh: killed     julia5

This is on linux 64-bit with 16GB RAM and no swap. The expected behavior would be to throw a OutOfMemoryError like immutables do:

julia> immutable A
       a::Int
       end

julia> [A(3) for i=1:100000000000];
ERROR: OutOfMemoryError()
 in collect(::Base.Generator{UnitRange{Int64},##3#4}) at ./array.jl:305

Maybe related: #17061

[nalimilan]

Are you sure the OOM killer wasn't triggered (this is usually reported in system logs)? Julia has basically no control over that due to memory overcommitment.

[mauro3]

Yes, the log says:

Aug 29 13:31:03 x230 kernel: Out of memory: Kill process 16564 (julia5) score 797 or sacrifice child
Aug 29 13:31:03 x230 kernel: Killed process 16564 (julia5) total-vm:16732184kB, anon-rss:12932756kB, file-rss:0kB, shmem-rss:0kB
Aug 29 13:31:03 x230 kernel: oom_reaper: reaped process 16564 (julia5), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

Maybe this is one of the cases of smarter memory management @JeffBezanson refers to here: #14963 (comment) ?

[nalimilan]

Probably. Though I'm not sure what he meant exactly. Julia would need to choose a reasonable limit beyond which allocations should fail before triggering the OOM killer.

[musm]

fyi on Windows:

julia> type A
              a::Int
              end

julia> [A(3) for i=1:1000000000];

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: UNKNOWN at 0x7ff98390f808 -- RaiseException at C:\WINDOWS\System32\KERNELBASE.dll (unknown line)
while loading no file, in expression starting on line 0
RaiseException at C:\WINDOWS\System32\KERNELBASE.dll (unknown line)
Unwind_RaiseException at C:\Julia\Julia-0.5-latest\bin\libgcc_s_seh-1.dll (unknown line)
_cxa_throw at C:\Julia\Julia-0.5-latest\bin\libstdc++-6.dll (unknown line)
Znwy at C:\Julia\Julia-0.5-latest\bin\libstdc++-6.dll (unknown line)
ZNSt6vectorIN4llvm17DWARFDebugAranges13RangeEndpointESaIS2_EE19_M_emplace_back_auxIIRyRjbEEEvDpOT_ at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm17DWARFDebugAranges11appendRangeEjyy at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm17DWARFDebugAranges8generateEPNS_12DWARFContextE at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm20DWARFContextInMemory6anchorEv at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm12DWARFContext24getCompileUnitForAddressEy at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm12DWARFContext25getInliningInfoForAddressEyNS_19DILineInfoSpecifierE at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: UNKNOWN at 0x7ff98390f808 -- RaiseException at C:\WINDOWS\System32\KERNELBASE.dll (unknown line)
while loading no file, in expression starting on line 0
RaiseException at C:\WINDOWS\System32\KERNELBASE.dll (unknown line)
Unwind_RaiseException at C:\Julia\Julia-0.5-latest\bin\libgcc_s_seh-1.dll (unknown line)
_cxa_throw at C:\Julia\Julia-0.5-latest\bin\libstdc++-6.dll (unknown line)
Znwy at C:\Julia\Julia-0.5-latest\bin\libstdc++-6.dll (unknown line)
ZNSt6vectorIN4llvm26DWARFDebugInfoEntryMinimalESaIS1_EE7reserveEy at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm9DWARFUnit9DWOHolderC2ENS_9StringRefE at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm9DWARFUnit19extractDIEsIfNeededEb at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm9DWARFUnit23getSubprogramForAddressEy at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm9DWARFUnit25getInlinedChainForAddressEy at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm12DWARFContext25getInliningInfoForAddressEyNS_19DILineInfoSpecifierE at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: UNKNOWN at 0x7ff98390f808 -- RaiseException at C:\WINDOWS\System32\KERNELBASE.dll (unknown line)
while loading no file, in expression starting on line 0
RaiseException at C:\WINDOWS\System32\KERNELBASE.dll (unknown line)
Unwind_RaiseException at C:\Julia\Julia-0.5-latest\bin\libgcc_s_seh-1.dll (unknown line)
_cxa_throw at C:\Julia\Julia-0.5-latest\bin\libstdc++-6.dll (unknown line)
Znwy at C:\Julia\Julia-0.5-latest\bin\libstdc++-6.dll (unknown line)
ZNSt6vectorIN4llvm26DWARFDebugInfoEntryMinimalESaIS1_EE7reserveEy at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm9DWARFUnit9DWOHolderC2ENS_9StringRefE at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm9DWARFUnit19extractDIEsIfNeededEb at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm9DWARFUnit23getSubprogramForAddressEy at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm9DWARFUnit25getInlinedChainForAddressEy at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
ZN4llvm12DWARFContext25getInliningInfoForAddressEyNS_19DILineInfoSpecifierE at C:\Julia\Julia-0.5-latest\bin\LLVM-3.7.dll (unknown line)
lookup_pointer at /home/Administrator/buildbot/slave/package_win6_2-x64/build/src/home/Administrator/buildbot/slave/package_win6_2-x64/build/src\debuginfo.cpp:690

[yuyichao]

This is the OS behavior and we can't change that. A default limit also doesn't really make sense. A user controlled version can possibly be part of #17987 so close as dup.