long tuple problems

[yuyichao]

Minimum working example

Enumerate((Int, 1:10000...))

Segfault in GC and is possibly an GC issue so the full backtrace is not that interesting...

Trace before gc fires

#97 jl_gc_collect (full=full@entry=0) at gc.c:2227
#98 0x00007ffff6afb25f in maybe_collect () at gc.c:777
#99 alloc_big (sz=sz@entry=40080) at gc.c:851
#100 0x00007ffff6afb98e in allocobj (sz=sz@entry=40072) at gc.c:2449
#101 0x00007ffff6add409 in newobj (nfields=5009, type=0x7ffdf2398010) at julia_internal.h:28
#102 jl_new_uninitialized_datatype (nfields=9998) at alloc.c:520
#103 0x00007ffff6a7e981 in inst_datatype (dt=0x7ffdf239c370, p=0x153d7d0, iparams=<optimized out>, ntp=9998, cacheable=<optimized out>, isabstract=0, n=0, env=0x0, 
    stack=0x0) at jltypes.c:1968
#104 0x00007ffff6a9334a in jl_apply (nargs=<optimized out>, args=0x1529f20, f=0x7ffdf23ace50) at julia.h:1290
#105 jl_f_apply (F=<optimized out>, args=<optimized out>, nargs=<optimized out>) at builtins.c:472

This is arguably not a good thing to do but is actually what is done in showerror(::IO, ::MethodError) for a long argument list.

Edit: This is not a stack overflow because that would be #11321 and it is much slower and would raise a StackOverflowError instead of a SegFault.

Mailing list link: https://groups.google.com/forum/#!topic/julia-dev/BLvIjoiJtaI
@ScottPJones
@carnaval

[ScottPJones]

Thanks for opening an issue on this!

[yuyichao]

Just to clarify. The branch I had to reproduce this already has the fix for #11313. A version before it is fixed seems to hit that SegFault first and end up with a slightly different backtrace pattern.

[yuyichao]

OK. The issue is that for a tuple longer than 8192 * 8, the offset field of jl_fielddesc_t overflow and since the first field of the I have here is a pointer field (DataType) the value of the 8192'th field get's filled into the first one which is later interpreted as a invalid pointer.

At the least an overflow check for offset would be helpful. A real solution will probably be related to #11321.

Should also be related to #10380 (which is the reason such a huge data structure is created.) @JeffBezanson

Might also be related to #11187 @Keno (although this one is not limited to codegen....)

[yuyichao]

I'm glad this one seems to be GC free LOL.....
Although it triggers in the GC since it is probably the first one who tries to read the field....

Nonetheless, thanks @carnaval, your debugging techniques works =).

[andrewcooke]

i'm not sure where to report the issue discussed here. my original problem is complex to reproduce, but executing

julia> vcat([(symbol(i), [symbol(i/7.0), string(i)]) for i in 1:10000]...)
ERROR: OverflowError()
 in anonymous at no file

triggers the same error in trunk (commit cf6b3c3). also, running a smaller version (1000) works and the larger as above then works. (not sure that's clear - the above is reliably giving an error only if it's the first command run after julia starts).

[JeffBezanson]

Fixed by #13147