1000 fields + #derive(PartialEq) => stack overflow at compile time

[stepancheg]

Source

% rustup run stable rustc --version
rustc 1.15.1 (021bd294c 2017-02-08)
% rustup run stable rustc unittest_enormous_descriptor.rs
thread 'rustc' has overflowed its stack
fatal runtime error: stack overflow
zsh: exit 1

% rustup run nightly rustc --version
rustc 1.16.0-nightly (83c2d9523 2017-01-24)
% rustup run nightly rustc unittest_enormous_descriptor.rs
zsh: exit 1

% uname -a
Darwin MacBook-Pro-9.local 16.3.0 Darwin Kernel Version 16.3.0: Thu Nov 17 20:23:58 PST 2016; root:xnu-3789.31.2~1/RELEASE_X86_64 x86_64

[leonardo-m]

For me 654 fields suffice for overflow Rustc stack. The problem is more general: how to test/modify Rustc to be sure there aren't many other similar stack overflows?

[abonander]

I suspect it's a recursion that's dependent on the number of fields. Recursion seems elegant, but in languages without TCO it's more of a liability than anything, I think.

[sinkuu]

Debugger pointed out hir::lowering::LoweringContext::lower_expr, that transforms expression tree using recursion.

[leonardo-m]

How do you fuzzy-test Rustc to discover many of such cases of possible stack overflow?

[abonander]

So this probably affects any long chain of binary operations, since I suspect the derive just expands to

self.field_1 == other.field_1 &&    
self.field_2 == other.field_2 &&
// etc. etc.

A simpler repro might just generate a long chain of expressions like this.

[ghost]

@abonander you are right.

https://gist.github.com/Insomgla/573a162768a36163a0d9d17ab4fa3c41

And I get the same error.

[ghost]

My gist is at the exact critical point where the overflow is reported by the compiler on my computer. If I remove 1 operation, I have another failure report. See :

https://gist.github.com/Insomgla/ef1de205f587ef64d3b3dd323903dd48

[abonander]

Cargo likes to eat error messages. Since you've got just a single file and no dependencies you can just invoke rustc directly and see what it outputs: rustc repr2.rs

The exact point that it overflows is highly dependent on environment as each OS sets a different default stack size for new threads. If you set a much larger default stack size it'll probably compile successfully. That's the difference between deep and infinite recursion.

So it's looking like HIR expression lowering needs to be refactored to eliminate the recursion. Probably not a hard job, something that could be mentored maybe.

[leonardo-m]

So it's looking like HIR expression lowering needs to be refactored to eliminate the recursion.

It's also quite good to look for all the similar bugs in the codebase.

[abonander]

Yeah, but these things can be addressed incrementally. It may be a good idea to form a metabug tracking deep recursion issues in the compiler.

[ghost]

@abonander thanks for the information :p

[oli-obk]

How do you fuzzy-test Rustc to discover many of such cases of possible stack overflow?

You use a crate like syn and combine it with quicktest to generate arbitrary Crates and feed them into the compiler.

[ghost]

So how do we handle this ?

[qarmin]

I can't reproduce this with rustc 1.51.0 (2fd73fabe 2021-03-23)