Unittest compilation of std.string crash (Windows): too deep recursion

[JohanEngelen]

On Windows, this stops compilation (-w!) without any diagnostic:
C:\projects\ninja-ldc\bin\ldc2.exe --output-o -c -IC:/projects/ldc/runtime/druntime/src -IC:/projects/ldc/runtime/druntime/src/gc C:/projects/ldc/runtime/phobos/std/string.d -ofC:/projects/ninja-ldc/runtime/std/string-unittest-debug.obj -w -d -link-debuglib -unittest -IC:/projects/ldc/runtime/phobos

[JohanEngelen]

(Perhaps the bug is not directly related to -w)

[kinke]

On merge-2.069 only, right? I'll investigate once I'm home (in 6+ hours).

[JohanEngelen]

yeah.
I thought I had isolated the issue, reduced it to a test case, and solved it by removing -w for now, but... somewhere I went wrong and the issue is still there.
Disabling the unittests up to line 1200 or so will fix compilation.

[JohanEngelen]

Perhaps it is an out-of-memory issue? I am able to turn the crash on/off by duplicating/removing the asserts around line 1220.

[JohanEngelen]

Nope, with a loop, I've been able to get LDC to allocate more than 10GB of memory, no problem.

[JohanEngelen]

Looks like an infinite recursion of TemplateInstance.needsCodeGen()

[JohanEngelen]

I think I have found the problem: TemplateInstance.needsCodeGen() is a recursive function. As it turns out, for some of the unittests of std.string (e.g. line 1222) the recursion depth is very very deep. The call from declarations.cpp, void visit(TemplateInstance *decl), reaches a recursion depth that is too much on Windows and the program simply exits.
I don't know how to solve this, other than rewriting needsCodeGen to something non-recursive; I think it can be turned into a loop.
What do you think?

[JohanEngelen]

(in one case, the depth is >10000)

[JohanEngelen]

@9rnsr Do you think it is possible to rewrite TemplateInstance.needsCodeGen() into something non-recursive? Or at least a refactoring that reduces the depth of recursion?

[kinke]

Oh wow. That most likely also expains why we're apparently hitting a rare LLVM bug (#930) when compiling the std.string unittests with debug infos on Win64.
But as DMD seems unaffected by this stack limitation, is it something LDC specific?

[yebblies]

We do explicitly set the stack size for DMD on windows, since the default is too small. What is it set to in LDC?

[JohanEngelen]

Thanks, found it: https://github.com/D-Programming-Language/dmd/blob/master/src/win32.mak#L285
I'll try that.
Is there a maximum nested template instantiation depth? (LDC has an experimental feature for setting that)

[kinke]

Apparently the default stack size used by MSVC is a single MB. Linux uses 8, and that's what DMD chose on Windows as well. Let's go with the same size.

[yebblies]

Yes, it has nested template instantiation limits but they are set up for the 8MB stack. We use 8MB for dmd because I looked up the linux default and went with that.

[JohanEngelen]

Mac also uses 8MB as default.

[JohanEngelen]

Fixed in merge-2.069 branch