Stack Overflow with dotnet build, Visual Studio fine

[halfbro]

I am working on a large mixed .NET Core 2.2/.NET Standard solution with VS 2019 16.1.3. My SDK version is 2.2.300.

If I build my solution with Visual Studio, it works. This seems to be using the VS-deployed compiler. Here's a snippet:

Target CoreCompile:
38>  c:\program files (x86)\microsoft visual studio\2019\community\common7\ide\commonextensions\microsoft\fsharp\fsc.exe -o:obj\Debug\netcoreapp2.2\TypeScriptFilesGenerator.dll
38>  -g
(rest elided)

However, if I run build from the command line via dotnet build, I get a StackOverflow:

C:\Program Files\dotnet\sdk\2.2.300\FSharp\Microsoft.FSharp.Targets(277,9): error MSB6006: "dotnet.exe" exited with code -1073741571. (project elided)

I can't share the source code, but I was working directly with @cartermp who verified the behavior and that it shouldn't be failing like this.

This does come from a single project that contains a huge record type with 200+ fields if that helps (since this seems related to #6246 but not a struct).

[cartermp]

Verified that with SDK 2.2.105 this does not reproduce.

SDK 2.2.10 --> F# 4.5
SD 2.2.300 --> F# 4.6

This is a regression.

See SDK and language version mappings here: https://dotnet.microsoft.com/download/dotnet-core/2.2

[cartermp]

FYI @dsyme @TIHan @KevinRansom @jonsequitur

[TIHan]

I'm looking into this. I managed to create and try to compile a 1000 field record and it caused the compiler to stack overflow. I found it's happening in IlxGen.

[TIHan]

This is a horrible hack, but it works (no, this is not the way we should do it :) ):

let rec GenExpr cenv cgbuf eenv sp (expr: Expr) sequel =
    cenv.exprRecursionDepth <- cenv.exprRecursionDepth + 1

    if cenv.exprRecursionDepth > 1 then
        if cenv.exprRecursionDepth >= 400 then
            async { 
                do! Async.SwitchToThreadPool ()
                GenExpr { cenv with exprRecursionDepth = 0 } cgbuf eenv sp expr sequel
            } |> Async.RunSynchronously
        else
            GenExprAux cenv cgbuf eenv sp expr sequel
    else
        GenExprAux cenv cgbuf eenv sp expr sequel
        assert (cenv.exprRecursionDepth = 1)

    cenv.exprRecursionDepth <- cenv.exprRecursionDepth - 1

Basically, once we reach a certain depth, shove the computation to a different thread :)

[TIHan]

@halfbro on your large record, can you put [<NoEquality;NoComparison>] attributes on it? I managed to reproduce the stack overflow but after I put these attributes on it, it doesn't stack overflow and that's because it's not trying to generate the equality and comparison methods, which is where the stack overflow is coming from.

[halfbro]

I don't have access to my work computer right now, but I will be able to test this on Monday. I'll let you know then.

[halfbro]

@TIHan I can confirm that adding [<NoEquality;NoComparison>] to the type stops triggering the stack overflow

[TIHan]

@halfbro thank you for confirming that. Whether or not you need equality + comparison for your type, it's up to you. Basically, the generated equality/comparison methods are enormous. Normally if methods become too large, the compiler will split them into separate functions on its own, which would fix the issue. Looks like that may not be happening for the generated methods.

So, there are a couple of ways to solve this:

1. Trampolining computation to a different thread (worse case scenario).
2. Changing IlxGen in a way that will stop this type of recursion from happening, essentially using a Stack collection type instead. (this would be severely invasive to all of IlxGen and is not recommended).

[TIHan]

#6294 is related. Seems continuation passing was added to handle large specific linear expressions which resolved the stack overflow, at least that is what I think based on the change.

[halfbro]

We have our SDK pinned to a previous version for other reasons, so it isn't affecting us right now, but good to know if we need a solution in the near future.

[TIHan]

Ok, correction: The compiler seems to already split the methods for the generated equals/compareto. I guess I'm not super clear on why it doesn't resolve the issue.

[TIHan]

Closing as it's resolved.

[cartermp]

@halfbro This will be in F# 4.7/.NET Core 3.0/Visual Studio 16.3.