Spec requires infinite recursion in compilers.

[gafter]

Ported from dotnet/roslyn#12061

from @khyperia
Reproduction steps:

Compile the following code on the command line (or paste it into VS, which hangs VS).

delegate Del1 Del1();
delegate Del2 Del2();

class Program
{
    static void Method(Del1 del1) { }
    static void Method(Del2 del2) { }
    static void Main()
    {
        Method(() => null);
    }
}

Version Used:

1. master (ish): stack overflow
2. 1.2.0.60325 (roslyn): stack overflow
3. 4.6.1038.0 (native): test.cs(10,9): error CS0121: The call is ambiguous between the following methods or properties: 'Program.Method(Del1)' and 'Program.Method(Del2)'

I think this is due to a recursive loop in OverloadResolution.BetterConversionTarget - we get into the case where both types are delegates, get their return types, and then recurse with BetterConversionTarget(r1, r2, ref useSiteDiagnostics) (r1/r2 are the return types of the delegates). They're the same type as the original delegates, so, boom.

---

from @gafter

@MadsTorgersen This is an infinite recursion required by the language specification, and this is only the simplest example of it. It would be consistent with the language specification to conclude that Del1 is a better conversion target, or that Del2 is a better conversion target, or that neither or both are better conversion targets. What should the spec say to terminate the recursion? This problem does not occur in the ECMA version of the specification, as it appears to have been introduced recently with better betterness.

---

from @gafter

Here is an example that is very difficult to catch in the compiler using a simple "are we already asking that question" recursion breaker

delegate Del2<Del1<T>> Del1<T>();
delegate Del1<Del2<T>> Del2<T>();

class Program
{
    static void Method(Del1<string> del1) { }
    static void Method(Del2<string> del2) { }
    static void Main()
    {
        Method(() => null);
    }
}

---

[BillWagner]

@MadsTorgersen Should this move to the standard repo, or is this an implementation issue introduced with better betterness?