cmd/compile: depth limit reached in type formatting routine

[griesemer]

Place holder issue for cases of overly deep types or (invalid) recursive types. See #29264 for an example.

[gopherbot]

Change https://golang.org/cl/154583 mentions this issue: cmd/compile: increase nesting depth limit for type descriptors

[griesemer]

A recursive type where we currently fail:

type T interface {
	M(interface {
		T
	})
}

(from #16369).

[ianlancetaylor]

#32547 is a test case that fails due to the recursion limit, although the test itself does not use recursion. It just uses a very very very deep pointer type.

[empijei]

An update on this bug:

At Google CTF finals this year a team was able to exploit this behavior and obtain remote code execution without importing any package except for "fmt".

The issue is that some code that was supposedly safe had a memory corruption due to a very deep type.

Here you can find a write up of the challenge, and here is the exploit.

When there is a very deep type (253 in this case) the compiler generates an incorrect program. In my opinion this should not be the case and it should instead reject the invalid code (after all, the generated program will not work anyways).

/cc @mvdan @FiloSottile