Avoid mono collection hangs from recursive type growth

[naganohara-yoshino]

Description

The mono item collector currently detects infinite recursive monomorphization
with a depth-based recursion limit. This can be too late for non-regular
recursive instantiations, where each recursive step produces much larger
instance arguments.

This PR starts checking the structural length of instance.args once recursive
instantiation is partway to the recursion limit. If the arguments already exceed
the type length limit, rustc emits the recursion limit diagnostic instead of continuing to process enormous types.

This keeps the diagnostic consistent with the error produced when the same code
is compiled with a smaller #![recursion_limit].

Related Issue

Fixes #156272.

[rustbot]

r? @dingxiangfei2009

rustbot has assigned @dingxiangfei2009.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: compiler
• compiler expanded to 73 candidates
• Random selection from 18 candidates

[naganohara-yoshino]

With this change, the code now terminates and reports the recursion-limit diagnostic:

#![allow(dead_code)]

enum Foo<A> {
    Fst,
    Snd(Box<dyn Fn() -> Foo<(A, A)>>),
}

fn recursive<A>(x: Foo<A>) {
    match x {
        Foo::Fst => (),
        Foo::Snd(f) => recursive(f()),
    }
}

fn main() {
    let p0: Foo<()> = Foo::Fst;
    recursive(p0);
}

error: reached the recursion limit while instantiating `recursive::<((((((..., ...), ...), ...), ...), ...), ...)>`
  --> ./app/src/main.rs:11:24
   |
11 |         Foo::Snd(f) => recursive(f()),
   |                        ^^^^^^^^^^^^^^
   |
note: `recursive` defined here
  --> ./app/src/main.rs:8:1
   |
 8 | fn recursive<A>(x: Foo<A>) {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^
   = note: the full name for the type has been written to 'main.long-type-8599535653221736848.txt'
   = note: consider using `--verbose` to print the full type name to the console

error: aborting due to 1 previous error

UI test is added accordingly.

[rustbot]

This PR modifies tests/ui/issues/. If this PR is adding new tests to tests/ui/issues/,
please refrain from doing so, and instead add it to more descriptive subdirectories.