JIT: move and revise fgExpandRunRarelyBlocks

[AndyAyersMS]

Move to fgopt, and rework to make its operation a bit more obvious.

[AndyAyersMS]

cc @dotnet/jit-contrib

[AndyAyersMS]

No SPMI diffs.

[EgorBo]

No SPMI diffs.

Can we teach SPMI to detect "invalid weights" differences?
E.g.
was: 100 methods with "invalid weights" flag
now: 120 methods with "invalid weights" flag

something like that

[AndyAyersMS]

Can we teach SPMI to detect "invalid weights" differences?

Not easily, at least with the current mode of operation. SPMI diffs key off of things observable via the jit interface.

We have discussed adding a callback to the runtime that would only happen under SPMI, to allow the jit to report "facts" about each method. The runtime would ignore this callback but SPMI would not.

Having something like that would allow SPMI to compare the facts for each method between two jits, and then it could count / summarize and describe the nature of those diffs.

[EgorBo]

Thanks, another question:
So imagine I somehow got a layout like this:

BB0 [IBC: 100]
  ↓
BB1 [BBF_RUN_RARELY]

The current algorithm will just make BB0 cold, am I right? I just wonder if it could notify that it's going to ignore IBC data for a block and make it cold and most likely something went wrong - or we'll just rely on fgComputeEdgeWeights for that?

[AndyAyersMS]

The current algorithm will just make BB0 cold, am I right?

If profile data is consistent then it shouldn't -- but if it is inconsistent, it might. So these should be flagged by the consistency checker. It might make sense to add a jitdump note if we ever change a profiled non-zero weight to zero.