This is a reduced version of #1567 for M2.5. For tests that are fragile and falsely triggering the old checks we should just remove those checks.
Triage regression failures and determine what should be removed to bring pass rates up to standard
estimate 8
