The Mach test runner doesn't actually make these failures yet, which is
tracked by #3482.