The Opposite of FlakeAttempts

[astrieanna]

There's a set of tests that we want to rerun 10 times per run of the suite; kind of the opposite of FlakeAttempts, since the goal is they have to run and pass 10 times to count as passing.

The goal is to ensure that a setting that causes the code-under-test's output to be ordered is actually working, rather than the random order happening to align with sorted order sometimes.

// Test this 10 times to eliminate randomness
DescribeTable("Sorted",
  func(x, y, z SomeType) {
    // Expect stuff to be sorted
  },
  basicTests,
  // TODO find a better way than copy pasting
  basicTests,
  basicTests,
  basicTests,
  basicTests,
  basicTests,
  basicTests,
  basicTests,
  basicTests,
  basicTests,
)

Is there a better way to write that in Ginkgo currently?

My dream is a new decorator that does the opposite of FlakeAttempts, so maybe RequiredPasses. This has only come up once in our test suites that I know of (and thus isn't really worth investing time in improving), so I figured I'd layout the request to see if comes up for anyone else.

[onsi]

Hey @astrieanna - this totally makes sense and is something I've done as well (usually as a for loop either around an It or in the body of an It). There is a CLI flag ginkgo -repeat=N which will repeat the entire suite (and you could pair with filters to just repeat one test) but really this should be a decorator likeFlakeAttempts.

I'll add it to the backlog.