Always repeat Test x times

[cfuerbachersparks]

Currently only repeat until success RetryAttribute and repeat until failure RepeatAttribute are possible.
There should be a way to repeat a test always x times, for stress testing a system for example.

This would probably need the refactor mentioned in the RepeatAttribute here

[stevenaw]

I think this is a good idea to increase usability. We might want to consider exposing a property to opt-in to this new behavior with a default value compatible with the current behavior. Something like:

[Retry(5, StopOnSuccess = true)]

I'm not sure if there's a better name for that property. @cfuerbachersparks @nunit/framework-team thoughts?

[cfuerbachersparks]

Would probably name it StopOnNonFailure as it would better reflect what happens in the background, started work on my branch https://github.com/cfuerbachersparks/nunit/tree/RetryWithoutStop.

[manfred-brands]

I think that [Repeat(5, StopOnFailure=false)] is a better option than Retry which implies repeat until success.

[stevenaw]

I'm not against that idea. I'd been trying to think about if we could avoid making a new attribute for this, though I agree with the sentiment that the Repeat name is clearer for this purpose.

Anecdotally, I suspect the "flaky test" use case met by Retry may be more common than the "load test" based on my perception of issues filed here, so I don't know about Repeat completely replacing Retry. My thinking is we'd want to keep something with defaults setup for that use case, in which case perhaps any new Repeat attribute wouldn't need a property to configure its behavior. It would always complete n times.

[cfuerbachersparks]

https://github.com/cfuerbachersparks/nunit/tree/RepeatWithoutStop

moved to logic to the RepeatAttribute on a new branch

[manfred-brands]

@cfuerbachersparks Can you please put up a PR where you want to merge that branch in your repository into this one.