Conversion from using Timeout attribute to Cancellation

[simontom]

Hey guys 👋 , please, I'm wondering

Using TimeoutAttribute was a safety net, is there anything like that?
We used Timeout set to 20s, CTS to 19s
We use blame-hang but it stops everything

[manfred-brands]

In your case you use your own CancellationToken. Doesn't that cancel your tests?

Note that you are not actually using NUnit's CancelAfter as your are not using its CancellationToken but using your own is also fine.

[simontom]

I know, was rewritten by my colleague in a hurry 😃

E.g. once in a while when multiple runs of the PR testings are ran simultaneously (multiple branches), it might hang and using Timeout+Retry was a good combo TBH

Just curious if there is something you can come with off the top of ur head

[jzapletal95]

As a colleague of simontom...

After upgrading from NUnit3.x to NUnit4.x, TimeoutAttribute has been marked as obsolete. Therefore, I rewrote all tests as follows:

At this moment we are looking for the best and most correct way

[stevenaw]

@jzapletal95 That looks pretty good! Using CancelAfter will prompt NUnit to manage the cancellation token internally so there shouldn't be a need to have a CancellationToken field anymore. NUnit's cancellation token can be read from the test context:

-var exception = Assert.ThrowsAsync<ScriptExecutionException(() => ScriptExecutor.ExecuteScript(configuration, CancellationToken));
+var exception = Assert.ThrowsAsync<ScriptExecutionException(() => ScriptExecutor.ExecuteScript(configuration, TestContext.CurrentContext.CancellationToken));

Or automatically passed in as a test argument:

-public Task Execute_InvalidDirectoryParameter_Throws()
+public Task Execute_InvalidDirectoryParameter_Throws(CancellationToken cancellationToken)

There's a bit more info on it here: https://docs.nunit.org/articles/nunit/writing-tests/attributes/cancelafter.html

[jzapletal95]

Thanks @stevenaw,
To avoid adding a CancellationToken parameter to every test, can I retrieve NUnit’s injected token inside [SetUp] instead?
Also, if I understand correctly, this still doesn’t solve our real issue. We already use our own CancellationTokenSource with a 20‑second timeout, so CancelAfter doesn’t really add anything for us.
What we actually need is a way to really stop a test after 20 seconds, even if it gets stuck in a synchronous blocking call or deadlock. Is there any supported way in NUnit to forcefully abort such a test?

[stevenaw]

Some more information would be helpful. What runtime are you targeting (.NET Framework or .NET 10) and which runner are you using (dotnet test or the nunit console runner or nunitlite? A self-contained and runnable repro of your code would be better still.

The best answers we can give for now are generalized:

To avoid adding a CancellationToken parameter to every test, can I retrieve NUnit’s injected token inside [SetUp] instead?
Also, if I understand correctly, this still doesn’t solve our real issue. We already use our own CancellationTokenSource with a 20‑second timeout, so CancelAfter doesn’t really add anything for us.

You can indeed. The docs I linked touch on it a bit and I've just submitted a PR to clarify this a bit further nunit/docs#1144 As far as whether it "adds anything" is a bit of matter of opinion. NUnit has added this in order to save test writers from having to manage, maintain, and spend developer time handling it yourself. Of course, there is nothing from stopping anyone from having a custom implementation which best suits your specific needs if you wish or have already done so.

What we actually need is a way to really stop a test after 20 seconds, even if it gets stuck in a synchronous blocking call or deadlock. Is there any supported way in NUnit to forcefully abort such a test?

Timeout uses Thread.Abort() which is inherently unsafe and can leave things in a corrupt state, which has led to it's deprecation and removal from newer runtimes. There's a bit more info on it here. The currently accepted "safe" model is to use cooperative cancellation. Cooperative cancellation can be used in async operations as well as in synchronous "busy loops" within your control. The docs I linked show an example of each of these in action.

For other cases, if you use the dotnet test runner you can use their --blame-hang or --blame-hang-timeout option however "this will stop any further execution of the remaining tests.". For truly safe control and isolation of misbehaving code which could deadlock you may need to consider per-test process isolation. This doesn't exist in the framework today, however a issue has been open requesting it: #3788 No one has submitted a PR to add this feature yet. We would welcome a PR to add it to the framework if you did choose to pursue any custom implementation and were able to share it and contribute it under the MIT License which governs NUnit's contributions.

[OsirisTerje]

Also be aware that Thread.Abort is not supported for .net 5+. It will throw PlatformNotSupportedException. This is the reason we don't support Timeout anymore for .net core, as it is using Thread.Abort.