Length of generated test names should be limited

[CharliePoole]

Since NUnit uses the value of a string argument to a test as part of the name, it's possible for the names to be arbitrarily long. This causes problems, particularly in the Gui, where GDI+ exceptions may be thrown and in the VS adapter, which may either reject the name or truncate it arbitrarily.

Names generated in the framework should be limited to a reasonable length <= 449 characters, the length that is enforced by the adapter. However, we don't want to simply truncate the name after generating it. Ideally, each individual parameter should be truncated in a way that limits the overall length of the name while preserving the general outline of FixtureName(args).TestName(args).

Since any truncation may cause duplicate names, which NUnit itself does not mind but which can break some clients like TestExplorer, name generation should include adding a unique value like the test id where needed. Note that it would be possible to use the test id alone for a unique name, but that this would not result in a good user experience in runners (TestExplorer again) that use the test name as a display name.

We may want to provide the raw materials for separating the display name from the "real" unique test name for those runners that want to take advantage of it.

[yetibrain]

We have the same problem regarding long test names, mostly with parameterized tests. This caused big problems because we use the testnames also as filenames (we have a tool that extracts failed tests from a testresult file in order to re-run just the failed tests) We already shorten long namespaces by using a kind of namespace-prefix instead of the entire name. Concerning parameters, we ran into trouble because a filename cannot have certain characters like double quotes. So we use a regex with a named group to find the parameters and replace the entire parameters with the hashcode of it. In case of ambiguous names we add simply numbers to the end of the name.

[CharliePoole]

See issue #68 which should be worked in tandem with this one.

[CharliePoole]

Marked status:design since we need some sort of spec about how we will truncate the names.

[jpierson]

We have run across this too when using data driven tests in combination with the Visual Studio adapter for NUnit. If I remember correctly I believe it caused an error in the adapter or in visual studio when these tests were run. Temporarily we modified our tests so that the data passed was a custom object that implemented ToString so that we could control how it appeared in Visual Studio. I would be nice to be forced into such workaround though.

[CharliePoole]

On Launchpad, Dave Kipfer wrote:

When running repeated tests, the TestName is identical between
iterations. Adding an iteration number would be helpful to a listener
to distinguish one instance from another without counting them itself.

We should take this into account in the algorithm for name generation.

[CharliePoole]

Started work on spec at https://github.com/nunit/dev/wiki/Generation-of-Test-Names

[CharliePoole]

Spec is now complete. Please review.

[CharliePoole]

PR #469 is at least a partial fix to this problem. I'm postponing further action - if any is needed - until the beta.

[CharliePoole]

@nunit/core-team Please review the updated spec at https://github.com/nunit/dev/wiki/Generation-of-Test-Names at a high priority.

If we can agree on the approach there, we can quickly close several outstanding issues. If you have read all the boilerplate before, you'll find the new proposal near the end of the page.

[CharliePoole]

@nunit/core-team It has been a week and nobody has commented on the proposal for how to resolve this issue. Kindly let me know what you think so we can start implementing it.

[oznetmaster]

The proposal seems quite reasonable to me.

[CharliePoole]

Moving ahead on this as a feature, following the spec at least initially.

[CharliePoole]

The spec at https://github.com/nunit/dev/wiki/Generation-of-Test-Names has been updated to reflect the current status and to remove all discussion about NUnit 2.x problems. It will become a technical note in the documentation when this issue is complete. @nunit/core-team and @nunit/contributors please let me know if you think I've missed something.

[rprouse]

Fixed by #912