Support tuples in inline data

[MaxKot]

Feature request

Add support to passing tuples in inline parameter values. This would simplify passing complex values while using combinatorial and pairwise test cases generation.

Example usage:

[Test]
public void Test
(
    [Values(new object[] {"a", 1}, new object[] {"b", 2})]
    (string, int) a,
    [Values(1, 2, 3)]
    int b
)

Currently it can be achieved with ValueSource. Note that this feature will also need an update to NUnit1031 diagnostic.

[manfred-brands]

It is a pity [Values(("a", 1), ("b", 2))] doesn't compile and is something that had been requested but denied.

But I can't say that the new object[] {"a", 1} is very readable.
The road with a ValueSource seems nicer to me:

        private static readonly (string, int)[] ValueTuples = [("a", 1), ("b", 2)];

        [Test]
        public void TupleTest(
            [ValueSource(nameof(ValueTuples))] (string, int) a,
            [Values(1, 2, 3)] int b)
        {
        }

[MaxKot]

I find using ValueSource more cumbersome when there are similar tests that require different input values as you need to make up different field names for each test. Sometimes it results with long names like XDoesYWhenZInput1Values, XDoesMWhenNInput1Values etc so I thought that just specifying values inline would be a better way.

[stevenaw]

@MaxKot Thank you for raising the issue and all the effort putting together your PR. I also wish that the language support here were better, but have to agree with @manfred-brands here.

I think NUnit has historically taken the philosophy that inline TestCase or Values attributes must use compile-time primitive types compatible with attribute parameter types and that the "Source" attributes like TestCaseSource and ValueSource then allow for other more complex scenarios such as non-primitives or reuse of data across multiple tests. Some convenience conversions between these types have existed for historical reasons to support things like the .NET Compact Framework but I believe in general we've tried to continue to follow this philosophy.

I understand you are looking to keep with your chosen test naming convention here but that you are finding the test source names becoming verbose and that you desire simplification. In lieu of changing names of your data sources, is there any test reorganization which could aid in associating a non-reusable test source to it's test? Without knowing your codebase I am wondering if something like nested classes could provide a natural hierarchical way to tightly group these in your case.

Ex:

public class ClassAMethodXTests {
  public class DoesY {
    [Test]
    public void WhenZ([ValueSource(nameof(Input1))] inputs) {
    }

    [Test]
    public void WhenZ([ValueSource(nameof(Input2))] inputs) {
    }

    public static IEnumerable Input1() {}
    public static IEnumerable Input2() {}
  }
}

[MaxKot]

@stevenaw Thanks for the suggestion, I'll look into ways to organize tests so that there's no naming collisions.

What's the final verdict on tuple conversions? I could make a PR for the analyzers repository, but this feature is not desired, I'll close the issue and the PR here.

[stevenaw]

Thanks for asking @MaxKot
Seems we may have some differing opinions between team members. @jnm2 @OsirisTerje what are your thoughts on this being included in the framework?

[OsirisTerje]

I prefer to have readable code, so I lean towards @manfred-brands suggestion above, and also how @stevenaw organize it. Also NUnit1031 is good advice. The problem with tuples is (as @manfred-brands states) is that they are not constant expressions which can be used for attributes. So then you're forced to write the hack with the arrays. I don't think that is a good way to do it. Using a valuesource is a way more nice way to do it.

That said. I am all for freedom of expression - and for the developers freedom to write terrible code - , so if it could be introduced without compromising anything, I'm fine with that. But NUnit1031 should stand, and any user should then suppress that warning if they choose to go down that route.

Looking at the suggested code, I am not sure, it complicates a method that is already more than complex. It needs more tests, and it should not impact performance (meaning benhmarks are needed).

[jnm2]

But I can't say that the new object[] {"a", 1} is very readable.

In C# 12, it should be possible to write ["a", 1] instead.

[MaxKot]

I extracted and split tuples-related code to several methods, hope it's less complicated now.

Should I add benchmarks to nunit.framework.benchmarks or make a separate benchmark outside of the repository?

I see two possible ways to improve readability. One is to pass "flattened" tuples to ValuesAttribute constructor. Knowing the size of the target tuple and the number of constructor arguments, arguments can then be grouped to construct tuple values. Some additional separator e.g. three spaces instead of one may be used to separate tuples. This should look something like this:

[Test]
public void Test
(
    [Values("a", 1,   "b", 2)]
    (string, int) a,
    [Values(1, 2, 3)]
    int b
)

I don't like how it's pretty error-prone to use but it looks cleaner than sub-array instantiations.

Another way is to add an attribute providing exactly one argument value instead of all arguments and allow multiple arguments on a single parameter:

[Test]
public void Test
(
    [Value("a", 1)]
    [Value("b", 2)]
    (string, int) a,
    [Values(1, 2, 3)]
    int b
)

Handling tuples can be kept within the new attribute thus removing any performance and complexity impact on ParamAttributeTypeConversions. I'm not sure whether it should be included in the framework or it can be left to be implemented in the client code though.

And finally a way to provide custom conversions may be added. It's already possible with client-controller types via TypeConverterAttribute so why not allow client code to provide a TypeConverter for any type in a way like custom formatters are added? Tuples could be handled outside of the framework then.

[OsirisTerje]

Should I add benchmarks to nunit.framework.benchmarks or make a separate benchmark outside of the repository?

To nunit.framework.benchmarks please.

I seem to remember (but cant find it now) a discussion about creating a NUnit.Extensions package, which could contain any kind of user contributed framework extensions. A kind of pay-as-you-go libray (in the meaning, on your own risk) . This would then not have to be as rigorously treated as we have to do with NUnit itself.