Custom names for argument/parameter values

[IdanZel]

Hello,
I recently came across the following issue while writing benchmarks.

Say I have a benchmark that receives one argument of type List<int> through an ArgumentsSource, for example:

public IEnumerable<List<int>> Source => new[]
{
    new List<int> {1, 2},
    new List<int> {1, 3}
};

[Benchmark]
[ArgumentsSource(nameof(Source))]
public void Benchmark(List<int> argument)
{
    // Do stuff
}

The summary for this benchmark would look like this (max column width increased, measurement results are random):

|    Method |                                        argument |     Mean |     Error |    StdDev |
|---------- |------------------------------------------------ |---------:|----------:|----------:|
| Benchmark | System.Collections.Generic.List`1[System.Int32] | 1.234 ms | 0.0123 ms | 0.0123 ms |
| Benchmark | System.Collections.Generic.List`1[System.Int32] | 5.678 ms | 0.0456 ms | 0.0456 ms |

Since the text displayed in the argument column is the same for both cases, and only tells of each argument's type, there's no trivial way to distinguish between them and know which exact value was passed to each case. This might not be the exact case in this specific example, but this is the general idea.

Note that this is relevant not just to List<T>, but to any non-primitive type, and also when using ParamsSource instead of ArgumentsSource.

I would like to have the aforementioned ability, and one way I thought of doing this is by attaching a custom name to each value in the arguments' source. For example, if I attach the name "One Two" to new List<int> {1, 2} and "One Three" to new List<int> {1, 3}, the summary could look like this:

|    Method |    argument |     Mean |     Error |    StdDev |
|---------- |------------ |---------:|----------:|----------:|
| Benchmark |     One Two | 1.234 ms | 0.0123 ms | 0.0123 ms |
| Benchmark |   One Three | 5.678 ms | 0.0456 ms | 0.0456 ms |

As far as I understood from looking at the source code and existing issues, there's currently no simple way of doing this.
I could, for example, create a type inheriting from List<int> for each value and override ToString() with the name I want it to have, but the more values I add the more effort this would take. Furthermore, if I wanted to use a sealed type as an argument, this would not be possible.

Another way of distinguishing between cases that's currently possible (which I'm aware of) is by accounting for the order by which the cases appear in the summary. However, I imagine this could get confusing when working with many values or with values of different types.

I've already thought of a way to implement this (both for arguments and params), and I'm willing to open a PR, but beforehand I'd like to know if there's anything I missed (e.g. another way of doing what I described above) and if this is a feature you think fits for this library.

[timcassell]

What if BDN has this special NamedArg struct that it can check for when parsing the Source? That is much clearer intention in the code and should translate easily to the table.

public struct NamedArg<T>
{
    public string Name { get; }
    public T Value { get; }
    
    public NamedArg(string name, T value)
    {
        Name = name;
        Value = value;
    }
}

public IEnumerable<NamedArg<List<int>>> Source => new[]
{
    new NamedArg<List<int>>("One Two", new List<int> {1, 2}),
    new NamedArg<List<int>>("One Three", new List<int> {1, 3})
};

[Edit] It could even include an implicit converter from tuple to make it cleaner:

public static implicit operator NamedArg<T>((string, T) tuple)
{
    return new NamedArg<T>(tuple.Item1, tuple.Item2);
}

public IEnumerable<NamedArg<List<int>>> Source => new NamedArg<List<int>>[]
{
    ("One Two", new List<int> {1, 2}),
    ("One Three", new List<int> {1, 3})
};

[timcassell]

Or another option, maybe better, is BDN to have a special enumerable type just for this:

public interface IEnumerableWithNames { } // For source type checking.
public interface IEnumerableWithNames<T> : IEnumerableWithNames, IEnumerable<(string, T)> { }
public class ListWithNames<T> : List<(string, T)>, IEnumerableWithNames<T> { }

public IEnumerableWithNames<List<int>> Source => new ListWithNames<List<int>>
{
    ("One Two", new List<int> {1, 2}),
    ("One Three", new List<int> {1, 3})
};

This would be easier to implement than checking the generic types, and I think also a little cleaner in benchmark code.

[dhymik]

Custom names for argument/parameter values is something I also miss. I have to hand-edit my result tables to make sense of my results.

[YegorStepanov]

You don't have to inherit every List<T>.

You simply need IEnumerable type with overloaded ToString and with implicit casting to argument/param type (List<int> in your case)

var config = DefaultConfig.Instance
    .WithSummaryStyle(
        DefaultConfig.Instance.SummaryStyle.WithMaxParameterColumnWidth(50));

BenchmarkRunner.Run<BenchmarkClass>(config);


public class PrettyEnumerable<T> : IEnumerable
{
    private readonly IEnumerable<T> _value;

    public PrettyEnumerable(IEnumerable<T> value) => _value = value;

    public IEnumerator GetEnumerator() => _value.GetEnumerator();

    public override string ToString() => string.Join(" ", _value);

    public static implicit operator List<T>(PrettyEnumerable<T> prettyEnumerable) =>
        prettyEnumerable._value.ToList();
}

public static class IEnumerableExtensions
{
    public static PrettyEnumerable<T> Prettify<T>(this IEnumerable<T> enumerable)
    {
        return new PrettyEnumerable<T>(enumerable);
    }
}

public class BenchmarkClass
{
    [Benchmark]
    [ArgumentsSource(nameof(Source))]
    public void Benchmark(List<int> argument) { }

    // non-generic IEnumerable not working for some reason
    public IEnumerable<PrettyEnumerable<int>> Source()
    {
        // you can write implicit casting from List<int> to PrettyEnumerable<int> instead calling Prettify(),
        yield return new List<int> { 100, 500 }.Prettify();

        // but you cannot write implicit interface(IEnumerable) casting
        yield return Enumerable.Range(0, 20).Prettify();
    }
}

[ogxd]

Unfortunately, creating a custom type is not possible in some cases. For instance, if your parameter is an array and you want to use it as a Span, you currently need to use "real" arrays. See: #774

[andznui]

Another possible way i would see it would be

public List<int> OneTwo => new List<int> {1, 2};
public List<int> OneThree => new List<int> {1, 3};

[Benchmark]
[ArgumentSource(nameof(OneTwo))] // notice Argument instead of Arguments
[ArgumentSource(nameof(OneThree))] 
public void Benchmark(List<int> argument)
{
    // Do stuff
}