linq: Select.Where or Where.Select

[giscus[bot]]

linq: Select.Where or Where.Select

LINQ is a very powerful tool for querying data. As the majority of functions are built on top of IEnumerable and it, in most cases returns IEnumerable as well, it is very easy to chain multiple functions together. That leaves you with a question: which one should I use, Select.Where or Where.Select?

https://steven-giesel.com/blogPost/57ed9867-4afd-4d02-9f35-e0941bc6f715?utm_source=csharpdigest&utm_medium=&utm_campaign=1674

[abatishchev]

Hi,
I'd suggest to make the original array static. And BDN idiomatic way would be to initialize it inside the [GlobalSetup] method.
For the baseline, use a plain foreach method, just out of curiosity, to see what's the cost of using LINQ here (disclaimer: I'm a big of of LINQ, and use whenever I can). But that might be harder since you're including sorting. Try without.
And try returning IEnumerable directly for the test methods, to make the experiment more pure.

[linkdotnet]

Yes these are fair points - and obviously, I did exaggerate with the benchmark bringing it to the extreme.

Using your considerations:

private static readonly int[] Numbers = Enumerable.Range(0, 100_000).ToArray();

[Benchmark(Baseline = true)]
public List<int> WhereForEach()
{
var result = new List<int>();

foreach (var n in Numbers)
{
    if (n % 2 == 0 && n % 3 == 0)
    {
        result.Add(n);
    }
}

return result;
}

[Benchmark]
public int[] WhereWhere() =>
Numbers
    .Where(n => n % 2 == 0)
    .Where(n => n % 3 == 0)
    .ToArray();

[Benchmark]
public int[] Where() =>
Numbers
    .Where(n => n % 2 == 0 && n % 3 == 0)
    .ToArray();

Leads to the following result:

|       Method |     Mean |   Error |  StdDev | Ratio | RatioSD |
|------------- |---------:|--------:|--------:|------:|--------:|
| WhereForEach | 117.7 us | 1.05 us | 0.82 us |  1.00 |    0.00 |
|   WhereWhere | 364.7 us | 1.84 us | 1.63 us |  3.10 |    0.02 |
|        Where | 258.5 us | 2.47 us | 2.31 us |  2.19 |    0.02 |

[abatishchev]

btw, I nowhere arguing against your conclusions, totally agree with you! Just some brain exercise :) And thanks for provoking it!

Ideally, to compare apples to apples, and not arrays with lists, all test methods should return IEnumerable directly, which in the first case can be achieved by leveraging yield return.

[linkdotnet]

Oh sorry - I did not take it that way :D

Here is the updated version - I am forcing ToArray as IEnumerable doesn't force any enumeration with Benchmark.NET:

[Benchmark(Baseline = true)]
public int[] WhereForEach() => WhereForEachEnumerable().ToArray();

private static IEnumerable<int> WhereForEachEnumerable()
{
    foreach (var n in Numbers)
    {
        if (n % 2 == 0 && n % 3 == 0)
        {
            yield return n;
        }
    }
}

Results:

|       Method |     Mean |   Error |  StdDev | Ratio |
|------------- |---------:|--------:|--------:|------:|
| WhereForEach | 179.3 us | 0.71 us | 0.63 us |  1.00 |
|   WhereWhere | 360.1 us | 1.02 us | 0.91 us |  2.01 |
|        Where | 256.9 us | 2.72 us | 2.54 us |  1.43 |

[abatishchev]

as IEnumerable doesn't force any enumeration with Benchmark.NET

I learned something new today. Thanks!

[sandman633]

I'm not sure, but won't a pass through the collection, even with multiple where, be performed once? As long as we don't call one of the To methods, that collection may not be counted at all, and none of the where methods will be called at all, otherwise there will be one crawl through the original collection regardless of how many times where and select were called, as well as their order.
Try this example

class MyEnumerable : IEnumerable<int>
{
    private List<int> arr = new() { 1, 5, 23, 7, 6, 19, 20, 9, 8 };

    public IEnumerator<int> GetEnumerator()
    {
        foreach(var i in arr)
        {
            Console.WriteLine($"Here {i}");
            yield return i;
        }
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }
}


MyEnumerable arr = new MyEnumerable();
var arr2 = arr.Where(x=>x>5).Where(x=>x>10).Select(x=>new { val1 = x }).ToList();

[linkdotnet]

That is a good point. From the looks of it you are right, but I have to invest some time into it. Where creates might has his own iterator that could lead to the seen picture, where Console.WriteLine is not shown for "both" Where calls.

In any case, I will update the blog post today or tomorrow as I found this interesting.

[jamescurran]

What is happen is that chain is that the Where()s and the Select() wrap the previous enumerable into a larger enumerator object, and nothing happened until we reach the ToList();

The ToList() basically asks the Select() for the next object. Having none, it asks the 2nd Where() for the next object, which, likewise passes the request to the 1st Where, which passes it to arr. The first Where() reject any item not greater than 5, and asks arr for the next item until it finds one, which it finally passes onto the 2nd Where(). If this item is also greater that 10, it's given to the Select to transforms; otherwise the 2nd Where() tosses it out, and sends another request to the next item back up the line.

[jamescurran]

void Main()
{
	List<int> arr = new() { 1, 5, 23, 7, 6, 19, 20, 9, 8 };
	var arr2 = arr
	.Where(x =>
	{
		Console.WriteLine($"Is {x} > then 5");
		return x > 5;
	})
	.Where(x =>
	{
		Console.WriteLine($"Is {x} > then 10");
		return x > 10;
	})
	.Select(x =>
	{
		Console.WriteLine($"Making new object with {x}");
		return new { val1 = x };
	}).ToList();
	
	arr2.Dump();

}

[linkdotnet]

That is a really cool example! Just tried it out on sharplab.

Also thanks for the detailed explanation - I will update the article and will reference to your comment as I found this very valuable! Thanks @jamescurran

[jamescurran]

That would be great, but when you do, please fix my typo (it's "than", not "then")