Enabling List<T> to store large amounts of elements

[giscus[bot]]

Enabling List to store large amounts of elements

List is one of the most versatile collection types in .NET. As it is meant for general-purpose use, it is not optimized for any specific use case. So, if we look closely enough, we will find scenarios where it falls short. One of these scenarios is when you have lots of data. This article will look at precisely this.

https://steven-giesel.com/blogPost/37b32c2a-245f-4cb9-9e32-73a1091b025b

[adc-wnorvelle]

For what it's worth, if you can set the capacity ahead of time, then list performs as well or better than ChunkedList, at least as far as speed in this benchmark goes:

Method	Items	Mean	Error	StdDev	Ratio	RatioSD	Gen0	Gen1	Gen2	Allocated	Alloc Ratio
List	1000	2.034 us	0.0380 us	0.0481 us	1.00	0.00	1.7891	0.0343	-	8.23 KB	1.00
ListPreAllocated	1000	1.559 us	0.0107 us	0.0084 us	0.76	0.02	0.8602	0.0114	-	3.96 KB	0.48
ChunkedList	1000	4.753 us	0.0715 us	0.0558 us	2.32	0.06	6.7978	0.6180	-	31.34 KB	3.81
List	200000	1,237.303 us	19.2001 us	17.0204 us	1.00	0.00	498.0469	498.0469	498.0469	2048.58 KB	1.00
ListPreAllocated	200000	628.876 us	2.5489 us	1.9900 us	0.51	0.01	249.0234	249.0234	249.0234	781.39 KB	0.38
ChunkedList	200000	698.180 us	5.5163 us	4.6064 us	0.56	0.01	128.9063	125.0000	-	784.99 KB	0.38

Of course, the list is still on the LOH whereas ChunkedList is not.

[linkdotnet]

Good point - The same could be applied to the ChunkedList removing the need for the Array.Copy and friends.