Startup loading behavior batch vs all

[ledjon-behluli]

While getting all data collected from the director grain is the more efficient way, since it involves n + 1 calls, where n is the number of
partitions (i.e. store grains), and 1 is a call from the agent to the director, and all calls to the store grains are done in parallel via Task.WhenAll. We need to keep in-mind the potential size of the whole tuple space, so when the number of partitions grows a lot, this call to the director might result in contention for ThreadPool threads and may lead to thread starvation, therefor degrading performance.

An alternative is for the director to expose an IAsyncEnumerable way to load the data, where each call to it will result in a single batch of loading the data. This "batch" basically means the director calls the store grains one-by-one and streams back the result which the agent call append to its in-memory dataset. This does result in a slower loading of the whole tuple space, as there are 2*n calls, each time a call to the director + 1 call to the partition of a store, but ultimately should not lead to potential thread starvation.

We should have a configurable behavior for this in the SpaceClientOptions.