Refactor Orleans.Streaming into separate packages

[ReubenBond]

Ideally, the core of Orleans only contains the basic functionality. This is a first-draft at moving the core streaming functionality out into separate packages: Orleans.Streaming & Orleans.Streaming.Abstractions.

We could aim for 3+ packages or 1 package, instead, depending on how we want to divide things.

[benjaminpetit]

Can this method be called concurrently?

[ReubenBond]

It's only called during shutdown, so it shouldn't be

[benjaminpetit]

I think for now two packages is better than what we have now.

We can see later if we want to move MemoryStream and SimpleMessageStream in different package later on, in smaller PR

[pentp]

From a user's perspective it's better to have fewer separate packages if they are just for Orleans internal code separation. Even then, if this separation is really needed, it would be better to achieve by packaging multiple dll-s in the same nuget package.

However, the reverse is true if the packages pull in additional external dependencies.
One particular pain point for me was that Orleans.Persistence.AzureStorage depends on both Microsoft.Azure.Cosmos.Table and Azure.Storage.Blobs, but you'd typically actually only use one of these (the latter used to be Microsoft.Azure.Storage.Blob and we avoided any versioning issues by using Azure.Storage.Blobs in our own code).

Ideally, Orleans.[Clustering/GrainDirectory/Persistence/Reminders].AzureStorage should all be merged into one (e.g., Orleans.Azure.Cosmos.Table or something like that) as they all have identical dependencies (and a lot of duplicated code files actually) and blob storage provider split out into its own package (e.g., Orleans.Azure.Storage.Blobs).

Related to this, we should start multi-targeting Orleans.Core.Abstractions package to avoid the additional dependencies on .NET Core and to enable using newer APIs in core abstractions code.
And Orleans.Runtime.Abstractions should be merged into Orleans.Core as it's not really a classic abstractions package because it pulls in everything that Orleans.Core uses already and doesn't contain much code actually.
Orleans.Server and Orleans.Client could be removed entirely.

[ReubenBond]

I agree with what you're saying. In this case, moving these pieces into a separate project is intended to help us iterate on them more quickly and keep the core more agile (less special cases, pay only for what you use in each activation/call). Even if we package them all into Orleans.Sdk/Orleans.Server/Orleans.Client or whatever metapackages we end up with (and I think we should aim for one or two packages per end-developer project. CodeGen should be included automatically as a part of the SDK package)

The existing packaging scheme made sense originally, before Cosmos DB took over the SDK for one Azure Storage service but not the other... Maybe now is time to split them.