vNext / 1.0

[SebastianStehle]

Storage Provider

• Improve the performance of the StorageProvider by preventing a serialization step
• Improve the performance of the StorageProvider by ensuring that we make only one call to MongoDB per operation.
• Implement optimistic concurrency with backwards compatibility.
• Write tests for StorageProvider (partially done)
• Fix a bug with wrong naming of collections when Grain is a nested class. (in vnext branch)
• Add logging.

Statistics

• Cleanup statistics
• Improve statistics by ensuring that we make only one call to MongoDB per operation.
• Write tests for statistics (or get them working again)
• Store statistics over time not only a snapshot (I have a test in vnext branch for it).

Reminders

• Cleanup reminders
• Improve reminders by ensuring that we make only one call to MongoDB per operation.
• [x ] Write tests for reminders (or get them working again)
• Implement optimistic concurrency for reminders.

Membership

• Cleanup membership
• Improve membership by ensuring that we make only one call to MongoDB per operation.
• Write tests for membership (or get them working again)
• Implement optimistic concurrency for membership, in away that only one collection is required (MongoDB is not ACID)

General

• Extend test app with statistics
• Use new configuration system.
• Check value of loggings.

[laredoza]

Question 1: I'd prefer to not break backwards compatibility. Without knowing the new structure, maybe on loading / deserialization, if there is an error we load it like we did previously, and then save the item in the new format again. Would that be possible? The other option is to add a version field to the document ( Our new documents could be version 2) and handle loading data appropriately. That way if we change this again it won't be a big deal. This would probably be the best way to go.

Question 2: Yes, it's for eTag support

The initial providers were all based on the MSSql providers.

[SebastianStehle]

The problem is that you loose the information which version your document has. Which basically means that you have to query the document before updating, which will hurt performance. Not sure how to do it with compatibility.

[laredoza]

How will the new document look?

As an example the test application saves the EmployeeGrain as follows. How will it look now?

[SebastianStehle]

e.g. like that:

{
   _id: "GrainReference=4edc4ec3a3f9faa17067449405e7e79f03ffffff95764fba"
   _etag: "ETAG",
   _doc: <SERIALIZED_JSON_DOC>
}

[laredoza]

Okay, maybe the first idea would be the best way to go then. Load it expecting the new format. If we get an exception try and load it the previous method and update it. The next time it gets loaded it would work and we would have minimum performance impact. Is that feasible?

[SebastianStehle]

that should work, will add a test for it

My idea was to use this approach for etag support:

collection.ReplaceOnAsync(x => x.Id == key && x.Etag == etag, doc, new Options { Upsert = true });

There are several cases:

1. No document exits => Makes insert and adds etag
2. Document exists and has correct etag => Makes update
3. Document exists and has incorrect etag => We have to ask for the old etag (to also generate the correct exception). If the etag exists we throw an exception, if the etag does not exist it is stored in the old format and we can make a ReplaceOperation.

[laredoza]

Should I look at upgrading to v2?

[SebastianStehle]

Do you want to keep the compatibility to 1.X? I think we can just make a v2 project in the same folder. Afaik the interfaces have no changed.

[laredoza]

I think we should just upgrade to TechPreview3(.Net.Core 2.0.) TechPreview2 isn't production ready in any case. If someone is continuing with that version the nuget package 0.1.6 will be installed.

The dotnet 4.6.1 target will still make the library comptible to the mainline version (1.51)

[SebastianStehle]

What is with Beta1?

[SebastianStehle]

I am not sure if they have already migrated TestSilo. My last info is that there was an issue because of appdomains

[laredoza]

My suspicion is that the nuget packages are behind. The project is referencing myget at the moment.

I think this is the latest version:

https://blogs.msdn.microsoft.com/orleans/2017/09/13/announcing-orleans-2-0-tech-preview-3/

The big move seems to have gone to .NET Standard 2.0

[SebastianStehle]

Beta1 is the newest: https://github.com/dotnet/orleans/releases ... 13 days old :) ... I think they moved to nuget.org with the beta.

[laredoza]

Well, I'm confused. I always thought previews came after the beta :)

[SebastianStehle]

Just confirmed it in the gitter channel ;)

[laredoza]

According to : dotnet/orleans#3629 the plan is to release v2 four weeks after the beta release. That gives us two weeks and a bit(probably more).

My suggestion is to get your changes out as 0.1.7-preview. Get some testing done in the wild. That way we don't need to worry about v2 breaking anything else. We can then create a new vnext branch using v2, which can then become 0.1.8-preview. What do you think?

[SebastianStehle]

Which changes are you talking about? The vnext branch is a little bit of chaos and solves too many problems right now.

[laredoza]

The changes on vnext after everything has stabilized.

[laredoza]

OK great, let me know if I can help. My time is unfortunately very limited for the next week or two. I suspect you might be done by then :) Thanks for all the work

[AwsomeCode]

Isn't that our version number should match with Orleans Version?
As if any new storage feature in Orleans comes. And if that feature Orleans.Providers.MongoDB supports.
Then people will know its compatible with that version of Orleans.

Because Orleans other packages also follow this pattern as well.
For eg.
https://www.nuget.org/packages/Orleans.StorageProvider.Arango/
https://www.nuget.org/packages/Microsoft.Orleans.OrleansAzureUtils/1.5.2

[SebastianStehle]

Of course, because they maintain it together

[laredoza]

I think v1 is fine for now. We can then create an issue on the main project for a code review.if they like it, it can then be put in the main project.

I think with the release of Orleans v2, Mongo will become a larger part of the market as Linux adoption increases.

[AwsomeCode]

As we already support Membership, Storage, Reminders, and etc. we should push to same version 1.5.2.

[SebastianStehle]

The version should not indicate compatibility to orleans, but has an internal meaning. E.g. with Semantic Version the data format could change between v1 and v2.

[SebastianStehle]

@laredoza Can you review the storage provider?

[SebastianStehle]

@laredoza I updated the membership provider. I think there is no way to implement the extended protocol in a reliable way, because it requires multi-document transactions. Therefore I have removed the code for the table version and only implemented etag support. It is described here and implemented for DynamoDb in the same way: https://dotnet.github.io/orleans/Documentation/Runtime-Implementation-Details/Cluster-Management.html

[SebastianStehle]

@laredoza I am (almost) done. What is pending is to update the Test Application. I have imported the TesterInternals assembly from Orleans to run their tests and found some bug (in my version).

[SebastianStehle]

Test App Migrated

[laredoza]

Thanks, sorry I've got a tight deadline for today. Will have a look at the changes tomorrow. Great Progress.

[SebastianStehle]

Nice, would be nice to get a beta out to nuget tomorrow.

[SebastianStehle]

I Simplified the options and configuration extensions.

[laredoza]

It looks great. I'll continue going through it this weekend but I've posted a new preview package in the meantime: https://www.nuget.org/packages/Orleans.Providers.MongoDB/2.0.0-preview1

[SebastianStehle]

Thank you: I also asked if we can use the icon: dotnet/orleans#3660 ... minor thing, but would give the package an official touch :D

[laredoza]

yeah, that would be great. If you've got a nuget account please send it to me on gitter so that I can give you admin rights as well.

[SebastianStehle]

Orleans Team is fine when we use their icon. Can we finalize it? We should also make PR for the Orleans docs to add our link to the documentation.

[SebastianStehle]

I published a preview2