Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support RavenDB 4.2 #412

Closed
DavidBoike opened this issue Sep 25, 2018 · 12 comments

Comments

Projects
None yet
5 participants
@DavidBoike
Copy link
Member

commented Sep 25, 2018

In Support for NServiceBus 7 on dotnet core, @zEko asked if there was any news on support for RavenDB 4.0. This issue will track that effort.

Update

This is released. See the upgrade guide for more details.

@DavidBoike

This comment has been minimized.

Copy link
Member Author

commented Sep 25, 2018

RavenDB 4.0 is a challenging upgrade for us, and the honest response right now is that I just don't know exactly what we're going to do or when. Let me attempt to explain our situation:

NServiceBus started its journey with RavenDB when NServiceBus only supported MSMQ, under the assumption that there would be small RavenDB databases located physically on each server taking care of just the NServiceBus storage for endpoints on that server, or a few others for any endpoint that was scaled out. Inherent in this assumption was that any consistency between RavenDB and business data stored elsewhere would be handled by the DTC.

We later learned that there was a bug in RavenDB's DTC implementation which could result in message loss, and as a result we were forced to change our policy to not support DTC use with RavenDB. And of course, RavenDB 4.0 doesn't include DTC support at all.

If you follow that change to its logical conclusion, that means that you probably should not use RavenDB on each server just for NServiceBus storage and store your business data elsewhere. That means that the only logical reason to use RavenDB persistence is if all your business data is in RavenDB already, in which case storing your NServiceBus data in Raven is a natural extension.

Going one step further, this means that the failover characteristics of RavenDB persistence are much more important. Saga data, in particular, does not support primary/primary replication. There's no way to resolve a merge conflict from two threads simultaneously modifying the same saga data, so we can't support an eventually consistent primary/primary topology.

Windows Failover Clusters were a way to mitigate this, but RavenDB 4.0 no longer supports Windows Failover Clusters, with Raven Clustering being the recommended path forward. This fundamentally changes a lot of assumptions that require us to rethink how RavenDB persistence can/should work.

In addition, The ESENT storage engine is no longer supported, so customers will have to migrate from ESENT to Voron. This really complicates matters as we prefer to allow customers to update one endpoint at a time, but if you're already on one central database for business data and NServiceBus data, the requirement to migrate all RavenDB data at once makes that impossible.

In summary, RavenDB 4.0 is the culmination of a LOT of changes that fundamentally alter all the assumptions that RavenDB persistence is built upon. It would be irresponsible of us to simply update the bits and say "Here you go!" We need to reevaluate everything.

@zeko77

This comment has been minimized.

Copy link

commented Sep 25, 2018

I understand completely. This is indeed challenging. We will use other persistence methods for saga data for the time being.

@DavidBoike

This comment has been minimized.

Copy link
Member Author

commented Sep 25, 2018

Our newish SQL Persistence is a great option if your business data is already in any of MSSQL/Oracle/MySQL/PostgreSQL. We also have an experimental RavenDB-to-SQL-Persistence conversion app you can check out. We're also available to get on a call to discuss options for your unique environment any time you want.

@Grendel

This comment has been minimized.

Copy link

commented Jan 25, 2019

This is an issue for us - we use Raven for business data and NSB outbox/saga persistence. So this is forcing a "Solomon's Choice" on us. Have you reached out to the Raven guys to get their input on this - perhaps they can help with implementing robust cluster safe Persistence under 4.+. Have you looked into the cluster wide ACID support in Raven 4.1 using their compare exchange feature? See: https://ayende.com/blog/183426-C/ravendb-4-1-features-cluster-wide-acid-transactions

@DavidBoike

This comment has been minimized.

Copy link
Member Author

commented Jan 31, 2019

An update on RavenDB 4:

Because the upgrade to RavenDB 4 is so difficult on its own, we have decided that we will support a minimal change to RavenDB persistence. The only real change will be using the new cluster-wide transactions feature to ensure that sagas can work in a master-master cluster. Since that feature is experimental in RavenDB 4.1, we will wait and support only RavenDB 4.2 and onwards.

Since RavenDB started dropping nightlies of RavenDB 4.2 this week, we can now get started, but won't release until after 4.2 is fully released.

@Grendel

This comment has been minimized.

Copy link

commented Feb 1, 2019

That is great news David - thanks for the update. Very encouraging.

@tparvi

This comment has been minimized.

Copy link

commented Feb 10, 2019

@DavidBoike Hey what does we have decided that we will support a minimal change to RavenDB persistence mean in practice? Are there some features/configurations you are not planning to support?

@DavidBoike

This comment has been minimized.

Copy link
Member Author

commented Feb 13, 2019

@tparvi it means that we aren't going to try to redesign timeouts or make unnecessary breaking changes. We are going to update to the new RavenDB 4.0 API and fix things required by that. Then we'll update sagas to support the cluster-wide transactions because running Raven in a Windows Failover cluster is no longer supported, and the usage pattern of sagas won't work in a RavenDB cluster without it.

@kblooie

This comment has been minimized.

Copy link

commented May 24, 2019

@DavidBoike Could you provide a timeline now that 4.2 has been released? Are the bits in the develop branch currently supporting the minimal implementation that you described above?

@DavidBoike

This comment has been minimized.

Copy link
Member Author

commented May 24, 2019

@kblooie We're coming along. We've had some trouble because Raven 4 no longer supports non-string Id types and guess what IContainSagaData both has and cannot change. Also some trouble with cluster-wide transactions, so it looks as though we will have to release without support for that (we'll have a check to throw if the cluster topology has more than one full member, but unlimited watchers would be OK) and look at cluster-wide in a follow-up minor if there's enough demand for it.

There are some PRs to merge and then I would hope to be able to get a beta out there while we do the (somewhat extensive) documentation work.

@DavidBoike DavidBoike added this to the 6.0.0 milestone Jun 10, 2019

@DavidBoike

This comment has been minimized.

Copy link
Member Author

commented Jun 10, 2019

Getting ready to release this. Closing so that our automated release notes generator can pick it up.

@DavidBoike DavidBoike closed this Jun 10, 2019

@DavidBoike DavidBoike changed the title Support RavenDB 4.0 Support RavenDB 4.2 Jun 10, 2019

@DavidBoike

This comment has been minimized.

Copy link
Member Author

commented Jun 10, 2019

This is released. See the upgrade guide for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.