[WIP] Multi data center support #3284
Conversation
|
Issue #3261 |
Horusiath
force-pushed
the
multi-dc
branch
2 times, most recently
from
d9f9a2f
to
f7b57dc
Compare
v1.3.7 Production Akka.NET Release
Akka.NET v1.3.8 Stable Release
* disabled some Akka.NET core specs * Update ActorModelSpec.cs * Update TailChoppingSpec.cs * disabled racy BackoffSupervisorSpecs
…et#3744) * fixed typo in RemoteActorRefProvider comment * Working on akkadotnet#3414 - bringing SerializeWithTransport API up to par with JVM * added spec to help validate CurrentTransportInformation issues Based on the equivalent JVM spec * working on bringing serialization up to snuff * brought serialization class up to snuff * wrapping up RmeoteActorRefProvider implementation * WIP * cleaning up Serialization class * looks like there's a Lazy<SerializationInfo> translation from Scala to C# that we haven't quite done * fixed Serialization class * fixed bug with Akka.Remote.Serialization.SerializationTransportInformationSpec * forced a couple of specs using default akka.remote configs to run sequentially This was done in order to avoid the two specs trying to bind on the same port at the same time. * added serialization verification to the Akka.Persistence.TCK * fixed issues with default Akka.Perisstence.TCK specs * fixed IActorRef serialziation support in Akka.Persistence journals and snapshot stores * fixed compilation issuyes * fixed Akka.Sql.Common serialization in a backwards-compatible fashion * had to disable serialization specs for Sql Journals * Added API approvals * updated creator and serialize-all-messages serialization * added ITestOutputHelper to Akka.Cluster.Sharding.Tests.SupervisionSpec * made changes to LocalSnapshotSerializer * fixed bug in WithTransport method * updated Akka.Remote MessageSerializer
…3741) * added stages responsible for streams context propagation * fixed WithContextUsageSpec * updated API approvals * renamed StartContextPropagation->AsSourceWithContext and EndContextPropagation to AsSource * AsSourceWithContext: API approvals
edit the code sample of the main app to properly utilize the "...recommended creational pattern for actors; define a static Props() method". i.e. using the static Props method instead of Props.Create.
Minor rewording/spelling changes.
I'm new to Akka and spent ages trying to figure out why the above test kept intermittently failing despite me copying it directly into visual studio. Turned out it was because of this issue: akkadotnet#2130. So, I suggest including an Epsilon into the TestKit example.
* upgraded to JSON.NET 11.0.2 * upgraded to JSON.NET 12.0.2
|
Can we rerun test for this PR somehow? |
|
Issue might be the age of the PR - added before Azure DevOps was added. If updating it with the latest doesn't trigger it, might need to close this PR and just create a new one. Not 100% sure. |
|
@Horusiath @Aaronontheweb Multi-DC Persistence is a commercial addon, so are we going to have a custom implementation then? |
|
I wasn't aware of that. Putting on my Petabridge hat: at the moment, I'm not sure - we haven't been trying to follow Lightbend's roadmap when it comes to proprietary technology because the way Akka is consumed in .NET looks quite different than it does on the JVM. Our technology is consumed more by enterprises (energy, finance, healthcare, transportation) and whereas Lightbend also has to support large-scale ecommerce, social media, advertising, etc clients who really do need similarly large-scale features like multi-DC persistence. We'd love to have that type of problem too as adoption of Akka.NET continues to expand rapidly ;) In terms of what Petabridge will support, that comes down entirely to economics - features that are expensive to support and are only needed by a small number of very high-utilization users should absolutely be proprietary, as those users should foot the bill for the engineering and QA time that goes into maintaining those. On principle I don't want to co-mingle what Petabridge and the Akka.NET project, but - necessarily, one of the reasons why people adopt Akka.NET in the first place is precisely because we do support it and we are often the ones who foot the maintenance costs for the project. Ultimately we have to strike a balance between what's sustainable for both the company + project and the users' best interests. That's a balancing act that we're still working on figuring out - so far Phobos is the only proprietary component but that will probably change in the future. As far as implementation goes, we'll need to get the serialization system fully standardized onto the Protobuf format, per #3811, so we can introduce features like this in a forward-compatible manner downstream. Obviously this will need to be merged in too before we can start on that. |
|
It makes total sense. I was just wondering, given you guys have put a lot of effort into this PR, what would be the use cases for Multi-DC support without Multi-DC Persistence? And this is just my ignorance talking now. |
|
@ismaelhamed @Aaronontheweb Multi-DC Persistence uses different technique as it allows to have mutliple incarnations of the same logical entity in different data centers. It uses state representation as CRDTs for conflict-avoidance (similar to our DistributedData module). Mutli-DC persistence is not necessary tho - IMHO the most important part is to have ensure that cluster sharding (ddata-based one) and split-brain resolvers are multi-dc aware. You don't need multi-dc persistence features in many scenarios. What you actually want most of the time is to have standard persistence with sharding working across DCs. Another thing, that I'm working over in very slow pace is port of Eventuate - it's a persistence library (written by original Akka.Persistence author) that introduces features like replicated event journals and gives comparable features, that multi-dc persistence provides - in fact it was inspiration for multi-dc persistence on the first place. |
This is port of Multi-DC support from https://doc.akka.io/docs/akka/2.5/cluster-dc.html