High Availability #120
High Availability #120
Conversation
src/Trinity.Core/Storage/Storage.cs
Outdated
| @@ -194,11 +194,11 @@ static Storage() | |||
| #endregion | |||
|
|
|||
| #region Message Sending Interfaces | |||
| internal abstract void SendMessage(TrinityMessage message); | |||
| internal abstract void SendMessage(TrinityMessage message, out TrinityResponse response); | |||
| protected internal abstract void SendMessage(TrinityMessage message); | |||
There was a problem hiding this comment.
Message passing interfaces may require further polishing:
- The ability to send a message to a cell (chunk routing)
- The ability to set message passing semantics (RR/All/LB/FirstAvailable etc.)
- The ability to create a "transaction": a transaction is a set of servers, that every message to one partition(AG) will be deterministically sent to a server in the transaction set. This guarantees that the messages within a transaction effectively share a context, without excessive synchronization. When the transaction ends, we can further "commit" this to the whole memory cloud.
There was a problem hiding this comment.
@yatao All of the things you enumerate here are of critical importance particularly in our use-case and application area of Ontological Semantic Reasoning in what we call Multi-Zoned Ontology (multiple AGs) with Global Semantic Event projection and Query/Answer Reasoning.
In an actual product scenario like ours, client's *KBs in the SF/GE Distributed (RDF+) member cloud is logically partition via secured licensed ID but the low-level distributed parsing, reasoning is local to entire cluster(s). So I need to be able to send(message) data using a variety of methods (i.e., direct message, Azure Events, Reliable Queues...).
*KB - def. W3C OWL 2 Full/DL Representation of Knowledge
*(RDF+) - def. Our Extended RDF Theoretic Graph Representation and Grafting Engine with FanoutSearch and extended/enhanced LIKQ capabilities.
…directory assignment, name hashing, memcloud disposal, etc.
…raphEngine into ikw-ge-availability
This reverts commit d9cdf01.
| /// A Microsoft Trinity Communiactions based listener for Service Fabric based stateless and stateful | ||
| /// service. | ||
| /// </summary> | ||
| public class TrinityCommunictionListener<TServiceContract> : TrinityServer, ICommunicationListener |
There was a problem hiding this comment.
@TaviTruman let's clean this up and rebase to the dynamic cluster :)
There was a problem hiding this comment.
@yatli Okay will do and I'm going to work against the Listeners you have in place but make it so that its extensible from the GraphEngineServiceBase.
There was a problem hiding this comment.
@TaviTruman good to know! I was also wondering how does a developer kick off based on what we have. 👍
|
|
||
| namespace Trinity.DynamicCluster.Storage | ||
| { | ||
| public partial class DynamicMemoryCloud : MemoryCloud |
There was a problem hiding this comment.
idea: remote UseCell via active message
| { | ||
| InitializeComponents(); | ||
|
|
||
| m_partitioner.Start(m_cancelSrc.Token); |
There was a problem hiding this comment.
note, implementations should make sure that components do not have dependencies on each other.. or, do we need to implement dependency check (like AutoFac)?
| class DynamicRemoteStorage : RemoteStorage | ||
| { | ||
|
|
||
| public DynamicRemoteStorage(ServerInfo server_info, int connPerServer, MemoryCloud mc, int pid, bool nonblocking) |
There was a problem hiding this comment.
Leo's solution is to publish the instance GUID together with ServerInfo, so we do not have to set the id later with a query. Not sure if it's a good idea to add this requirement to an adapter implementation...
| // When creating multiple listeners for a service, each listener must be given a unique name. | ||
| return new[] { | ||
| new ServiceReplicaListener(ctx => new GraphEngineListener(ctx), "GraphEngineTrinityProtocolListener", listenOnSecondary: true), | ||
| new ServiceReplicaListener(ctx => new GraphEngineHttpListener(ctx), "GraphEngineHttpListener", listenOnSecondary: true), |
There was a problem hiding this comment.
http port shall not be opened on passive units
| /// <returns>true if saving succeeds; otherwise, false.</returns> | ||
| public override bool SaveStorage() | ||
| { | ||
| throw new NotImplementedException(); |
There was a problem hiding this comment.
new save format, persistency adapters, and a storage iterator that does not lock up the whole local storage...
| #endregion | ||
| #region Base implementation | ||
| // XXX an implementation shall initialize this! | ||
| protected internal Storage[] StorageTable; |
There was a problem hiding this comment.
well on second thought... do not want to waste cpu cycles...
GraphEngine python binding - core components
…es are initialized
… into availability
… into availability
…n. service is wrapped into infra, remove listener package. now the user program is surprisingly easy to write.
|
good work people. this feature branch is now complete and coming back to |
Do not merge; Currently for unit testing purpose.