RPCs with Apache Thrift

What is Apache Thrift?

Apache Thrift is a software framework for scalable cross-language services development. It enables the communication and exchange of data in a heterogeneous service environment, including the efficient and seamless transfer of information between C#, C++, Python, JavasScript and many other languages.

What are Thrift Services and Thrift Data Types?

We can define function interfaces in apache thrift, which are called services. Services are comparable to class definitions. Once instantiated, the methods defined for the class can be called, just as with any other object in the native programming language. Hence, from a developer perspective, I can call any function on the thrift object regardless of whether it is implemented in my own language or in any other language.

Here is an example of a simple MotionModelUnit with a single DoStep method. Note, that the actual MotionModelUnit provides more functionality.

service MotionModelUnit
{
   MSimulationResult DoStep(1: double time, 2: MSimulationState simulationState)
}

In order to communicate complex data types, it is possible to define additional enums and structs in thrift. Services are called using these specific data types. In the example above, MSimulationState and MSimulationResult are such complex data types. Datatypes can contain required and optional fields. For example, to transfer 3 dimensional vectors, we cannot directly utilize Unities vector class, but have to define our own MVector3 data type:

struct MVector3
{
   1: required double X;
   2: required double Y;
   3: required double Z;
}

In case of the MSimulationState, it does contain optional fields for constraints, scene manipulations and events. These fields can be left empty, if there are for example no constraints defined, yet. The MSimulationState struct contains mostly other complex data types:

{   
   1: required avatar.MAvatarPostureValues Initial;
   2: required avatar.MAvatarPostureValues Current;
   3: optional list<constraints.MConstraint> Constraints;
   4: optional list<scene.MSceneManipulation> SceneManipulations;
   5: optional list<MSimulationEvent> Events;
}

What are Thrift Definition Files?

Thrift definition files are used to describe service interfaces and datatypes. The interface definitions can be separated into different description files and combined with an include statement. Hence, in the MOSIM case, the definition files describe the complete Modular Motion Interface in a very explicit way, similar to UML diagrams.

However, unlike UML diagrams, the Thrift Software Framework can be utilized to automatically generate source code based on these definition files for various programming languages. The auto-generated code contains struct definitions and interface classes, depending on the language environment.

How can I Implement the Interfaces?

Service interfaces can be implemented natively by inheriting from the respective interface class. Naturally, all of the defined functions of the service interface have to be implemented in the new class. For example in C#:

public class IntermediateSkeleton : MSkeletonAccess.Iface
{
   // ...
   // Implement Interface Functions here
   // ...
}

The other possibility is to use an implementation from a remote service. Given that there is already an implementation in a different language (e.g. in Python), the python software component can host access to the implementation via a TCP / IP connection. In the C# software component, we can implement a service interface using a thrift client. Although the C# client technically implements the interface, it just forwards every request to the remote python component and acts as a wrapper. For a user of the interface, however, there is no apparent difference.

public static MSkeletonAccess.Iface CreateSkeletonInstance(string host, int port) 
{
   var transport = new TBufferedTransport(new TSocket(host, port));
   var protocol = new TCompactProtocol(transport);
   MSkeletonAccess.client remoteClient = new MSkeletonAccess.Client(protocol);
   transport.Open();
   return remoteClient;
}

For this to work, however, there have to be two things ensured first: 1. there has to be an implementation already existing and this implementation has to be shared as a thrift server and 2. the host-address and port have to be known. To communicate service hosts and ports, the MOSIM Registry in the launcher can be utilized.

How can I host a Thrift Service?

Any interface implementing a thrift service can be hostet, but no interface is hosted automatically. To allow for good communication between remote clients and servers, there should be an agreement of transport layer and protocol. Throughout the MOSIM framework, we relied on the TBufferedTransport and TCompactProtocol.

In order to host a service, it has to be manually started in the server component. In addition to the server components (socket, transport, protocol), it requires an actual implementation of the service.

// ...
// MSkeletonAccess.Iface processor = ... ; // place instance of an implementing class here
TServerTransport = new TServerSocket(port);
var transport = new BufferedTransportFactory();
var protocol = new TCompactProtocol.Factory();
var Server = new TThreadPoolServer(processor, serverTransport, transport, protocol);
Server.Serve();