Multi language bidding agent

In this document, we shall present two distinct approaches to a multi-language agent for RTBbit. But since they both rely on a somewhat simplified interface to the bidding agent, we shall touch first about the advantages to rolling out a simplified C++ interface to the bidding agent in a first time -- and since these two sub projects are interrelated, perhaps conclude that the multi language agent project is an occasion to kill two birds with a same stone.

simplified C++ interface to the bidding agent.

The RTBkit framework is a complex framework involving many advanced concepts, which the user of the framework often have pain to apprehend. In a way a majority of our users are mostly concerned about business matters, that is realtime bidding in electronic/algorithmic advertising. And just because this topic involves advanced computing notions, does not imply that our users should master them all. Similarly, users of a modern no-sql engine (such as Cassandra) are not expected to know everything about distributed consistency.

This remark is also valid about the C++ level required in order to apprehend RTBkit: it should be possible to use the framework without having to be a medium to advanced C++ developer, which seems to be the case for many.

need for a multi language bindings to the bidding agent.

And for a start, many users do not apprehend C++ at all nor are willing to learn it. If C++ is massively used in Finance, the advertisement business is somewhat tainted with web-related technologies, such as php or node-js (or even worse java). Another incentive to multi-language bindings, is the need to quickly roll out new strategies, would it only be for the sake of testing, as it is often the case in finance: trading strategies are often prototyped and tested in Python, and only when performance require, turn it in C++.

Classically, every possible "exit" language (python, c#, java, ruby, ...) has some kind of C (or even C++) native API, which makes it possible to go back and forth from the exit to C/C++.

However, on from the software engineering perspective, this may imply result in as many specific sub-projects, as there are exit languages considered. This altogether make the case for considering third party softwares, such as SWIG, which we shall only consider as a solution thereafter.

simplified C++ interface: description.

A simplified C++ interface, should capture whatever is needed for an agent to interact with a router, and nothing more, obviously letting aside matters related to implementation choices/necessities. It is also in our interest to keep the data types involved as closed as possible to C/C++ primitive data types and plain old data structures. Basically, a bidding agent performs the following operations.

• "discovers" the various bits belonging to its performing environment.
• identifies itself against a router
• receive notifications from the router: bid requests, bid results, deliveries notification, error notifications.
• place bids against the router.

From this respect, it is an I/O automaton, exchanging (sending and receiving) messages with its environment:

• receives a "proxy" message describing its running environment
• sends a "config" message describing its performing context (configuration) to the router
• receives notification messages: "bid-request", "bid-resut", "delivery", "error" from the router
• sends "bid-response" messages to the router

Each of these messages is captured either by a string or a collection of strings representing JSON objects, which along with the verbs of the interface, are described in this piece of code.

simplified C++ interface: asynchronous.

The current C++ bidding agent is half asynchronous: it sends synchronously to the router, and receives asynchronously from the router (more precisely, it sends synchronously its bid-response to a local queue). Messages from the router are received via dedicated callbacks, also as described here.

simplified C++ interface: swig file.

Basically, SWIG a kind of pre-compiler, which takes a swig-file as an input, and generates whatever stubs are needed to build whatever software layer, in order to connect to the C/C++ code described by the swig-file, from the chosen exit language. Note that it is not a one way journey, as calls can originate from either the C/C++ or the exit language. Ideally, the swig file needs to be written only once for all the exit languages considered. Here is the swig file used for our asynchronous lightweight RTBkit interface.

simplified C++ interface: Cross language polymorphism.

Recent versions of SWIG provide for a feature called cross language polymorphism. This allows, in the exit language targeted, to override virtual member functions defined in C++. This of course, assumes that the exit language offers enough support for OO (unlike LUA, which does not support this feature).

In our case, the simplified C++ interface defines one callback object per message type, each with a virtual method call. This call back object needs to be extended in the exit language and the extension needs to override the inherited call method. The simplified C++ interface also exposes a doBid method, which can be called from the exit language (typically in the the call method of the extended BidRequest).