Extension
Extension plug-in is a templated system to extend some core components. By "extend", we mean adding new data members and member functions without having to change the core code. For now, extension plugin is used to extend the AgentConfig and Creative.
File extension.h defines the extension mechanism and contains following classes:
- struct Extension is the base class that all other extensions derive from
- struct ExtensionPool is built to contain a sub set of all the extensions registered to PluginTable
- struct ExtensionRegistry is used register extensions to PluginTable and to retrieve extensions from that same table
The following figure shows the steps taken during the initialization of extensions
Steps to follow to create and use an extension are:
- Implement an extension
- Load dynamically the extension into the PluginTable
- Load the extension from PluginTable into the ExtensionPool contained inside the component that needs to be extended and initialize the extension. Note: The struct Creative and AgentConfig both contain an ExtensionPool data member. This ExtensionPool is filled with extensions that the name is given in Creative or AgentConfig configuration file (see Creative::fromJson)
- Use the extension
In the rest of this document, we will describe the above steps by generating and using an AgentConfig extension.
Note that the extension may have many data members and member functions that have different parameters and return different types. For simplicity the extension of this example contains only void returning functions and 2 data members.
Also note that you can define two different extensions in the same file.
You can create a new AgentConfig extension under the folder rtbkit/plugins/extension.
#pragma once
#include "rtbkit/common/extension.h"
namespace CUSTOM {
struct AgentConfigExtension : public RTBKIT::Extension {
public:
NAME("AgentConfigExt");
AgentConfigExtension();
virtual void parse(const Json::Value & value);
virtual Json::Value toJson() const;
void doSomething();
void doSomethingElse();
int extension_variable_int;
std::string extension_variable_string;
};
} // namespace CUSTOM
#include "custom_agent_config_extensions.h"
using namespace RTBKIT;
using namespace ML;
namespace CUSTOM {
AgentConfigExtension::AgentConfigExtension() {}
void AgentConfigExtension::parse(const Json::Value & value) {
/* Lets say value = {"ext_var_int" = 5, "ext_var_string" = "my_string"}
Parse Json::Value value and set the new values
of extension_variable_string and extension_variable_int */
}
Json::Value AgentConfigExtension::toJson() const {
Json::Value result;
/* Build result probably using extension_variable_string
and extension_variable_int */
return result;
}
void AgentConfigExtension::doSomething() {
/* custom code */
}
void AgentConfigExtension::doSomethingElse() {
/* custom code */
}
} // namespace CUSTOM
namespace {
struct Init {
Init() {
RTBKIT::ExtensionRegistry::
registerFactory<CUSTOM::AgentConfigExtension>();
}
} init;
} // namespace anonymous
Once the extension is ready, you can add it to local .mk file
$(eval $(call library,my_custom_extension,custom_agent_config_extension.cc,rtb))
To load the extension, you can use the preload option of ServiceProxyArguments. You can add this option to all processes supporting preload option. In this example, the option to add is --preload build/x86_64/bin/libmy_custom_extension.so. This option will register your extension to PluginTable
{
"nodes": [
{
"name": "random",
"root": ".",
"tasks": [
{
"name": "custom-bidding-agent",
"root": ".",
"path": "build/x86_64/bin/custom_bidding_agent",
"arg": [
"-N", "st2.agent",
"-B", "sample.bootstrap.json",
"--preload", "build/x86_64/bin/libmy_custom_extension.so"
],
"log": true
}
]
}
]
}
Once the extension is in PluginTable, we need to add it to the ExtensionPool of AgentConfig. This is done in createFromJson function of AgentConfig. All names that does not feat any pre-defined field will be considered as an Extension plug-in name. Each extension will be initialized (create function calls to fromJson function of the given extension) and saved into the ExtensionPool of AgentConfig (add function)
Hence, in our example, the Json::Value passed to createFromJson should be of the following form:
{
"name" = "agentName",
"id" = "12345",
...
"AgentConfigExt" =
{
"ext_var_int" = 5,
"ext_var_string" = "my_string"
}
"AnotherCustomExtension" =
{
...
}
"YetAnotherCustomExtension" =
{
...
}
}
Note that toJson function of all extensions registered to ExtensionPool, is called inside toJson function of AgentConfig
Lets say we would like to use the AgentConfig inside a bidding aggent. Lets say our agent contains an AgentConfig and a pointer to our agent config extension.
#pragma once
#include "rtbkit/plugins/bidding_agent/bidding_agent.h"
#include "bidding_agent_ext.h"
...
namespace CUSTOM {
struct CustomBiddingAgent : public BiddingAgent
{
CustomBiddingAgent(std::shared_ptr<ServiceProxies> proxies,
std::string name,
std::string const & filename);
...
void aRandomFunction();
private:
Datacratic::AgentConfig config;
std::shared_ptr<const AgentConfigExtension> ext;
};
} // namespace CUSTOM
#include "custom_bidding_agent.h"
...
using namespace RTBKIT;
using namespace Datacratic;
namespace CUSTOM {
CustomBiddingAgent::CustomBiddingAgent(std::shared_ptr<ServiceProxies> proxies,
std::string name,
std::string const & filename) :
BiddingAgent(proxies, name)
{
ML::File_Read_Buffer buf(filename);
Json::Value agent_config = Json::parse(std::string(buf.start(), buf.end()));
// Initialize agent config and agent config extension
config = AgentConfig::createFromJson(agentConfig);
// extract the extension from AgentConfig ExtensionPool
ext = config.extensions.get<AgentConfigExtension>();
...
}
CustomBiddingAgent::aRandomFunction()
{
ext->doSomething();
ext->doSomethingElse();
std::cout << "My extension int variable: "
<< ext->extension_variable_int << std::endl;
std::cout << "My extension string variable: "
<< ext->extension_variable_string << std::endl;
}
}
int main(int argc, char** argv) {
using namespace boost::program_options;
ServiceProxyArguments globalArgs;
std::string filename;
options_description options = globalArgs.makeProgramOptions();
options.add_options()
("help,h", "Print this message")
("agent-configuration", value<std::string>(&filename),
"Configuration filename");
options.add(bankerArgs.makeProgramOptions());
variables_map vm;
store(command_line_parser(argc, argv).options(options).run(), vm);
notify(vm);
if(vm.count("help")) {
std::cerr << options << std::endl;
return 1;
}
if(filename.empty()) {
std::cerr << "missing configuration file" << std::endl;
}
auto proxies = globalArgs.makeServiceProxies();
// start the bidding agent
const std::string name = "custom-bidder";
CUSTOM::CustomBiddingAgent agent(proxies, name, filename, banker);
agent.start();
while(true) {
std::this_thread::sleep_for(std::chrono::seconds(10));
agent.report();
}
return 0;
}
The agent config extension is initialized in the constructor of CustomBiddingAgent . The extension is then used in aRandomFunction function. The file used to initialize agent config and agent config extension is passed to the process as a command line option agent-configuration