How to write a bidding agent

Introduction

In this document we'll write a simple fixed price bidding agent. This will be made up of three major components:

• The agent's configuration which controls the type of bid requests that the router will forward to the agent.
• The bidding logic which decides how much we want to bid on each spots.
• The pacer which controls the rate at which we spend our budget. The annotated source code for the full bidding agent that we will present in this example is available in rtbkit's repository.

Agent Configuration

In this section, we'll create an AgentConfig object which will hold basic information about our bidding agent as well as setup various filters which will ensure that the routers will only forward bid requests that we're interesting in.

Let's get started:

AgentConfig config;

void setConfig()
{
	config = AgentConfig();

Pretty straightforward but we had to start somewhere.

	config.account = {"a", "very", "long", "account", "name", "thingy"};

Here we set up the account using the new and fancy C++11 initialization list[^1]. Accounts are used to keep track of the budget associated with each campaign and they're organized in a hierarchy. How this hierarchy is setup is entirely up to the writer of the bidding agents. A typical setup would be <campaign>.<strategy> but, as you can see in the example, you can use whatever silly setup your mind can conjure.

A more lengthy description of accounts can be found in the banker's documentation.

    config.creatives.push_back(Creative(160, 600, "LeaderBoard"));
	config.creatives.push_back(Creative(300, 250, "BigBox"));
	config.creatives.push_back(Creative(728, 90,  "LeaderBoard"));

This bit sets up the format of the creatives that our agent is interesting to bid on. Any bid requests that doesn't contain at least one spot that fits one of these impression formats will be filtered out by the router and will never make it to our bidding agent. Less work for us, woo!

In addition to this mandatory filter, there are several other optional filters that can be specified:

• hosts and urls
• languages and locations
• segments
• exchanges
• postion relative to the page's fold
• hour of the week
• user partitions
• augmentation tags (see below)

	config.addAugmentation("frequency-cap-ex", Json::Value(42));
	config.augmentationFilter.include.push_back("pass-frequency-cap-ex");

Next, we tell the router that we'd be interested in having our bid requests augmented by our frequency cap example and set the frequency cap ceiling to 42. Since we don't want to see bid requests that haven't been augmented or that have hit our frequency cap ceiling, we also tell the router to filter out any bid request that doesn't contain the augmentation tag set by the frequency cap example.

In this case, we use the augmentors output purely for filtering but it's still possible to retrieve information injected by an augmentor while we're making our bid decision.

	doConfig(config.toJson());
}

Finally, we tell the world about our new shiny configuration using BiddingAgent's doConfig function. When called, doConfig will publish the agent's configuration to the AgentConfigurationService which will then repeat it to every AgentConfigurationListener which includes the router, the post-auction loop and any augmenters that have per agent configuration.

Note that an agent can change its configuration at any time by calling doConfig. This is useful to update certain filters like bidProbability or maxInFlight which regulates the volume of the bid request stream that makes it to the agent.

Bidding Logic

Now that we've told the router what kind of bid requests our agent is interested in, we can start bidding. Note that since all the nitty gritty filtering is handed off to the router, the agent can concentrate solely on its bidding logic. The router is also in charge of enforcing time constraints on each bid requests. That makes our life that much more easier.

void bid(
    double timestamp,
    const Id & id,
    std::shared_ptr<RTBKIT::BidRequest> bidRequest,
    Bids bids,
    double timeLeftMs,
    const Json::Value & augmentations)
{

For our example, we'll shove the entire logic of the bidding agent within the bid function which will be called by the BiddingAgent class everytime it receives a bid request from a router.

    for (Bid & bid : bids) {

We'll use the bids object to make our bid. It contains an entry for each of the available spots that match our agent configuration.

		int availableCreative = bid.availableCreatives.front();
		(void) config.creatives[availableCreative];
		(void) bidRequest->spots[bid.spotIndex];

Each Bid object contains:

• A set of creative indexes which indicates the creatives of our configuration that we can use to bid on the given spot.
• A spot index which we can use to query the bid request for more details about the spot we're trying to bid on. Additionally, we can peruse the bid request object and the augmentations json blob to gain additional data to inform our bid decision. For our example, we don't need fancy logic so we'll just pick the first available creative and continue on our merry way.

	    bid.bid(availableCreative, USD_CPM(2));
	}

Here we register our bid with the Bid object by passing along the creative we wish to display and the amount we believe our bid is worth. To specify monetary values, we use the Amount class which ensures that we don't go bankrupt due to scale bad currency conversions or a mis-scaled value. For example, if we wanted extra precisions for bid price, we could use the MicroUSD_CPM struct.

    Json::Value metadata = 42;

If required, we can also attach some metadata to our bids which will be passed back to our bidding agent in the bid results and the post auction events.

	doBid(id, bids, metadata);
}

Moment of truth! Time to place our bid with the router by calling BiddingAgent's doBid function. Once the bid is placed, Within the next few milliseconds, the router will reply with the result which will trigger one of the following BiddingAgent callbacks:

• onWin: we won the auction for at least one of our bids.
• onLoss: we loss the auction for all our bids.
• onNoBudget: our account doesn't the required budget to place the bid.
• onTooLate: the auction was sent back to the exchange before we could place our bid.
• onDroppedBid: the router didn't receive a bid response for a given bid request.
• onInvalidBid: something went wrong with our bid. If we won the auction, the post-auction loop may trigger these additional callbacks:
• onImpression: our creative was shown to the user.
• onClick: the user clicked on the creative.
• onVisit: the user did an action after having clicked on the creative.

Pacer

The final missing piece for our bidding agent, is the periodic allocation of budget also known as pacing. Allocating our budget in small chunks distributed over the entire duration of the campaign ensures that we don't blow our entire budget in the middle night. Budgets are managed by the master banker which uses the account of the various agents to distribute budgets from parent accounts to child accounts.

The following pace function will contain our pacing logic and will be called periodically (we'll see how a little later).

SlaveBudgetController budgetController;
bool accountSetup = false;
void pace()
{
	 if (!accountSetup) {
		 accountSetup = true;
		 budgetController.addAccountSync(config.account);
	 }

So the first step is the get the account information for our bidding agent from the master banker. We do this by initiallizing a SlaveBudgetController object which will act as a proxy to the master banker. This controller periodically communicates with the master banker to keep its budgets up to date.

	 budgetController.topupTransferSync(config.account, USD(1));
}

All that remains is to transfer an amount from the our parent's account into our agent's account and we're done. Simple as pie!

Putting It All Together

Almost done! All that's left is a little bit of glue code.

struct FixedPriceBiddingAgent : public BiddingAgent {

Let's create a agent's class which will house the setConfig, bid and pace function that we developped earlier. This class should either compose or derive the BiddingAgent class which handles the boring router protocol details.

    void init()
    {
		strictMode(false);
        onBidRequest = bind(
                &FixedPriceBiddingAgent::bid, this, _1, _2, _3, _4, _5, _6);

In the init function we initialize the various components that make up our service. We start by setting up the bidding agent's callbacks. The strictMode function call supresses errors when BiddingAgent receives a message for which there's no callback defined. Disabling these checks is useful when writting simple example agents or tests.

        budgetController.init(getServices()->config);
        budgetController.start();

Next we setup our proxy to the master banker used by our pacer.

        addPeriodic("FixedPriceBiddingAgent::pace", 10.0,
                [&] (uint64_t) { this->pace(); });

Here we exploit the fact that BiddingAgent is a MessageLoop by adding a periodic event source which will call our pacer every 10 seconds.

        BiddingAgent::init();
    }

Finally, we make sure that BiddingAgent is also initialized.

    void start()
    {
        BiddingAgent::start();
        setConfig();
    }

In the start function we start the various message loops that compose our service. While we're at it, we also take the opportunity to tell the world about the configuration for our agent so that we can start receiving bid requests.

Note that I'm skipping the shutdown function because, as we'll see below, our service is not meant to be shutdown. In fact, the shutdown functions in the RTBKit services are only ever usefull when writting tests. Otherwise, none of the production executables call shutdown. Closing a production service can be accomplished by simply issuing a signal to the process. This implies that all our services are crash resistant (or will be eventually) and so should any bidding agent you write.

int main(int argc, char** argv)
{

Home stretch!

	ServiceProxies servicesProxies;

RTBKit requires a Zookeeper instance to do service discory and a Carbon instance to dump some runtime metrics. These are specified to the service using the ServiceProxies class which acts as a proxy to these services and the easiest way to construct an instance of this class is to use a bootstrap.json file.

	FixedPriceBiddingAgent agent(serviceProxies, "fixed-price-agent-ex");
	agent.init();
	agent.start();
	while (true) this_thread::sleep_for(chrono::seconds(10));

We finish things up by instantiating our agent using the service proxies object we just created and a unique service name which will be used for discovery. All that's left now is to initialize the agent, and start it and put the main thread to sleep while the background message loop does its thing.

And that's it. You now have a production ready-ish fixed price bidding agent. Enjoy your stay in RTB land!

[^1]: RTBKit makes heavy use of C++11 so learn it and love it. Well makes heavy use of the subset of C++11 that gcc 4.7 supports.