Frequently Asked Questions

What is RTBkit?

RTBkit is an open-source software framework that takes much of the hard engineering work out of creating a Real-Time Bidder for online advertising. Its open, service-oriented architecture can be used to assemble a bidder as simple or complex as desired. The RTBKit core connects to ad exchanges via Exchange Connectors and routes bid requests and data through a configurable set of components which can be extended to implement a customized bidder. More details.

Permalink   ·   Posted on 02/06/2013 in Business

Who can use RTBkit?

Anyone can use RTBkit to build a real-time bidder. RTBkit is open-source software, so it is free to use within the bounds of the Apache License.

Permalink   ·   Posted on 02/18/2013 in Business

How much does RTBkit cost?

RTBkit is open-source software, it is feely available within the bounds of the Apache License.

Permalink   ·   Posted on 02/18/2013 in Business

How can I get started with RTBkit?

RTBkit is software framework which developers can use to build real-time bidders. The first step is to download and explore the code and design your bidder around the RTBkit core and plugin architecture. If you need help, don't hesitate to join our Google Group or find a member of the RTBkit ecosystem to help you.

Permalink   ·   Posted on 02/18/2013 in Business, Technology

What is needed to build a bidder using RTBkit?

Building, deploying and operating a real-time bidder is a complex project. RTBkit can help you with only part of it. RTBkit is not a solution in itself, only a framework, and it doesn't help you with any of the following, all of which you will probably need to complete your project:
  • Develop the GUI to manage your RTB campaigns and to offer reporting to your users
  • Implement the logic that decides how much to bid for each campaign
  • Launch, configure and monitor your bidding agents
  • Launch, configure and monitor your RTBkit installation
  • Handle billing
  • Serve the ads
  • Upload the creatives
  • Find the right servers to host your services on
  • Develop and maintain relationships with the ad exchanges
In terms of skills, keep in mind that working around RTBkit requires a fair understanding of Linux, C++ and make.
 
Permalink   ·   Posted on 09/13/2013 in Business

How can I get involved?

The best way to get involved is to download the code and start playing with it, join the Google Group to ask/answer questions, and eventually contribute some code back to the project.

Permalink   ·   Posted on 02/19/2013 in Business, Technology

Which software language is RTBkit built with?

RTBkit is written in high-performance C++11 and runs on Linux (Ubuntu 12.04).

Plugins extend C++ classes and plugin bindings currently exist for Javascript (V8/NodeJS). Bindings for other languages (Java, Python etc) should be reasonably straight-forward to build.

RTBkit services and components communicate with each other via ZeroMQ and HTTP. RTBkit uses Graphite to record statistics and Apache ZooKeeper for service discovery.

Permalink   ·   Posted on 02/18/2013 in Technology

What license is RTBkit released under?

RTBkit is released under the Apache License, v2.0.

Permalink   ·   Posted on 02/18/2013 in Business, Technology

Who is behind RTBkit and why?

RTBkit was developed and released by Datacratic and is intended to become a community-led open-source project, contributions are welcome by all.

For more information on why Datacratic released RTBkit see RTBkit: An Exciting Day for Real Time Advertising on Datacratic's blog.   

Permalink   ·   Posted on 02/18/2013 in Business

Is RTBkit secure?

RTBkit is a collection of components which are meant to run behind a firewall which permits connections only to authorized users and exchanges. It is not intended to exposed to the open internet. RTBkit is open-source software, which encourages open collaboration, especially around issues such as security.

Permalink   ·   Posted on 02/18/2013 in Technology

Is RTBkit scalable?

The RTBkit core is horizontally scalable and was designed to support 20,000 queries per second per high-end commodity server.

Permalink   ·   Posted on 02/18/2013 in Technology

Is RTBkit stable / production ready ?

RTBkit is stable. The 0.9 release has a stable API.
RTBkit is production ready.  Multiple RTBkit bidders are currently in production and the number will significantly increase in the upcoming months (August 2013).  
Permalink   ·   Posted on 09/12/2013 in Business, Technology

Does RTBkit have a user interface?

RTBkit does not include a user interface. The RTBkit core is a collection of software components which expose HTTP and ZeroMQ API's, which can be used as a back-end for an external user interface.

Permalink   ·   Posted on 02/18/2013 in Technology

Which Ad Servers is RTBkit compatible with?

RTBkit is ad-server agnostic and is designed to work with any ad-server. RTBkit bidding agents specify which ad-server tags to bid with via a configuration message to the RTBkit core. 

Permalink   ·   Posted on 02/18/2013 in Technology

Which advertising exchanges can RTBkit connect to?

RTBkit is an exchange-agnostic framework and is meant to be compatible with any ad exchange. The RTBkit Core communicates with exchanges via Exchange Connector plugins. A connector for The Rubicon Project and for Gum Gum are already available, and connectors for AppNexus, Nexage, The Doubleclick Ad Exchange and others will be released in the coming weeks and months. Community contributions of Exchange Connector plugins are welcome!

We're maintaining a list of connectors available and being developed here.

Permalink   ·   Posted on 02/18/2013 in Technology

Is RTBkit OpenRTB-compatible?

RTBkit is an exchange-agnostic framework and is meant to be compatible with any ad exchange. The RTBkit Core communicates with exchanges via Exchange Connector plugins and these plugins depend on bid-request parsers. RTBkit's internal representation of a bid request is similar to OpenRTB's representation, with full round-trip serialization as a goal. This means that OpenRTB-formatted bid requests are easy to parse, although exchange-specific connectors or parser extensions may be required, depending on the specifics of each exchange's use of OpenRTB.

Permalink   ·   Posted on 02/19/2013 in Technology

Does RTBkit support Video/Mobile exchanges?

RTBkit is an exchange-agnostic framework and is meant to be compatible with any ad exchange, including video and mobile exchanges. The RTBkit Core communicates with exchanges via Exchange Connector plugins, and it is easy to write such plugins for OpenRTB-compatible exchanges. Connectors for major exchanges, including video and mobile exchanges, are expected in the coming weeks and months. Community contributions of Exchange Connector plugins are welcome!

Permalink   ·   Posted on 02/19/2013 in Technology

What is an augmenter?

An augmentor is an optional component responsible for augmenting a bid request with data as it goes through the router. This data is delivered to each agent that asked for the augmentation and can also be used by the router to apply custom filtering on the bid requests destined for a given agent.
 
A frequency cap augmentor for example could add the frequency count (the number won bids by an agent for a given user) to the bid request so the router can decide whether the bid request should make it to the agent.
Permalink   ·   Posted on 09/12/2013 in Technology

Is there support for languages other than C++?

Currently there is support for building Bidding Agents in Javascript or Coffeescript. There is also an HTTP Bidder Interface which allows you to write a bidding agent in any language that can accept and respond to HTTP (so, basically, any language).

The goal on the roadmap for RTBkit 2.0 is to open up Augmenters, Augmenter Data integration, logging and analytics plugins to be available to the RTBkit core through HTTP interfaces.  The BidderInterface that is in the current master branch is an early taste of this functionality.

The core of RTBkit is written in C++ and many of the components (i.e. exchange connectors) can only be written in C++. Therefore, it is highly recommended that developers working on RTBkit have C++ experience. 

Permalink   ·   Posted on 08/29/2014 in Technology

What are some of the key operational considerations when thinking about RTB?

Running an RTB system is not trivial. 

You need to consider the following areas:

Cloud vs. buying and hosting your own hardware (i.e. lease vs buy )

Operations

  • Hosting - in the cloud or dedicated data center
  • 24-hour logging, monitoring and support
  • Supporting a high-volume distributed system, running on commodity hardware and using basic protocols, connected to other partners on the public Internet. You want ops people with this kind of experience.

Operating Costs

  • Don't overspend on handling more bid requests then you need. Think about your planned spend, and the number of impressions you want to buy. A good rule of thumb is that you will win about 1 out of 100 bid requests you see incoming, so if you assume a CPM typical for your campaigns you can calculate a rough number for the expected volume you will need to support.
  • From that you will want to prototype your system architecture and load test to understand how many hosts you project you will need to run.
  • From that you can calculate a rough monthly operating cost for hosting.
  • You should target 5% of media spend as the limit of what you spend on hosting
Permalink   ·   Posted on 08/29/2014 in Technology

How responsive is the community in case I run into development challenges?

The RTBkit Google Group has new posts every day. Several Datacratic developers answer questions on the group daily, including developers who have worked on the project since its inception. In addition, a core group of developers from the RTBkit community regularly participates. In general, you can expect an answer within 24 hours to a question you post on the group.

Visit the group at: https://groups.google.com/a/rtbkit.org/forum/#!forum/discuss
 

Permalink   ·   Posted on 08/29/2014 in Business

Does RTBKit uses or have hooks to KV database like Redis?

Yes, RTBkit uses Redis in the Banker and also ships with a Redis Augmenter connector.  You can review the code here:

Permalink   ·   Posted on 08/29/2014 in Technology

We would like to log every bid request we receive. Where would you log it? On every exchange connector?

It is probably not advisable to log every incoming bid request you receive. This is a massive amount of data and the storage costs will quickly escalate, probably beyond any value you might get from having all of this data on hand. Depending on your use case and your reasons for wanting to log the data you might want to consider sampling the bid stream or logging the full bid request for each impression won, and/or click and/or conversion.  You might also consider forwarding bid requests to a streaming endpoint that retains a rolling limited window of data, such as Kafka or Storm, if your goal is to feed bid requests into an analytical system looking at (and storing) data for a limited time-frame.

With all that said, bid requests can be logged via the RTBkit data logger by setting the Router option log-auctions to true.

 
Permalink   ·   Posted on 08/29/2014 in Technology

Is RTBkit scalable enough to add new bidding agents?

RTBkit should scale comfortably at least to low 100s of agents. The only issues we have heard about in production were when customers had 1000s of agents. 

The recently released HttpBidderInterface and MultiBidderInterface may also help with scalability, because agents can now run out of process from the RTBkit core, on separate hosts in their own processes. This decoupling allows you to be able to truly scale agents horizontally separately from RTBkit core.

Also, if you have this many agents, and the reason is that you have many agents per campaign, you should perhaps think about your agent design. Often, non-RTB systems execute broad optimization strategies, such as segment-based targeting, but subdividing a campaign budget into many child campaigns, each with different targeting settings. If this is your reason for having so many agents, perhaps a different approach that puts more of the work of optimization in dynamic decision making by a smaller number of bidding agents might make more sense.

Permalink   ·   Posted on 08/29/2014 in Technology

Are there plugins (open-source or paid) for campaign management, pacing or frequency capping available?

There are examples of how to implement frequency capping included in the RTBkit wiki, and example code.

Wiki:
https://github.com/rtbkit/rtbkit/wiki/How-to-write-an-augmentor

Code:
https://github.com/rtbkit/rtbkit/blob/master/rtbkit/examples/augmentor_ex.cc
https://github.com/rtbkit/rtbkit/blob/master/rtbkit/examples/augmentor_ex.h

For pacing, the only commercial system we are aware of is our Datacratic Bid Optimizer, a commercial, hosted, RTB optimization service compatible with RTBkit through the HttpBidderInterface and with any third-party bidder. Bid Optimizer includes pacing as a feature.

For campaign management, RTBkit offers the Agent Configuration Service, but not a UI. The Agent Gateway (https://github.com/USMediaConsulting/agent_gateway) is a community-contributed service for managing this configuration.

 
Permalink   ·   Posted on 08/29/2014 in Technology

Is there a Recency Control? As an example, the bidder controls how often we bid for a user based on when we served an impression. It is not exactly a frequency cap?

There is not. But, as with frequency capping, you can implement this using an augmenter. The pattern is the same - upload the current state of your user data (timestamp of the last time they saw an impression from Campaign X), and read that per bid request in your augmenter. The augmenter would implement the business rule to set a filter per eligible campaign based on the last time the user had seen an impression from that campaign.

Permalink   ·   Posted on 08/29/2014 in Technology

Are there any requirement/guidelines to integrate a homegrown DMP to RTBkit?

There are not, in that the system boundary defined by RTBkit stops at providing an Augmenter base class that you can derive from, and the RedisAugmenter class you can use as is, that connect RTBkit core to your Augmenter logic and data store.

That said, there are some general design issues that you should consider. 

First, very fast key/value stores such as Redis or Aerospike have appropriate performance characteristics for an Augmenter, so you should think about a schema for your Augmenter data that does lookups based on a single key, for example a userId or deviceId. You want to minimize reads from the database, so a good design is often to have a long record of attributes associated with each key, for example, all the attributes you have available for a userId or deviceId. This way you can fetch all the data for the augmenter operation with one read against a key.

Second, you should think about how fresh your data needs to be. Data integrations are generally operationally complex and costly, and you should only load your data as often as you need to to support your use cases. If you are retargeting users very near time of purchase ("bottom of the funnel"), you might need near real-time data updates. If you are putting users into a segment targeting users considering a major purchase, for a 30-day campaign, you may only need to upload data daily.

Permalink   ·   Posted on 08/29/2014 in Technology

If we set the bidProbability to 1, would we have performance issues or other issues?

This can cause significant performance issues in RTBkit core, depending on how many bidding agents you are runing.   bidProbability is a factor for the bid requests that pass all filtering and reach your agents.  But if all agents bid, the router needs to run an internal auction over all of the responses, which causes additional load.

 
Permalink   ·   Posted on 08/29/2014 in Technology

What is the ballpark implementation time when using RTBkit?

It does really depend on the skill and experience of the engineering team. You should expect a small team of engineers should take from two to six months to implement a bidder using RTBkit. Much of that time will likely be spent on integration with inventory sources, your ad server, and an existing campaign UI, if you have one. Implementing an augmenter will certainly add development time due to the data transformation and integration required.

Permalink   ·   Posted on 08/29/2014 in Business

What is average response time for a bid request in RTBkit.

RTBkit itself typically adds 0.5 to 2 ms of latency to a no-bid request for the bid request parsing, filtering, internal auction and responding to the exchange. Augmentation will add another 2-7ms, and bidding agents will add extra time.  Note that RTBkit attempts to respond to exchanges within the time guidelines in the bid request, irrespective of the behaviour of augmenters and bidding agents.

Permalink   ·   Posted on 08/29/2014 in Technology