Thursday, May 6, 2010

Demystifying Real Time Bidding


Real-time bidding (or as we like to call it, RTB) is major buzzword in the ad industry today. It hints of the term "Behavioral Targeting" back a couple of years ago, where anyone who wanted a play in this industry claimed to do it. However, unlike behavioral targeting, RTB is quite real and very effective. Problem is its poorly understood and complicated to build against. I've read a ton of articles dismissing this solution due to lack of clarity (and just sat in another meeting discussing the how-to while typing this blog). What I hope to accomplish is to shed some light on both the advantages and disadvantages of RTB so you can make up your own mind and see if it works for you.

First and foremost, what is it?
Simply stated, real-time bidding brings the ad buy down to the impression level.

Historically, when you buy through an exchange (ADX, RMX, name it) you create bids that are pseudo-static -- every time you see an impression that fits the criteria you are looking for, you bid a set price. There are obviously subtle complexities to all of this but for simplicity lets assume that if bid 25-cents, you will always bid 25 cents. If you want to bid higher or lower, or on different targets, you change bid price and/or create more line items and targets and hope for the best. Thus, bidding in the standard exchange is like bidding for a crate of goods, hoping the crate contains the items you sort of asked for, and praying everything is in good condition.

Real-time bidding is more like buying each product in the crate, at varying prices, and assembling yourself. In other words, you can inspect a wide variety of attributes about the impression and make a dynamic decision with respect to price. You can still do the bulk buy and hope for the best, but you have all the information you need to make a quantitative decision (something you cannot do on historical exchanges without creating thousands of line items).

How does it all work?
When an end-user goes to a site that wants an ad, the real-time exchange receives this request and sends the information to the various bidding machines that are connected to it. Each bidder answers back with price and a creative they want to show. The exchange simply chooses the highest bidder and sends the ad along its merry way. So how is that different? The exchange didn't do any magic here: no predictive modeling, throttling, pacing, budget look-up, etc. The exchange was simply the conduit and final arbiter. The bidder did the bulk of the work, acting like its own mini-exchange, so to speak.

What is on a Bid Request?
A Bid Request contains a wealth of information about an impression so you can bid intelligently. Obviously you can't see everything, but in the old exchange you can't really see anything. Each real-time exchange offers their own flavors of special sauce, but everyone gives you the standard set of information you can target against in existing exchanges.  Some exchanges will also allocate some cookie space and/or third party pixel data, which is always valuable when used correctly.

Seems easy ... its not, right?
Everything is pretty straightforward and seemingly simple. However, if it were simple everyone would do it. Let's start getting into where this gets tricky.

1. You need a strong developer (or team) to build real-time bidder. This is someone with domain expertise, good coding practices, knowledge of network protocols, blah blah blah. They are expensive and drink too much coffee.

2. You have (give or take) 30 to 120 milliseconds to answer. This is really easy if you always bid $1.00 for the same ad over and over again, but lets be realistic, that sort of spits in the face of RTB. Compound this by the ever-continuous pelting of requests for bid and its quite tricky to pull this off in a timely fashion. Shoot for code that can handle 1 billion requests in a single day, per machine.  I've seen some go significantly higher and others way lower, but that number is a good scaling factor.

3. You SHOULD build an algorithm. This doesn't mean you need to hire NASA, or 30 quant guys from Wall Street, but you need something that can logically determine what the value of the impression is for the creative you want to show.  At a minimum, determine some prices for the targets you want to hit and get that into the machine.

4. Each RT exchange is a little different (the next article I will post is actually about exchange normalization, but lets not go there yet). You need to build the network protocols, requests, and responses in a vacuum.  That being said, doing things intelligently across exchanges becomes a little tricky.  Worst case, pick a big exchange, write code for them, and then see where it goes.

5. Data, more data, and even more data.  On the cool side, you have access to everything you bid on (transparency article coming soon too).  If you have a few terabyte storage devices sitting around, go save that data and analyze.  If not, well bust out some investor money and start planning.  It's a wealth of information that significantly improves your chances of successful ad buying.  Be prepared to hire a good data architect to mine this stuff (see issue #1 as these people tend to hang out with those people).

So where does that leave us?
The more and more I sit in meetings to discuss RTB, the clearer the picture gets. Yes its complicated, but the positives far out-weigh the complexities because you can really target effectively at the EXACT time you need to. It also gives you immediate insight into response rates and efficacy. Furthermore, it adds a tremendous amount of transparency to the spot buy, something everyone has wanted for way too long.  Just think long and hard when deciding if you build it yourself, or utilize one on the DSPs out there to be your execution arm.

3 comments:

  1. Great post.
    Can you elobprate a bit about the data. I keep reading that its data data data but don't quite get what kind of data you can gather and what you can do with it.

    Thanks again.
    JJ

    ReplyDelete
  2. I would also agree. Awesome post. I to find myself getting the picture the more I talk about it with others, as its an ever evolving monster. There are just so many complexities in the online ad eco system.... can't wait to read more.

    ReplyDelete