From deterministic to probabilistic SEM bid optimization

The “sleeper” problem

Imagine an extreme (but likely) scenario where ad spend is significantly over the allocated budget for the month. The SEM expert can do a few things to quickly get the account back in shape:

• Decrease bids until cost falls to acceptable levels
• Install or lower the budget ceiling for some campaigns
• Pause under performing ad units that contribute to the majority of spend

The account will eventually stabilize after a week or two but by then the ad auction might operate within a completely new set of parameters. One symptom of this is revenue would be lower than prior time periods.

In our first example, the original bid could have started at $1.00 but bidding down -15% for 5 consecutive days would result in a final max CPC of $0.44. Although helpful to curb spend, this drastic -56% change has the potential to diminish the ad units efficacy & competitiveness where it no longer shows up in the marketplace.

To complicate matters this behavior might be observed across hundreds of thousands of other ad units as well. More specifically though, we effectively put the account to “sleep”.

Our goal for this use case: identify and set max CPC bids at scale for criteria which no longer generate any revenue.

How do we go about solving the above? Before showing the interesting stuff let’s first work through our problem with a typical approach.

Deterministic optimization

Note: this is also known as rules-based bidding in the industry

One solution is to complete a YoY gap analysis:

1. Pull performance data for the last X number of days
2. Pull data for the same time period last year
3. Highlight ad units which used to drive clicks and/or revenue
4. Filter on criteria where CPC is lower than last year
5. Set max CPC bids to what it was for the previous year

Mission accomplished!

Of course, there are always drawbacks to any solution. Below are some which immediately come to mind:

• The SEM specialist needs to think about the appropriate lookback ranges (14 days? 28 days? 105 days?)
• We consciously neglect information when we hand pick dates
• There is the risk of “waking up” historically under performing ad units

The above approach can reset our bids to produce conversions again but it may lead to repeated costly mistakes and diminish overall bid optimization efforts.

Probabilistic optimization

A different method to decipher our “sleeper” problem is with the use of probabilities (also see: Monte Carlo simulations).

Let’s suppose we have daily historical data for a single ad unit and its average CPC comes out to $0.85. We can take that information and plot it out as such (assumes a normal distribution):

What this chart tells us is the range of possible CPC values for this ad unit. There are instances where we observe a CPC as low as $0.01, though most of the frequencies fall around the average of $0.85 yet it is unlikely to surpass $10.

An introduction to probabilities is out of scope for this article but the key takeaway for the next part is to calculate the probability when a given ad unit’s CPC meets or exceeds the target goal (1) versus when it did not (0).

From that data we can then model the average CPC distribution curve for every ad, their respective segments, and the likelihood of achieving its designated goal. We can use ROAS (short for return on ad spend) for our goal where the chart below visualizes the separate relationships from the previous example.

This is quite insightful and valuable:

• The range of “Exceeded ROAS” has an average CPC roughly less than $0.50 and a little over $1.00
• The “Missed ROAS” has a much broader average CPC range which can be anywhere from $0.01 to as high as $2.00

Unlike the latter segment, our “Exceeded ROAS” cohort has a tighter bound on potential CPC values - indicating lower variability but relatively higher confidence in the outcome.

To set our new max CPC bid for this criteria we then “walk” along both probability curves and calculate the greatest difference between them. In other words, we can be fairly confident by setting our new bid to $0.76 our ad would not only generate revenue again but also offer a positive return on investment.

Note: the bidding system can dial up/down the confidence threshold

What I particularly like about this approach is automatically filtering out ads which have lower confidence thresholds for the “Exceeded ROAS” (image below).

With all that being said and done though this isn’t a silver bullet for a couple reasons:

• We focus only on the exploitation side of the explore vs exploit equation
• This builds a model on historical data but performance marketing goals change all the time (ex: LY goal of 120% but TY set to 300% which means everything falls under “Missed ROAS”)

To address these problems we would then need to write a different set of SEM bidding algorithms.

Wrapping up

By leveraging probabilities and simulation methods, we can create a model which understands the range of ROAS certainty for a given max CPC bid per ad unit. Additionally, with this technique we also resolved some of the issues that plagued deterministic bidding:

• Remove the need to set lookback date ranges
• Incorporate the full range of historical performance data
• Minimize the risk of bidding up horrible criteria (again)

Other applications

• Significance testing of click-through rate (CTR) changes
• Computing the probabilities of organic search keyword ranking movement
• Estimating the likelihood an onsite change would impact search engine crawler behavior

Related

• Google Ads: digging deeper into target ROAS bidding
• 10x SEM performance: unlock the power of your own data
• Examining drug effectiveness studies via simulation