By Jack Chung, Senior Programmatic Product Manager – AOL Canada
With programmatic media buying becoming mainstream and the proliferation of demand-side platforms (DSPs) in recent years, optimization capability has become a must-have feature for any worthwhile DSP. Surprisingly, easy-to-understand information on optimization systems and how to evaluate them is still very rare. Many people are still treating optimization systems as a mysterious black box that is supposed to “just work”.
In this article, we will look into what a programmatic optimization system is, in easy to understand terms, and the questions you should be asking when evaluating different systems.
What Is Programmatic Optimization?
In a nutshell, optimization software takes historic campaign data, runs it through a mathematical calculation, and then tries to establish the best bidding decision for every bid request that the DSP receives.
This decision typically includes two main components:
| 1. Select a campaign and creative out of every active campaign and creative in the DSP to respond to the bid request |
| 2. Determine the best bid price |
Some optimization systems are also responsible for pacing control and frequency capping. These are not mandatory responsibilities of the optimization engine. Other parts of the DSP platform can provide these functionalities. However, by building these into the optimization engine it is possible to deliver some additional optimization benefits.
How Do I Evaluate Optimization Systems Without A Phd In Math?
To evaluate optimization systems, it is not necessary to understand the complicated math behind it. However, you need to find out what it can do for you, and have a good understanding of how effective it is. How do you do that?
At the end of the day, nothing beats running a campaign and seeing how it performs. However, there are some questions you can ask without spending a penny and they will definitely give you a better understanding of the effectiveness of your system.
| 1. Does your optimization system perform the two main tasks: choosing a campaign and determining the best bid price? It is odd to think that this is an important question, but some optimization systems don’t actually perform both tasks. Some DSP systems choose a campaign at random (based on pre-determined targeting filters) and then ask the optimization engine for a bid price. On the other hand, some systems use machine learning to choose a campaign but the bid price is arbitrary and static. |
| 2. Can your system optimize to KPIs that you care about? How about secondary KPIs? Your campaigns will have different KPIs such as CTR, eCPC, eCPA, engagement rates, viewability, etc; you need to ensure that your optimization system can also accommodate your different KPI tracking needs. It is also a good idea to ask about secondary KPIs. For example, you may want to have a minimum CTR while optimizing for the lowest CPC cost. |
| 3. How does your optimization system choose which campaign to bid? Will it favour certain campaigns over others? Remember that the optimization system doesn’t just work for your campaigns, it is also optimizing all other campaigns running on the DSP at the same time. When a bid request is good for multiple campaigns, how will the optimization engine choose? If it prioritizes campaigns based on max bid price, will your lower cost campaigns always be trumped by someone else’s campaigns? Will you be left with the less desirable inventory? |
| 4. How soon can the optimization system start performing? What is the cost of learning? Optimization is all about using historical data to predict the best decisions for the future, which means you first need a certain amount of historical data before it can perform its magic. That initial data is basically the first X% of your budget where performance is not as good as the later part of the campaign. This is the cost of learning. The larger it is, the less ideal your campaigns’ overall average performance becomes. |
| 5. Can your system scale? Can it perform well under different circumstances or does it require the “perfect campaign” in order to shine? |
| A. Scale out: |
| A variety of types of verticals and KPIs Ask for a list of campaign verticals on which the system has successfully optimized. Different verticals behave very differently and optimization systems need to be tuned for different scenarios. This is an area where an optimization engine with a longer history usually performs better because the team would have had the opportunity to deal with more specialized cases. |
| B. Scale up: |
| Larger budgets A system that works wonders giving you 200 quality clicks per day may not be able to deliver 2,000 of the same great clicks a day. After picking out the initial set of best performing audience, it becomes increasingly challenging to find more of them as there is a limit to the number of people interested in your campaign.High volume of campaigns This is more of a software engineering issue than a mathematical issue, but it is just as important for the performance of the optimization system. For every bid request, the system is supposed to choose the best campaign and its bid price. Doing that for 100 active campaigns and doing it for 10,000 campaigns is a different level of engineering challenge. Can the optimization engine handle an increasing number of campaigns without losing its power? Will the campaigns still get exposed to all the available bid requests or will they be split up in order to decrease the computational challenge? |
| C. Scale down: |
| Small campaign budgets Due to the need for learning data as explained earlier, can the optimization system handle small campaigns? How small is too small? |
| 6. Can your system optimize across medium and channels? If you are buying across display, mobile, video, social, etc can the optimization system pool data from all the sources and optimize across everything? By pooling together data from various sources you can gain better performance than if each channel is optimized independently. |
| 7. Is it really programmatic? Does it require a human analyst? How quickly can it utilize new learning data? It’s fine if the technology vendor has human analysts to help further optimize campaigns, but if a human is required to get optimization started on every campaign, then you are likely missing out on opportunities to optimize as soon as possible. Likewise, find out how quickly the system can start utilizing new data, even if it is automated. Automation does not always mean real time. |
Bottom line: it’s not just about the math. The software is an equally important component in how well an optimization system performs.
Other Things To Consider
Importance of inventory quality
You can have the best optimization technology but if the platform does not have access to quality inventory then it’s not going to meet your goals. It is important to keep in mind that DSPs don’t all have access to the same quality inventory, even if they are all plugged in to the most common exchanges and SSPs. A high QPS or requests processed per day number also does not indicate the quality of those bid requests, as a huge percentage of publicly available exchange inventory is very low quality or fraud traffic.
Data ownership
After learning how all campaigns go through an expensive period of collecting learning data, do you want your campaign data to be used to help optimize your competitors’ campaigns? It is important to work with reputable vendors who protect the integrity and exclusivity of your data. Ask for whatever proof and terms of agreements you need to make yourself feel comfortable.
Are financial interests aligned?
Make sure your optimization system is optimizing for your best interest and not the DSP’s. Be careful and think through the implications of how you pay your technology vendor. If you are paying a fixed cost per click, does it matter if the optimization technology is driving down the true cost per click? Paying a percentage fee on the underlying media cost can align the client and the DSP’s interests the best as DSPs are motivated to get you what’s best for you in order to have your continuing business.
