Creative Testing for ROAS, ROI and Predictive Analytics

Can Creative Testing Improve ROAS, ROI or Predictive Analytics?

The analytics community has never had a sufficiently robust set of data that captures creative impact. In ABX testing of 150,000 ads across 18 industries, creative is the largest contributor by far at 60-80% of sales lift.  Without creative data factored in, it’s no wonder that it is hard to correlate advertising to business outcomes.Chalkboard with Algebra to illustrate analytics success.

At present, insights directors and marketers use media spend data as inputs into their marketing mix models.  As explained in the first resources page, “How do I know which half of my advertising isn't working,” spend data doesn’t perform well because not all ads are equal creatively!  Some ads are effective, but at least sixty percent are not.  Also, accurate ROAS, ROI or Predictive Analytics data can’t be produced from the small portion of one’s advertising that has been through pretesting. 

Thus, for advertisers to see the results of a campaign in the language of management, they need to put ALL their advertising through creative testing.  The creative scores are then married to ad spend data and the rest is history.  Fortunately, new technologies for measuring all ads is making this strategy affordable.

Some Quick Definitions of Success

Before jumping into the case studies, it usually helps to take a quick look at commonly-used measurement terms that management is likely asking for.  The first, ROAS, is heavily searched today, which means it has become popular currency for estimating success.     

  • ROAS - Return On Advertisement Spending)Searchify defines ROAS as a KPI (Key Performance Indicator) that is used to determine media Man jumping between two high cliffs shows the idea that getting good results in ROAS, ROI and Predictive Analytics isn't easy. But advertising effectiveness measurement can make it much more so.effectiveness. It can be calculated for online and offline media campaigns ... calculating the ROAS allows online marketers to see whether their advertising budget generates sufficient revenue.  To calculate ROAS, take the revenue generated by a campaign or ad source and divide it by its media expenses.  A ROAS higher than one means that the campaign is generating more revenue than expenses. 

Of course, everyone knows and uses the term “ROI.”  However, ROI was really built as a measurement of an entire operation or business.  It can be used at a smaller level, such as for a marketing or advertising department.  But to do it right, one must factor in every cost - personnel, real estate, lights, computers. etc. 

  • ROI – Return on Investment (ROI) measures the gain or loss generated on an investment relative to the amount of money invested. ROI is usually expressed as a percentage and is typically used for personal financial decisions, to compare a company's profitability or to compare the efficiency of different investments.”  The return on investment formula is: ROI = (Net Profit / Cost of Investment) x 100.  Molly Galetto of Thought Leadership NG Data provides the definition below: Gypsy w/crystal ball looks to predict the future. 
  • Predictive Analytics - Predictive Analytics involves extracting data from existing data sets with the goal of identifying trends and patterns. These trends and patterns are then used to predict future outcomes and trends. While it’s not an absolute science, predictive analytics does provide companies with the ability to reliably forecast future trends and behaviors.”  Finally, there are the wizards who figure out the success of campaigns whether they be company-wide, just paid media, only earned media, or some combination of the above.  There was a day when hiring these super minds was cost-prohibitive for all but the largest companies.  Today, there are many small modeling companies and freelancers who make this kind of accountability possible. 
  • Marketing Mix Modeling – Decision Analytics, another major research and analytics firm, writes the following: “Typically, marketing mix modeling involves the use of multiple regression techniques to help predict the optimal mix of marketing variables. Regression is based on a number of inputs (or independent variables) and how these relate to an outcome (or dependent variable) such as sales or profits. Once the model is built and validated, the input variables (advertising, promotion, etc.), can be manipulated to determine the net effect on a company’s sales or profits.”

Whether management is looking at ROAS, ROI or Predictive Analytics through marketing mix modeling, none will tell an accurate story of marketing success without creative being factored in since it accounts for at least 60% of advertising success on average.  

Case Studies


Health & Beauty Brand - Media Spend vs Creative Impact on Revenues 

This study compares the effects of media buys versus creative testing scores on sales revenues for one of the world’s largest cosmetic brands.  ABX provided a robust set of advertising effectiveness measurement scores for client ads across all media types over a period of 30 months, which were integrated into the marketing mix model.  Insights included: 

  • First, it demonstrated that when media dollars were spent regardless the quality of the ad, virtually no correlation was found between media spend and revenues.  This picture changes drastically when more spend is dedicated to the ads with higher creative power.  In this case, had the advertiser spent more on highly effective creative and less on poor creative, the sales impact would have been $26M. National beauty brand saw huge improvement in correlations to outcomes through ABX creative scores.
  • Second,  in an ideal world, media spend on an ad by ad basis would align perfectly with ad sales lift, but this is not the case  in this study nor in the real world.  As we see below, the correlation between the quantitative ad spend-per-spot and sales lift had a very low correlation of r²=.1138.  When we married creative testing scores as a qualitative measure to the media spend, the correlations soar to r²=.743. 

  • If creative isn’t good, media buys are virtually wasted. As mentioned above, had the client used creative scores to allocate media budget, it could have generated an incremental +$26 million in revenue. Creative effectiveness drives sales whereas ad spend is a much weaker driver. (Courtesy Bottom-Line Analytics).




The first beauty brand chart by Bottom-Line Analytics  shows correlations were very low between sales lift and media spend.


The second beauty brand chart shows media spend weighted by ABX ad creative measurement data and correlates it against sales lift.  Correlation is r-squared = .743.


Soft Drink Brand Case Study - Weighted Media Spend vs Sales

This soft drink study by Bottom-Line Analytics compared ad creative-weighted media spend against sales lift with a high correlation of .71.2%.This case study is about one of the largest soft drinks in the world.

A similar approach was taken with one of the top carbonated beverages in the world.  The marketing mix modelers weighted the media spend with creative testing data and compared it to sales revenues over a three-year period.  A high correlation of 71.2% showed the weighted creative scores did very well against sales.  (Courtesy Bottom-Line Analytics).

State Lottery Study - Creative Testing and Incremental Sales
Predictive analytics by Bottom-Line Analytics shows how much revenue is generated for every increase in creative scores from ABX.
This case study is about a state lottery.

A large state lottery’s average creative score index over the past year was 91, which was short of the 100 “average” in advertising effectiveness. This case was particularly interesting because the marketing mix modelers showed how much sales revenues could increase as creative testing scores went up. (Courtesy Bottom-Line Analytics)  Amazingly, for every single data point increase in the Florida Lottery average ad effectiveness score, the incremental revenue would be $715,000.  This is predictive analytics as its best.  In the chart below, as the Copy Score weighted average increases (horizontally), the incremental sales revenues also increase (vertically).  The creative team has a great deal of power to positively affect sales results. (Courtesy Bottom-Line Analytics) 

For Additional Reading:

Additional Resource Pages

  1.  Advertising Effectiveness Measurement
  2. Integrated Marketing Across All Media Types
  3. Competitive Intelligence in Advertising
  4. Creative Testing for ROAS, ROI and Predictive Analytics
  5. Gender Equality in Advertising












Like to Know More? Chat with a Researcher at 347-443-8869!