Creative Testing for Return on Ad Spend (ROAS)

Can Creative Testing Improve Return on Ad Spend (ROAS)?

The analytics community has never had a sufficiently robust set of data that captures creative impact on business results. In ABX testing of 300,000+ ads across the top 26 industries, we found that creative is the largest contributor by far at 50--60% of sales lift.  Without creative data factored in, it’s no wonder that correlating Ad Spend to business outcomes becomes very difficult.Chalkboard with Algebra to illustrate analytics success.

At present, insights directors and marketers use media spend data as inputs into their marketing mix models. However, the question John Wanamaker asks can't be easily answered because Ad Spend doesn’t account for the variability of creative.  Some ads are effective, but at least fifty to sixty percent are not.  Also, an accurate Return on Ad Spend or ROAS can’t be produced from the small portion of one’s advertising that has been through pretesting. 

Thus, for advertisers to see the results of a campaign in the language of management, they need to put ALL their advertising through creative testing,   which can be done today affordably.  The creative scores are then married to Ad Spend and the Market Mix Models snap into place.  See the great case studies below, and then see definitions for Return on Ad Spend (ROAS), ROI, Predictive Analytics and Marketing Mix Models.

Case Studies

Health & Beauty Brand - Ad Spend vs Creative Impact on Revenues 

This study compares the effects of the media portion of Advertising Spend versus creative testing scores on sales revenues for one of the world’s largest cosmetic brands.  ABX provided a robust set of advertising effectiveness measurement scores for client ads across all media types over a period of 30 months, which were integrated into the marketing mix model.  Insights included: 

  • First, it demonstrated that when media dollars were spent regardless the quality of the ad, virtually no correlation was found between media spend and revenues.  This picture changes drastically when more spend is dedicated to the ads with higher creative power.  In this case, had the advertiser spent more on highly effective creative and less on poor creative, the sales impact would have been $26M. National beauty brand saw huge improvement in correlations to outcomes through ABX creative scores.
  • Second,  in an ideal world, media spend on an ad by ad basis would align perfectly with ad sales lift, but this is not the case  in this study nor in the real world.  As we see below, the correlation between the quantitative ad spend-per-spot and sales lift had a very low correlation of r²=.1138.  When we married creative testing scores as a qualitative measure to the media spend, the correlations soar to r²=.743. 

  • If creative isn’t good, media buys are virtually wasted. As mentioned above, had the client used creative scores to allocate media budget, it could have generated an incremental +$26 million in revenue. Creative effectiveness drives sales whereas ad spend is a much weaker driver. (Courtesy Bottom-Line Analytics).


    The first beauty brand chart by Bottom-Line Analytics  shows correlations were very low between sales lift and media spend.

The second beauty brand chart shows media spend weighted by ABX ad creative measurement data and correlates it against sales lift.  Correlation is r-squared = .743.

Soft Drink Brand Case Study - Weighted Media Spend vs Sales

This soft drink study by Bottom-Line Analytics compared ad creative-weighted media spend against sales lift with a high correlation of .71.2%.A similar approach was taken with one of the top carbonated beverages in the world.  The marketing mix modelers weighted the media spend with creative testing data and compared it to sales revenues over a three-year period.  A high correlation of 71.2% showed the weighted creative scores did very well against sales.  (Courtesy Bottom-Line Analytics).


State Lottery Study - Creative Testing and Incremental Sales

Predictive analytics by Bottom-Line Analytics shows how much revenue is generated for every increase in creative scores from ABX.
This case study is about a state lottery.

A large state lottery’s average creative score index over the past year was 91, which was short of the 100 “average” in advertising effectiveness. This case was particularly interesting because the marketing mix modelers showed how much sales revenues could increase as creative testing scores went up. (Courtesy Bottom-Line Analytics)  Amazingly, for every single data point increase in the Florida Lottery average ad effectiveness score, the incremental revenue would be $715,000.  This is predictive analytics as its best.  In the chart below, as the Copy Score weighted average increases (horizontally), the incremental sales revenues also increase (vertically).  The creative team has a great deal of power to positively affect sales results. 


Some Quick Definitions of Success

Before jumping into the case studies, it usually helps to take a quick look at commonly-used measurement terms that management is likely asking for.  The first, ROAS, is heavily searched today, which means it has become popular currency for estimating success.     

  • ROAS - Return On Ad Spend)ROAS is a KPI (Key Performance Indicator) that is used to determine advertising Man jumping between two high cliffs shows the idea that getting good results in ROAS, ROI and Predictive Analytics isn't easy. But advertising effectiveness measurement can make it much more so.effectiveness. It can be calculated for online and offline media campaigns. Calculating the ROAS allows online marketers to see whether their advertising budget generates sufficient revenue.  To calculate ROAS, take the revenue generated by a campaign or ad source and divide it by its media expenses.  A ROAS higher than one means that the campaign is generating more revenue than expenses. 

Of course, everyone knows and uses the term “ROI.”  However, ROI was really built as a measurement of an entire operation or business.  It can be used at a smaller level, such as for a marketing or advertising department.  But to do it right, one must factor in every cost - personnel, real estate, lights, computers. etc. 

  • Marketing Mix Modeling – One form of Predictive Analytics is Marketing Mix Models.  Michael image-37
    J. Wolfe, CEO of Bottom-Line Analytics (and frequent partner of ABX), has crafted more than 200 marketing mix models throughout his career in major brand companies and advertising agencies.  He explains, "In most markets, sales are driven by a range of factors, such as the price, promotions, advertising and distribution amongst others.  Marketing mix models measure the sales impact of multi-channel marketing campaigns, quantifying their effectiveness and return on investment (ROI).  Michael has now tested many models using ABX creative input and found much higher correlations than ever.  Some of the questions that can be answered by these models include:
  • What are the most important sales drivers – TV, Radio, Outdoor, Print or Digital?
  • What has been the ROI of paid search and display banner ads?
  • Which marketing messages are most effective in driving sales? 
  • What would my sales be if I increased price by 10%?
  • How important are holiday promotions?
  • What is the effect of seasonality on sales?
  • What is the best way to allocate my marketing budget?

Whether management is looking at ROAS, ROI or Predictive Analytics through marketing mix modeling, none will tell an accurate story of success without creative being included since it accounts for 60% on average of advertising success.  

For Additional Reading:

Additional Resource Pages

  1.  Advertising Effectiveness Measurement
  2. Integrated Marketing Across All Media Types
  3. Competitive Intelligence in Advertising
  4. Creative Testing for Return on Advertising Spend (ROAS)
  5. Gender Equality in Advertising

Like to Know More? Chat with a Researcher at 347-443-8869!