© 2023 Advertising Benchmark Index. All rights reserved. Website by IMG Digital Marketing Agency
Creative effectiveness measurement is essential and imperfect. Every measurement firm claims it has the most accurate data in the industry. However, most keep their methodologies tucked away in a black box, rarely displaying them in the public square.
What exactly is in those black boxes? Are the methodologies sound? Do they rely on legacy systems and diverse media suppliers? Is the data accurate for CMO decision-making?
A May, 2022 CMO Council poll reported that 80% of marketing leaders consider data, analytics and insights “very important” to their business, but only 30% were “very confident” in their existing sources and systems. How can ad testing data improve so all CMO’s find them reliable?
Three Common Practices that Compromise Results
1. Inconsistent survey methodologies from different ad testing sources.
How can an integrated, holistic view of marketing success be visible if you’ve used a variety of measurement techniques and sources from survey to survey? Or across different media types? Or comparing instantaneous DIY surveys to full-service surveys? To reduce the bias inherent in these practices, look for solutions that offer the same measurement process in:
Most marketers don’t fully appreciate how different the results can be from using “mix and match” solutions and methodologies. Every one of these variances is critical to producing valid and reliable results. For example, mixing results from a DIY “instant” survey with established professional surveys introduces tremendous bias since recruiting identical panels surveyed at given times with different questionnaires and completions will produce very different results.
2. Using pre-test norms for pre-tests rather than more accurate in-market norms.
One Achilles’ Heel in creative testing, which is not generally known, is that most provider’s norms are based on their clients’ ads and not all ads in categories, brands or competitive sets. Thus, they do not represent a true universe.
Another Achilles’ Heel is measuring pre-tests against a provider’s pre-test norms instead of in-market norms. Since pre-test norms tend to skew lower than in-market because they include poor ads that never made it to market, advertisers may think their pre-tests are better than they really are. Furthermore, pre-tests are usually run against specific target audiences, making comparisons even more difficult against different audiences.
3. Evaluating campaigns with ads in different media types without accounting for differences in how each media type was tested.
Many marketers rely on ad testing firms that focus on singular media types. For example, one resource might measure TV and Online Video, but nothing else. Another firm might measure print and another radio. Digital creative is different again, primarily measured quantitatively in clicks, impressions, shares, going to the website and requests for information.
Each of these sources may provide excellent data for its media type, but disparate results are tough to rationalize into a single picture or model. They can be compared quantitatively through ad spend, CPM, or CPI. But those metrics are not weighted by creative scores, so every ad is considered to be the same. Two solutions to this problem have recently come to light:
In addition, the ABX Continuous Creative Feedback Loop™, powered by the ABX Global Integrated Single Solution™, provides real-time insights through the ABX Global Syndicated Platform. This provides the depth and granularity needed for making informed, time-sensitive business decisions. Since the ABX solution measures are consistent through every survey, media type, media channel, demographic, psychographic, geography, etc., marketers don’t have to rely on a black box or after-the-fact harmonization model at all. At any point, they can see a completely integrated picture and act.
# # #
Written by Angela Jeffrey, Vice President Brand Management.
Jeff has more than 25 years of market research experience including project management, client service, operations, production and sales.
Before joining the ABX Senior Management team, Jeff was a Senior Account Manager at Ipsos, working with Procter & Gamble and other client assignments. Prior to that, he was Senior Project Manager for Opinion One, where he was responsible for guiding the company’s online, mall and other initiatives for eight years.
Jeff began his market research career with Cooper Research while participating in the Professional Practice Program at the University of Cincinnati, Ohio. Upon graduating with his BS in Marketing and Management, Jeff led a team of data analysts as Group Manager at Matrixx Marketing Research (now Concentrix).
Jeff’s broad in-depth marketing research experiences enable him to bring a unique perspective to directing operations that creates effective and efficient environments to support a wide range of client activities. Having come up in the industry alongside the advent and progression of online interviewing, he has a unique perspective on the benefits and challenges of conducting quality research resulting in meaningful results.
Jeff considers himself an amateur music historian and sports enthusiast, as well as a family man, who takes advantage of the many outdoor activities in his community including biking, hiking, kayaking and swimming.
Dr. Peter Moomaw
Peter is the primary developer of operations technology and data analysis. As a chief visionary on the Research and Development team, Peter works to generate innovative ideas for the continued development of processes and protocols.
Peter brings 17+ years’ experience in software development in the market research industry. He was previously at Opinion One, where he was integral in the development of research software systems covering a number of areas.
Peter has a BA in Mathematics, a BS in Physics, and a PhD in Physics, all from the University Cincinnati.
Michael is one Founders of Ocucom, of which ABX is a division, and has over 12 years’ of software engineering experience in the market research industry. As a Lead Software Engineer with Ocucom, he brings a unique blend of a technological background along with hands-on market research experience. Michael specializes in providing custom software solutions utilizing Ocucom’s dynamic software platform to address unique client needs in an efficient and cost-effective manner.
Prior to starting with Ocucom, Michael was a senior software engineer at Opinion One in Cincinnati, where he was one of the principal architects of the OASIS 2 Survey Software Package. He also played a key role in developing a licensable solution to bring to the market place as well as handling the logistics of on-site training and support of end-client users.
Michael obtained his B.S. in Computer Science from the University of Cincinnati.
Greg Tunstill, a founder of ABX, is the Chief Information Officer responsible for ABX’s technical infrastructure and maintenance. He also leads the design and development of ABX’s communications tools, a critical component of its technology services.
Prior to joining ABX, Greg was Director of Network Operations of Opinion One since its inception in 1992. He designed and developed the communications systems that were the core of its data collection networks and was responsible for continuing development and implementation of information technologies. Prior to his position at Opinion One, he designed inventory control and expert systems for BellSouth field offices and worked as a consultant and independent contractor. In 1991 he joined Americom Research as a systems engineer where he advanced to Director of Network Operations and met members of the team that formed Opinion One.
Greg is a graduate of Middle Tennessee State University where he obtained a Bachelor of Science in Computer Science.
Julian brings 20+ years of software engineering and market research experience to ABX. As one of the Founders and in his position as Chief Technology Officer, Julian is the chief visionary responsible for strategic planning and development.
His primary day to day roles include management of software development, oversight of survey production, and lead architect for custom software development projects. As the principal technology architect, Julian adapts emerging technologies for use in market research. He is responsible for maintaining the “continuous development” philosophy.
Before becoming one of the Founders, Julian was Vice President, Software Engineering at Opinion One. In this capacity, he was responsible for the design and development of Opinion One’s core technology and licensing products.
Prior to Opinion One, Julian worked for Americom Research, starting as a Software Engineer and progressing to Director, Software Development. While at Americom Research, Julian pioneered the design and creation of one of the market research industry’s first fully interactive computer-aided interviewing suites.
Julian received his Bachelor of Science in Computer Science from Middle Tennessee State University.
Gary has an extensive background in product development, strategic planning, and sales/marketing. He is a frequent speaker to the advertising, marketing, and public relations communities.
Gary is a pioneer in efforts to measure and analyze the combined impact of news coverage, social media, and advertising communications on key business metrics, such as sales or product preferences, with ground-breaking work linking communications activity to business outcomes. He was also one of the first to apply artificial intelligence to better understand how communications impacts consumer behavior.
As part of these pioneering integrated measurements, the need to benchmark every ad and build a comprehensive database of advertising performance metrics for evaluating all advertising creative became clear. Gary has been part of the ABX management group that developed the methodology, technology, and criteria for measuring advertising performance globally, and at scale, in an affordable manner.
Now Gary and all ABX partners work with communications leaders across a diverse set of industries to expand that understanding by incorporating the first-ever Gender Equality Index™, creative measurement solutions for ESG, Diversity & Inclusion, Ethnicities (in and out-of-country), Product Placement/Integration, Brand Fit™, Gaming, Audio and more.
Gary holds a B.S., in Mechanical Engineering from Lehigh University and a Masters in Management Science from Stevens Institute of Technology.
J. J. Klein
JJ Klein has devoted his entire career to developing more valid and reliable information through syndicated and custom longitudinal research programs for Fortune 500 companies. This extensive knowledge base enabled him to create and develop ABX, which now offers the most accurate normative database for advertising effectiveness measurement in the marketplace.
JJ brings more than 35 years of marketing, research, consulting, and technology experience in designing and developing methodologies and programs for long-term growth. He has exceptional strength in domestic and international new business development, strategic planning, and identifying and tracking consumer, retail, and industry trends.
Leadership in Top Global Research Firms
Early in his career, after attending City University in New York as a math and science major, JJ joined Simmons Market Research Bureau as a Project Director. Over time, he rose to Associate Executive Vice President. Among his accomplishments, JJ created the first online interactive data analysis and delivery system for agencies, media, institutions, client, and service companies.
JJ and Bill Simmons then founded a new venture …Three Sigma Research Center. It was here that JJ, Chief Operating Officer, created the first major syndicated rating system for Newspapers in the U.S., which later became the Scarborough Report. Three Sigma Research was eventually sold back to Simmons; Simmons sold it to Scarborough; and then Scarborough was acquired by Nielsen.
Following the sale of Three Sigma, JJ joined Audits & Surveys Worldwide as a principal leading the sales tracking and market share audits for a broad range of retail and service establishments both domestically and internationally. Following the acquisition of Audits & Surveys by GfK, JJ joined Roper Starch as Executive Director, reporting directly to the Chairman.
Creation and Development of Ocucom™ and ABX™
In 2004, as he continued his industry visionary travels, all of JJ’s experience came together as he tackled new solutions for market research in the Internet age. He and four partners founded Ocucom™ and invented a research-proven, respondent-oriented Ocular™ approach to internet sampling and multi-modal interviewing methodology. For the first time, respondents could answer text-based questions within a visual communication world where the computer screen was the interviewer/communicator. Inventions continued such as Ocucom’s PreCast™, the first long-form video viewing solution in the industry.
Also, during this time, JJ’s early leadership in internet research methodology caught the eye of top-tier global research competitors. He ended up designing methodologies for their online survey capabilities.
Then, in early 2012, he formed ABX™ as an outgrowth of Ocucom. JJ had conceived the idea for ABX some 15 years prior but had to wait for internet maturation and online panel/sampling companies to evolve. ABX partners focused on JJ’s ABX concept and methodology by building an infrastructure and a 5-year formative database before opening to the market in 2018. The new company offered the first high-quality Ocular research system for all ads in all media types with detailed, never-before analytics; the first syndicated, all-media type ad effectiveness platform; results in 24-to-48 hours on dashboard, and significantly lower costs. At ABX, as at Ocucom, inventions have continued such as the expansion of the ABX syndicated program into the first-ever global solution, and the first Gender Equality Index™. Solutions have expanded to include creative measurement for ESG, Diversity & Inclusion, Ethnicities (in and out of country), Product Placement/Integration, Brand Fit, Gaming, Audio, and more.
A Career Devoted to Validity and Reliability in Market Research
JJ’s background brings together an extremely unique blend of extensive experience in designing and developing global and regional longitudinal research programs. His accomplishments include the design and initiation of programs and methods that produce more valid and reliable information. They also include contributions to the market research profession through association membership, speaking, writing, and presenting.
Like Nielsen for TV, Arbitron for Radio, and MRI for Magazine, JJ’s original design for the newspaper program continues as the Scarborough Report ratings standard for Newspaper. Today, JJ leads ABX as a syndicated and custom research service for ad effectiveness and gender equality, which have quickly become the gold standards of their own.
Angela Jeffrey, APR
Angela is an internationally known speaker and national award-winning veteran of public relations, advertising and marketing. She has been a teacher and leader in the use of metrics to measure communications impact for over 20 years, especially at the intersection of PR and Advertising.
Angela is primarily responsible for all ABX marketing efforts, and for projects that involve PR or social media measurement. She is the primary driver of our collaboration with the prestigious Institute for Public Relations as we develop methods to measure the impact of PR and social media on advertising effectiveness.
Prior to joining ABX, Angela was Strategy Director at Salience Insight, (now CARMA), a global provider of PR effectiveness measurement. Previously, she was a Vice President at Surveillance Data, Inc., an international analytics powerhouse that specializes in predictive modeling. Angela continued as a key leader upon the sale of her division to VMS and helped develop Vantage, with Gary Getto, the first platform to measure the combined impact of PR and Advertising.
Angela graduated summa cum laude from Southern Methodist University with a BFA in Journalism and BBA in Marketing.
Diane has 20+ years of experience helping Marketing, Advertising, and PR decision-makers at global brands, media companies, and agencies increase their ROI by connecting and authentically engaging their target customers through Sales, Business Development. MarTech, AdTech, Big Data, Market Research, SaaS, eCommerce, and Integrated Marketing.
She is passionate about her family, entertainment, health & wellness, empowering women, and environmental sustainability.