CIMM/4A’s report says methodology, rather than technology, is root cause of TV attribution outcome differences; calls for more stringent media measurement standards
New York, NY (September 23, 2020) – The inconsistency of key television attribution inputs used by providers, not technology, is the main cause of variance in outcome measurements, according to a report issued today by the Coalition for Innovative Media Measurement (CIMM), in conjunction with the 4A’s Media Measurement Task Force.
The analysis, which looked at the practices of eleven leading TV attribution providers, found that more stringent media measurement standards are required to ensure attribution results that are consistent and comparable from provider to provider, with exposure data, more than occurrence data having the biggest impact on outcome results. The report calls for the addition of standardization, such as commercial IDs similar to Ad-ID, for identifying occurrences and in defining exposure and reach.
“Getting Attribution Right: An Exploration and Best Practices for Television Data Inputs in Attribution Modeling,” was presented as part of The ARF’s AudiencexScience virtual event. The study was conducted by Sequent Partners and Janus Strategy and Insights and was designed to define best practices for better representation of television in attribution models. It is the fourth white paper in a series commissioned by CIMM and 4A’s examining TV attribution and was structured around two main variables: key inputs of ad schedules and ad exposures across attribution providers.
Additional findings include:
- Occurrences and exposure data are highly inconsistent across providers: The accuracy of spot detection and all of the different exposure data elements—GRPs, Reach, Frequency—differ from provider to provider. Identifying and the counting of exposures, or impressions, should be standard starting points. How providers then connect the dots from exposures to outcomes should be their points of difference. If they are all identifying different occurrence levels and evaluating different exposures, then they may as well be evaluating different campaigns.
- Lift outcomes differ significantly: Because of the differences in both occurrence and exposure data, measurement of lift provider to provider yielded different results, both in terms of magnitude and direction of lift.
- Provider exposure data impacts lift results more than occurrence data: The study found that while there are differences between providers for both occurrence data and exposure data, the differences in exposure data are a much larger contributor to differences in lift measurement.
- Methodology, rather than underlying technology, drives results: For both occurrence data and exposure data, the underlying data elements—monitoring of network signals to create an accurate ad occurrence file, ACR and/or set-top box data that measures what households are viewing—are similar. The methodology of converting that data into final ad occurrence files and exposure data, including weighting, editing and other data processing rules, is believed to be the cause of the differences between providers.
“Earlier this year, we set out to uncover what drives the difference in attribution results across industry attribution providers, each offering varying approaches, including relying on different modeling techniques and data sources,” said Alice K. Sylvester, partner, Sequent Partners. “While the findings of our study do not necessarily tell us how to solve for the attribution inconsistencies our industry currently faces, they do clearly indicate the need for the standardization of naming, definitions, and categorization and more careful quality assurance procedures.”
“In order to bring more transparency to attribution and move the industry toward new forms of measurement, we need to understand more about the sources of TV data, how accurate they are and what needs to be worked on,” said Jane Clarke, managing director and CEO, CIMM. “The industry will not be ready for true ad measurement until we have a better way to verify ad logs and can get fully accurate, representative samples.”
Howard Shimmel, president, Janus Strategy and Insights, added, “The importance of attribution to media companies, advertisers and their agencies grow each day. The industry is blessed by having so many high quality providers. Our hope is that this report provides a roadmap for providers to improve their offerings and at the same time reduce the dramatic differences between providers.”
The report provides three recommendations for TV attribution moving forward:
- For the industry: Establish standards to ensure that data is organized comparably across providers. Ad-ID and the standardized commercial IDs it affords could definitely assist modelers to align the data, eliminate coding errors and reduce the time associated with data reconciliation.
- For users: Before proceeding with an attribution study, make sure your occurrence data are validated. This might require using a specific validation study to ensure your data inputs are accurate. Otherwise, while directional guidance for tactical optimization may be trustworthy, ROAS or ROI estimates may be risky.
- For providers: Test your QA procedures to ensure accuracy and be prepared to adopt industry standards as they are developed.
“Getting Attribution Right: An Exploration and Best Practices for Television Data Inputs in Attribution Modeling” is available for download at www.cimm-us.org.
About CIMM
The Coalition for Innovative Media Measurement (CIMM) is comprised of leading TV and video content providers, media buying agencies, large advertisers, pay TV distributors, research and media technology vendors and consultants that aim to promote innovation and foster efficiencies in audience measurement for television and cross-platform video. In 2018, CIMM was acquired by the ARF (Advertising Research Foundation), with the goal of expanding the categories of members as well as its initiatives. CIMM initiatives to date have included exploration and identification of new methodologies in cross-platform audience measurement, with verification of these approaches through pilot tests and studies conducted with independent measurement companies. CIMM’s focus has been on two key areas: the current and future potential of television measurement through the use of return-path data, and new methods for cross-platform media measurement. For more information, visit www.cimm-us.org
About Sequent Partners
Sequent Partners is a consulting firm specializing in brand and media metrics. Founded in 2003, they help clients link marketing activities to financial outcomes. They focus on accountability, ROI and innovative metrics that matter. Sequent Partners recently conducted studies of current practices in marketing mix models, long-term effects of advertising, ROI measurement for Hispanic media and word of mouth marketing, among other topics, in addition to their proprietary consulting for advertisers and the media. www.sequentpartners.com
About Janus Strategy & Insights
Janus Strategy & Insights provides ongoing research consulting for both national and global companies, bridging the TV and streaming digital divide. A leader in strategy and positioning, they focus on emerging developments in attribution, advertising, and data to inform how audiences engage and act.
About The 4A’s
The 4A’s helps empower its members to deliver insightful creativity that drives commerce and influences culture, all while moving the industry forward. The organization is dedicated to, and vested in, its members’ success, just as it is dedicated to helping brands create, distribute, and measure effective and insightful advertising and marketing. The 4A’s provides community, leadership, advocacy, guidance and best-in-class training that enable agencies to innovate, evolve and grow. In 1917, the 4A’s was established to promote, advance and defend the interests of its member agencies, their employees and the industry at large. After 100 years, the 4A’s continues to support the evolving needs of its community. Today the organization serves 600+ member agencies across 1,200 offices, which help direct more than 85% of total U.S. advertising spend. 4A’s Benefits division insures more than 160,000 employees and its D.C. office advocates for policies that best support a thriving advertising industry. The 4A’s Foundation fuels a robust diversity pipeline of talent for its members and the marketing and media industry, fostering the next generation of leaders.
