So last week I penned a strong piece suggesting that the radio industry drop PPM.
The argument was stimulated by the discovery that Nielsen households can contain up to 16 meters. For me, this was the straw that broke the proverbial camel’s back.
The piece sparked a firestorm and almost a thousand Facebook shares, not because my points were so salient (although I hope they were) but because there does not seem to exist a single PPM subscriber who considers themselves satisfied customers. Nielsen is evidently in the business of optimizing unhappiness.
Nice work if you can get it.
You can read my original post for yourself. Nielsen responded in typical corporate fashion with this “official” response:
We strongly believe in and stand behind our portable people meter (PPM) technology. Our 70,000+ PPM panel is representative of the marketplace. While the tenure of individual Nielsen PPM panelists vary by demographics, on average, a person is part of the panel for 11 months. While less than half a percent of our in-tab persons come from households with 10 or more residents, our average household panel size for people 6 years and older is 2.5 persons per home for all PPM markets. This is in line with the household universe estimate of 2.4 persons per home. Nielsen’s PPM technology will continue to deliver currency grade metrics that the radio and advertising industries depend on to transact with confidence.
This is called “how to lie with statistics.” So let’s do some math.
By Nielsen’s own fuzzy estimate, about 350 panelists come from households featuring 10 or more meters. That would average to 35 households. Let’s say there are 50 PPM markets, and let’s assume that the 35 households are equally distributed by market (they’re not, but go with me here). That means the average PPM market contains between 1 and 2 10+ PPM households.
Does that make you feel better? How many 10+ PPM households is “not too many” in your market?
The problem is that Nielsen is trying to swamp the issue with big numbers. But your station’s ratings don’t vary because of big numbers, they vary because of small ones. They vary not because your ratings depend on 70,000+ PPM panelists, but because they hinge precariously on one or two.
To say nothing of 10 in one household.
I got a note recently from a client who broadcasts Religious spoken word programming on AM – a format targeting primarily White/Other listeners over the age of 35: “We found out that we are reaching 90% African American Women and 78% of our audience is 25-34! Of course, we have only 5 or 6 meters for the metro. Crazy, right?”
Right.
Thank you, PPM! All 70,000 of you!
At least these 5 or 6 meters were not all in the same household.
Or were they?
How will you ever know?
Is this what Nielsen means by “currency grade metrics that the radio and advertising industries depend on to transact with confidence”?
When we talk about meters we tend to think of them as disconnected from each other: Independent listening events. But this is hardly true. Households affect listening within that household to a significant degree, at least inasmuch as multiple panelists share the same car. That means there’s a huge group dynamic that affects station ratings caused by multi-panel households, and the bigger the household the stronger this dynamic. 10 meters in a single household is not the same as 10 meters in 10 households. Period. For Nielsen to pretend otherwise is disingenuous.
It’s one thing to represent the market with 70,000 panelists when that market is yours. It’s another thing to represent it when the difference between ratings success and failure is a handful of listening occasions from fewer panelists than you have fingers on one hand.
Meanwhile, (as I predicted) the average household size of a PPM family IS larger than the average US household size. 2.5 is bigger than 2.4 by 4%. And why be off by 4% when you have 70,000 panelists? That’s 2,800 panelists, Nielsen. You could err in the other direction for more sample stability and measurement accuracy, you know.
For radio stations, the sample you are concerned with is the one that determines your ratings and the volatility of those ratings. That sample is abjectly too small. And the danger of that too-small sample is only aggravated when between one and two households in your market are packed with 10 or more meters, thus placing that household on the Nielsen payroll.
Look, there are many wonderful, talented, committed, and hard-working people at Nielsen trying to do the difficult work of measurement. I respect and admire each and every one.
But when you stand behind the logo and spin the numbers, somebody has to point out that this Emperor lacks clothes.