06/10

PPM Ratings Horror Stories

Imagine you're around a campfire on a dark and cold night.

Is it time to tell a scary tale?  You bet it is.  

And if it's a tale of ratings horror, what could be scarier?

And if you prefer your horror in audio form, click here.

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Is it too early for Arbitron to go "Boo!"?

Nope, it's never too early for that.

(P.S. In my world 88% plus 22% does not really equal 100%)

* = required field
  • http://www.rj1200.com S

    I work for a South Asian radio station (catering to people from India and the surrounding areas), in Vancouver Canada. This community makes up 10% of the total market, oh sorry, 9.98%, which is why we aren’t being measured. The PPM setup is set for groups 10% and more. The Chinese community is only being measured if they speak English. The list goes on as to how these services don’t care to measure the ethnic communities.
    Another horror story about the PPM (this is all stuff I’ve heard from others) – A station that was sitting at dead last before the PPMs shot up to second place when the PPMs went in to effect! This caused some heads to turn. They came to find out, that one family’s PPMs were all being carried by the mother. The father would forget his and the mother was scared her 2 kids would break them, so she carried them around and seemed to love the station that was dead last, which was great for them.

  • buzz

    So what should the Canadians do about a 22% minority population? Why does race matter and why is race representative in 3 of your 4 examples?
    In Canada, we measure people with ears, and last time I checked most blacks and hispanics and asians had 2 ears.
    Not trying to be an asshat, just curious as to why race matters in your examples. Thanks.

  • http://profile.typepad.com/mramsey1 Mark Ramsey

    The precision of ratings requires a representative sample. A representative sample is one that represents all audience segments in proportion to their population.

  • http://profile.typepad.com/mramsey1 Mark Ramsey

    What a perfect illustration of the problem. Thanks.

  • Frank

    If it were 1989, I’d hire some ladies in hot pants and pay them to walk around the NAB Convention wearing a t-shirt emblazoned with Arbitron’s official opinion of its own product: “PPM ratings are based on audience estimates and are the opinion of Arbitron and should not be relied on for precise accuracy or precise representativeness of a demographic or radio market.”

  • http://www.donkeith.com Don Keith

    Of course, we could tell similar stories about diaries and telephone coincidental. And we all have. Eight diaries from a single household, all filled out in the same handwriting? Saw it myself. “Love Paul Harvey. Never miss him,” written in the “Notes” field on a diary. No indication in any of the diaries from that household of anyone there ever listening to Paul that week. I pulled those five diaries for a client myself.
    Are the panels large enough? Are they truly representative, especially of minorities, narrow age cells, and geo? I doubt it. But it isn’t the technology that is the problem. PPM is far superior for measuring true listening than diaries or a telephone call that interrupts dinner.
    I will agree–again–with your key point, Mark. Ratings are a tool, and should be used for what they are. Their importance will only continue to diminish as we come to rely more and more on response and results for advertisers. Still, customers should push for as much statistical accuracy as they can get. That is, after all, what they are paying for. But they should also realize that if you live by the shares, you die by the shares.
    And by the way, pay-for-performance is coming to radio and TV. Get ready for it!
    Don Keith
    http://www.donkeith.com
    http://www.n4kc.blogspot.com

  • http://profile.typepad.com/mramsey1 Mark Ramsey

    Hi Don, the only thing I’d add to amplify your points is that the panel size at any given time for PPM is much smaller than it was during any aggregated diary period. And since it is panel data, it is more vulnerable to tiny – and long-lasting – changes.
    And PPM doesn’t measure listening. It measures exposure.

  • http://www.rj1200.com S

    But that’s the problem! The belief that everyone is the same. The point of metrics is to give an accurate representation of the listening audience. So if the focus group is all “non-ethnic”, how is that representative of the whole audience? Do you think people who arrived from China yesterday are going to start listening to your program with a limited knowledge of English? Shouldn’t they have programming that is representative of them? Shouldn’t someone know that they should be producing programming for them?
    I was born and raised in Canada, but have to be classified as South Asian. Even then I don’t feel most of the radio stations have me in mind when they are talking to me. We all have ears, but all have very different view points. If you are catering to different cultures/ethnicities shouldn’t you have an idea as to who you are talking to?
    I’m not trying to sound like I’m jumping on you, I just think that the Radio Industry really doesn’t understand their audience as a whole, and feel they don’t have to cater to everyone. They figure that’s for someone else to look after. When others try to look after the other communities, we do it with our hands tied. Agencies don’t want to look at us because we aren’t measured, and those agencies who do decide to give us a little budget, have to justify it to the bigger stations in the market
    PS visible minority populations (Chinese, South Asian, Persian, Vietnamese, Phillippino, etc.) equal almost 50% in Toronto and Vancouver–it’s a huge market.

  • http://www.kink.fm Dennis Constantine

    Great rant Mark!
    I noticed that our STREAM got a 2.0 midday share with 25-54 adults last week in the PPM. Checked the numbers, it had a cume of 11,200. We figured it out and that’s about the value of one meter in our market. So, one listener listened for 10 hours and gave us a 2.0 share. Too bad she didn’t listen to the on-air signal. I could have used a 2.0 boost there!
    Dennis

  • http://www.donkeith.com Don Keith

    And diaries don’t even measure exposure! They measure top-of-mind for the most part. And I’ll take “exposure” over what somebody on Thursday morning thinks they listened to for the past seven days.
    I’m still unclear on why Arb is not using larger panels. Money? Sure. But that is what I meant about customers asking for representative sampling. Truly representative sampling. And radio stations who tout tiny demos in tiny dayparts to sell ads are doomed to the live by/die by cycle.
    Like someone told me about book reviews: if you believe the good ones, you have to believe the bad ones, too.
    Don Keith
    http://www.donkeith.com
    http://www.n4kc.blogspot.com

  • http://profile.typepad.com/mramsey1 Mark Ramsey

    I could legitimately argue that diary is better than meters, actually. That’s because listening you recall is listening that is meaningful to you. And listening that is meaningful to you puts you in the presence of messages that are more likely to make an impression on you.
    I’ll have to write a post on this!
    Arb is not using larger panels because the industry can’t afford the precision it seeks. Both ARB and the industry have lived in denial from the beginning.

  • http://profile.typepad.com/mramsey1 Mark Ramsey

    Hi Dennis!
    So if one listener is worth a 2.0 share – and many stations in a ranker are separated by far less than a single share – what does that say about the veracity of our numbers?
    That riddle is rhetorical, BTW.

  • David

    Hi Mark
    In what way are these specifically PPM horror stories? I get that they’ve happened in PPM markets, but they’re essentially tales of ‘bounce’ and sample composition, aren’t they? And haven’t those things always generated ‘stories’, whether you’re looking at PPM, diaries, or any other research?
    Interestingly, the root of these issues is sample weighting, which is always lower in PPM than in diaries…
    In other words, aren’t these actually issues that the industry has vast experience in dealing with? I understand that fear sells, but still….

  • http://profile.typepad.com/mramsey1 Mark Ramsey

    When your audience tanks because panelists roll out – for good – it’s hard to take solace in the idea that it’s just bounce.
    I think the victims would classify these as horrors. But that’s in the eye of the beholder.

  • http://cfox.com n

    Clearly it obvious. There are too few representing the entire market, no matter what the ethnic breakdown or sex. Adn what about metres pciking up stations being played ion say, Doctors offices and places where they use the easy listening stations as background? And if any retail outlet has local radio on ( which isn’t that many these days where i live ) it’s alwas the local top40/chr station. Do the devices not pick that up as well? Thoughts?

  • http://profile.typepad.com/mramsey1 Mark Ramsey

    Sure they do. Wherever there is an encoded signal the device can pick it up in most cases.
    That’s the beauty of an “exposure” metric.