Why the Arbitron diary isn’t fair to satellite or Internet radio

Yesterday I made the brash comment that we should place no credence in Arbitron’s satellite radio ratings numbers (the proportions, maybe, but not the numbers).

A couple folks have asked me to explain why I feel this way, so here goes…

First, I should note that I am talking about diary-based ratings here, not PPM ones. Assuming that all satellite channels are encoded, the measurement there should be state-of-the-art.

Here is a graphic of the Arbitron instruction page which tells diary respondents what to do.


1. Note the big words: “Time,” “Station,” “Place.” Not “Channel.” Not “URL.” Biasing factor number one.

2. Top line: “When you hear a radio, write down…” Do you mean terrestrial radio or satellite radio? I’m guessing my satellite radio doesn’t count, right? And you can’t possibly mean Internet streaming, right? Biasing factor number two.

3. Under “Station” – “Write the call letters, dial setting, or station name.” Not “channel name”. Not “URL.” Biasing factor number three.

4. Under “Station” – “If you listen over the Internet or to a Satellite Radio Service, please include the station name or channel number.” Note that this is the ONLY place on the instruction form that refers to satellite or Internet, and it appears as a throwaway. Easy to miss. Note, too, that you have to provide more information for a satellite or Internet station than for a radio station. It’s more work to complete a response, and when it’s more work to do something fewer folks will do it. Especially if you have more than 100 ways (channels) to say you listen to satellite radio. Biasing factor number four.

5. Look at the picture. Under “Station” show me the box to check “Satellite.” Um, no box. How about the one to check “Internet”? Um, no box. Very strong biasing factor number five. Implying what the right answer is by simple exclusion of other “right” options. In fact, the instruction specifically asks the respondent to “mark AM or FM.” It doesn’t get more biasing than that, folks.

It is obvious that the better job a respondent does in recording satellite and Internet listening, the more shares will move in that direction. Thus it’s in our incentive to stage the measurement tool in such a way as to benefit the radio industry, since it’s we who pay for the service in the first place.

Arbitron can do as good a job at measuring listening as our industry will allow it to do.

* = required field
  • tim wallick

    I knew prior to the information being released..
    I kept going why release whats not been paid for…
    Mark you keep pounding away at the issues and bring honest insight.
    thanks tim

  • Jeff Schmidt

    does sat radio spend ANY time “gaming” the rating system like we do?

  • http://www.donkeith.com Don Keith

    Remember, Arbitron’s real job is to sell radio stations, rep firms, and ad agencies/advertisers listening estimates for stations and networks that sell advertising. I would not expect Arbitron to go too far in measuring satellite radio or streaming for two reasons:
    1) They measure the stations and networks to whom they sell their data and do the best they can for that constituency. I’m not clear if any satellite or Internet-only stream pays for that data yet. If they did, and the revenue was sizeable, I believe you would see a different looking diary.
    2) Arbitron’s current customers are not too keen on seeing numbers for competitive media as a part of “the book.” How many stations would be happy about non-commercial or NPR stations showing up?
    Does Nielsen measure DirecTV or Dish Network? I rest my case.

  • http://profile.typekey.com/mramsey1/ Mark Ramsey

    I don’t blame ARB – except for the fact that they published those numbers knowing their limitations. And they in no way advised about those in their report.

  • http://classicrockfm.blogspot.com Dan Kelley

    Unless Arbitron changes the diary to make it use-friendly to record internet and satellite listening, how will the company’s terrestrial customer base know the full impact of these other radio distribution paths?
    If I wasn’t able to access the non-comm numbers in Maximizer, how would I ever know the impact those stations are have in my market – who I’m sharing audience with and so forth.
    Likewise with satellite and internet.
    Resisting the idea of Arbitron gathering and providing this information (even in the “protected world” of Maximizer ) is foolish. You can’t succeed by hiding your head in the sand.
    How can we not succeed without the right data?

  • http://www.donkeith.com Don Keith

    Don, you hit the nail on the head. As long as radio lives in a SHARE world instead of an “ears” world, it will never reach its true potential. Most advertisers don’t care what SHARE you have of people who are listening to radio. We care how many people in the target are listening. 25-54 is goofy! Few advertisers target a demo that broad. Example: I want ethnic females, 22-28, who have no or some college. I don’t care if you have a 20 share 25-54 but nobody in that target, I won’t buy you.
    And SHARES are relatively stable. RATING POINTS are dropping like a cement block balloon.
    There has to come a day when a station’s over-the-air, streaming, podcast and time-shifted audiences all combine to let an advertiser know how many people hears his message. Then radio will have something to brag about…and to sell aggressively.

  • http://www.orbitcast.com/archives/why-arbitrons-satellite-radio-ratings-are-wrong.html Orbitcast

    Why Arbitron’s satellite radio ratings are wrong

    “When the only tool you have is a hammer, then every problem begins to look like a nail.”- Abraham MaslowIn the world of statistics and research, methodology is paramount. That’s pretty basic. If you’re going to draw a conclusion from the polling of a …

Dive Into The Blog