Yesterday I made the brash comment that we should place no credence in Arbitron’s satellite radio ratings numbers (the proportions, maybe, but not the numbers).
A couple folks have asked me to explain why I feel this way, so here goes…
First, I should note that I am talking about diary-based ratings here, not PPM ones. Assuming that all satellite channels are encoded, the measurement there should be state-of-the-art.
Here is a graphic of the Arbitron instruction page which tells diary respondents what to do.
1. Note the big words: “Time,” “Station,” “Place.” Not “Channel.” Not “URL.” Biasing factor number one.
2. Top line: “When you hear a radio, write down…” Do you mean terrestrial radio or satellite radio? I’m guessing my satellite radio doesn’t count, right? And you can’t possibly mean Internet streaming, right? Biasing factor number two.
3. Under “Station” – “Write the call letters, dial setting, or station name.” Not “channel name”. Not “URL.” Biasing factor number three.
4. Under “Station” – “If you listen over the Internet or to a Satellite Radio Service, please include the station name or channel number.” Note that this is the ONLY place on the instruction form that refers to satellite or Internet, and it appears as a throwaway. Easy to miss. Note, too, that you have to provide more information for a satellite or Internet station than for a radio station. It’s more work to complete a response, and when it’s more work to do something fewer folks will do it. Especially if you have more than 100 ways (channels) to say you listen to satellite radio. Biasing factor number four.
5. Look at the picture. Under “Station” show me the box to check “Satellite.” Um, no box. How about the one to check “Internet”? Um, no box. Very strong biasing factor number five. Implying what the right answer is by simple exclusion of other “right” options. In fact, the instruction specifically asks the respondent to “mark AM or FM.” It doesn’t get more biasing than that, folks.
It is obvious that the better job a respondent does in recording satellite and Internet listening, the more shares will move in that direction. Thus it’s in our incentive to stage the measurement tool in such a way as to benefit the radio industry, since it’s we who pay for the service in the first place.
Arbitron can do as good a job at measuring listening as our industry will allow it to do.