Internet Ratings For The Big Boys (pt 1)
The Alexa web site ranking service is very widely recognized as an authoritative resource for determining web site popularity. But Alexa ratings are very biased towards websites catering to webmasters and Internet business types. The reason is that Alexa determines the ratings based on a biased sample of Internet users who happen to be webmasters and Internet business types.
So if we wanted an unbiased rating system for web site traffic, we’d need an unbiased, statistically random sampling of web surfers. But how on earth do you do that?
If you do as Alexa has done and recruit the members of your sample by offering a free browser tool bar that promises to show you the ratings for every site you visit, then your sample is biased towards people who give a darn about that stuff (geeks, like you and me). Therefore, the ratings you produce from that sample will only have meaning when you are talking about that narrow audience.
And if you entice new members to join your sample with offers of free ___<fill in the blank>___, your sample is biased towards freebie seekers and the numbers you produce would only be relevant when considering a narrow, freebie seeking audience.
And God forbid you offer to pay members of your sample. Then you’ll get a sample of money grubbing mercenaries and your results would, again, not be applicable to the general population.
So how do you avoid tainting your sample and building a bias into your ratings research from the start?
I worked at Nielsen Media Research for about 7 years, so I can tell you a bit about how ratings are done in the television industry.
You’ve probably heard about “Nielsen Families,” right? These are households across the U.S. who’ve been sent a log book in the mail and asked to record their television viewing in the log book for a week. They are supposed to mail it back to Nielsen where all the log book data from all the Nielsen families who respond (not all of them do… shocking, isn’t it?) are entered into a database for number crunching and reporting.
This happens a couple of times a year during what are called “sweeps” weeks. And it’s during these weeks that the networks and the cable channels pull out all the stops and present us with some of the most expensive, outlandish, breath taking, shocking, etc, etc, programming they can possible come up with.
All to leave a lasting impression in the skulls of Nielsen families. Because the networks and the cable channels know something darned well. Nobody… NOBODY… has that log book with them the entire week while they’re watching TV. They get to the end of the week and fill it out all at once. And if all they can remember about Monday night is that “Zombie Hookers From Mars” was supposed to be on at that time because they saw a commercial for it, that’s what they’re putting in the book, whether they actually watched that cinematic masterpiece or not!
Nielsen knows this just as well as the networks and cable channels, and that’s why they came up with the People Meter system. The People Meter is a box that hooks up to a TV (and VCR and DVD player and just about any other media device you can think of) and keeps track of what people are REALLY watching at 15 second intervals.
So, instead of relying on what people SAY they watched, Nielsen “knows” what they actually watched. The data gleaned from the Nielsen log books are merged with the data from the People Meter sample for the same week to produce a set of combined reports for the networks and cable channels. These reports are used by the networks and cable channels to set their advertising rates. (Needless to say, a few tenths of a rating point either way represents a lot of money of ad revenue!)
That’s enough for today. Tune in tomorrow to find out how Nielsen builds an (almost) unbiased sample and what all of this has to do with your Internet business.
- Daniel Joseph Moran
PS: I hope you weren’t offended by the ‘Zombie Hookers’ thing. But, if you were, you can always cuss me out in the comments, below.
