On The Trail

There is an interesting article on Slate today regarding analytics and the difficulties of knowing exactly how many people visit a website. He compares numbers from comScore, Nielsen/NetRatings and Slate’s own IBM SurfAid analytics and finds that they are quite different. These differences ALWAYS crop up whenever you measure a site using different systems, and the panel based comScore and Nielsen numbers are rarely comparable with the onsite analytics numbers. The author does a good job of explaining why the differences occur (and why the onsite analytics is much more reliable), but it highlights one of the big frustrations with Web Analytics in general: it seems like there is very often an asterisk next to our numbers.

I was watching Aladdin with the kids this weekend and sometimes I feel like the genie in this scene:

ALADDIN: You’re gonna grant me any three [reports] I want?
GENIE: (As William F. Buckley) Ah, almost. There are a
few provisos, a couple of quid pro quos…

So what can be done? I think the best solution is to pick a good measurement package and stick with it. Question and try to verify the numbers occasionally, particularly if the metric in question is very unexpected. But don’t waste too much time on it — it will drive you insane and it will get you nowhere. The biggest value can usually be had by trusting your numbers spending your time acting on them.