Showdown at high noon: 7 more myths about web analytics
Following on from my last post, some more myths about web analytics.
Myth: I should use the total metrics instead of the domestic metrics, because total is always bigger.
Fact: Speaking for NZ (where I’m based) – If web analytics really is like the Wild Wild West, then total traffic numbers are as alluring as Salma Hayak. They’re so nice and big and plump compared to domestic (NZ only) traffic numbers. But while total is always bigger, bigger isn’t always better.
If you’re pitching to advertisers, what they really want to know is how many people visit your site that could buy their products. About 99% of the time that’s just people in New Zealand, so that’s really the numbers they’re after.
Myth: Monthly stats are the best.
Fact: Web analytics often has to deal in monthly stats, because most companies have monthly reports to file to bosses. Unfortunately, monthly stats are much less accurate than weekly stats. For example, if I’m looking at monthly UBs for March vs February, I’ve got a problem – February only has 28 days, and March 31. You shouldn’t compare the two.
You can sort of get around this by using average daily UBs, but then you run into the problem of weekends – different months have different numbers of weekdays vs weekends, and that can have a major effect on traffic.
You can compare average daily UBs between different sites for the same month reasonably accurately though. But there are issues with this too.
So stick to weekly traffic unless you have to do otherwise.
Myth: Lies, damn lies and web analytics. They’re just numbers, and I can make numbers say anything.
Fact: You can usually pull numbers to support your pet notions, but that shouldn’t be your goal. If you go in with an open mind and good intentions, you’ll find the numbers start to tell their story pretty quickly, and you’ll get an honest answer. It may prove the opposite of what you set out to prove, but you’ll get an answer which better reflects the underlying reality, and in the long run that’s much more useful.
Shaky numbers won’t stand up when they’re challenged, and someone eventually will challenge them.
Instead of looking for evidence to support what you think is going on, look for ways to test your hypothesis. That’s how you’ll get good answers.
Myth: So if I’m honest and know what I’m doing, the numbers can tell me exactly what’s going on?
Fact: No, they can’t do that either. The traffic data you see is only a reflection of reality. While the numbers look deliciously precise, they should be seen as approximations. Look for trends, patterns and differences. When in doubt go to the help files and see exactly how things are being measured.
Myth: I should stick to the established metrics.
Fact: Established metrics are “established” because everyone agrees they’re useful. But this is the Wild Wild West, and I’m like the other guy in the movie, the one who invents stuff. I have about a dozen metrics I use that I made up by mashing up the numbers into new combinations. This lets me see new patterns and comparisons.
Just, you know, mash carefully. Make sure you understand exactly how things are being recorded beforehand, otherwise it’ll be garbage in, garbage out.
Myth: Wow. My Gisborne community news site has a lot of traffic from Auckland.
Fact: Geotargetting readers is a great burgeoning technology. Unfortunately for us Kiwis, it doesn’t work so well in New Zealand, because of the structure of the internet here. It’s all very nerdy, but essentially a lot of internet traffic gets pushed up through Auckland, even if you’re down in the South Island and the site you want is hosted across town. So you can’t trust the geotargetting that much, but again, comparing over time or between sites can be useful (since the bias is shared, the differences should be real, but not the exact numbers.).
It does seem to be getting better though. But for now, surveys are your best bet.
Myth: It doesn’t matter which web analytics service I use.
Fact: You get what you pay for. If you use a free service like Google Analytics, don’t be surprised if the numbers look a bit squiffy.
To be fair, I really like Google Analytics. The interface is simple, it’s fast, and you can actually get a lot of interesting stuff out of it. I just don’t trust the data, because of the way Google treats referred traffic (basically artificially inflating it, especially search – more on the problem here and here).
This isn’t really a bug, more how Google looks at web traffic, but the data isn’t saying what most people would think. You can configure GA to get around this, but by “you” I really mean “a consultant”, so be prepared to pay.
The unconfigured Google Analytics is really meant to be the starting point, then you customise it to your own needs. If web analytics isn’t crucial to your business, the unconfigured version will probably do just fine. If you really want to know where your web traffic is coming from, then it’s worth hiring a consultant to get it configured right.
If you shell out for a business-grade service (eg Nielsen NetRatings, Omniture), you’ll see much different (and more accurate) data for referals (partly because you’ve spent time going over your requirements with the vendor).