There are three kind of lies: Lies, damned lies and statistics.
— Benjamin Disraeli, British politician (1804 – 1881)
The thing about statistics and browser share has come up at Asa Dotzler’s blog, commented on at OperaWatch, and finally scowled at by Arve Bersvendsen at Virtuelvis.
So, what seems to be the problem? It starts with some problems figuring out how 60 million downloads of Opera 7 can translate into 1% browser share, while 50 million downloads of Firefox 1 translate into 8-10% browser share. That there were some versions of Opera starting with 7 does indeed explain quite a bit – people upgrade as new versions are available, even if it isn’t a full version number. However, not everyone download the browsers from sites which count the downloads: Some get it from CDs/DVDs on computer magazines, or download it from other download sites. Some download one copy but installs it on several computers. Trusting download stats is therefore fraught with insecurities when it comes to number of users.
Then there is the browser share of 1% – how accurate are the statistics? You may very well claim that the counters/services that provide the numbers for these stats are able to recognise Opera, even when masquerading as IE or Netscape/Mozilla, and you’d be right. Well – except when Opera 8 makes use of the two new, thoroughly camouflage modes. But the question of how accurate the services are remains, how good they are to recognise Opera every time – because Opera makes good use of the cache, and doesn’t show up as a new visit as often as other browsers. You can change the default settings for the cache in Opera, but if you don’t need to with your kind of browsing, why would you? Read more at Virtuelvis about the underreporting and overreporting of browsers in statistics.
What’s left for us is to remember that statistics can be read in different ways – we must be aware of what statistics tell us – and what they don’t tell us. Anything else would be a mistake.
I’m not critcizing and I’m not trying to pull off some statistically sound comparison here but it was very interesting to me that what Opera claims for downloads and what Opera claims for usage is such a radically differnt ratio than what Firefox has. It’s not even close. I find that interesting and worthy of comment.
You bring up the browser share accuracy issue and I’d like to repeat myself for about the third time on this; I’m quoting Opera’s own number. If they don’t believe the 1% number, then they shouldn’t be telling the press and the world that’s what they have. I’ll buy your suggestion that varios measures could be flawed, but if that’s the case, and Opera Software believes that’s the case, then why are _they_ claiming that number.
Again, this isn’t some attempt to make hard science out of something this difficult (even impossible) to measure. I’m simply wondering how there can be such a difference. I’m also wondering why no one else has questioned the Opera claim of 60 Million downloads and 1%. I’m not saying that the press should necessarily question the 60M part either. I’d be equally interested in someone saying “Well, Opera must actually at about 10% of the market so it matches up pretty well with what Firefox is seeing.” (not that I believe that to be the case.)
If Mozilla had made the statement that we accept the industry’s claims of ~9-10% marketshare and went on to say that we’ve had 500-600 million downloads of Firefox, I’ll bet there’d be a lot of people questioning that. Don’t you think? I mean, I’d be scratching my head. Wouldn’t you?
– A
I don’t doubt you when you say you’ve got the numbers from Opera, nor do I doubt you when you say Opera itself says there is 1% share for their browser. That 1% is the number the statistics provides us with, and we can’t just ignore it. We still have to know what the statistics tell us and not, and that means we have to know how the numbers are collected. Many don’t give this a thought – even many who should give it a thought (generally speaking.)
I don’t say that the 1% is wildly inaccurate – I say that we must keep in mind how this number is reached.
The download numbers? How many versions of Opera is included in those, and how many versions of Firefox? How many times is Opera and Firefox (and IE and…) downloaded from sites that don’t provide their download numbers? How many cover CDs/DVDs carry Firefox and Opera, and how many installs the browsers from these? So the download numbers are even less useful to find out the number of users, and can’t really be compared to the browser share.