Tobias Escher at the OII

Just in case you have not come across this site yet: Hitwise is a company that provides fascinating data on Internet usage. They are able to provide some rather comprehensive (and possibly more accurate) data as they are cooperating with Internet Service Providers (ISPs) and analyse their network traffic If you are interested, they’ve posted a nice video on their site outlining the methodology. It is a bit like standing on the street and counting the cars going by – with the big advantage that the cars also clearly display where they come from, where they go to and what they intend to do there (ie. “searching for Halloween costumes”).
So instead of analysing the logs of individual websites, they basically take a sample of Internet users (all that connect through the ISPs Hitwise has a contract with) and observes what they are doing. They do that on an aggregated level so that supposedly no personal information is extracted. However, to get some useful demographics they also run user panels. 

Take that all together and you can get a lot of insightful data such as that one out of 200 British Internet users was visiting a blog last week or that we should expect a lot of pirates, Darth Vaders and Wonder Women this Halloween (see also chart below). If you are interested in it, Heather Hopkins is an analyst at the company and her corporate blog reports on interesting findings for the UK with the Hitwise data.

The quality of the data is clearly related to how much traffic Hitwise is analysing and from which ISPs it comes from. They are not very open about that but in one post they claim to observe the online behavior of 25 million Internet users every day and in another place they claim to measure about 500,000 websites out of 160 industries every day. I would guess that this is the total over all the countries they cover which are the US, the UK, Australia, New Zealand, Hong Kong and Singapore.

I just got an update on that (see also comment) from Heather: Across all markets we cover, our sample is over 25 million internet users. In the UK, it is 8.43 million (as you mention). In the US it is 10 million. The rest covers Australia, New Zealand, Singapore and Hong Kong.

Early this year Matthew Hindman (Political Science Department, Arizona State University) who coined the term Googlearchy was a guest speaker at the OII and he was actually using data from Hitwise for his research on shifting patterns of information exclusivity.