This post was really informative, thanks. I wrote a script that calls the rss feed and basically re-runs a specific query for each day for a specific time period. For example I run “Madoff” (the financial ponzi scheme scandal) for the period of 2008 and 2009, which gave me around 600 and more result-entries.

This kind of historical blog search seems quite inaccurate from my initial results. I’ll give examples:

1-The total blogs for a day are inacurate estimates (simillar to what you’ve pointed out) except that this is for historical data from December 2008, Madoff was arrested for his Ponzi scheme on the 11th December 2008 (and hence this became widely publicly observed news), yet the blog count doesn’t jump from its average untill the 14th of December. God knows what google does, but as you hinted I also suspect that it uses some probabilstic estimation and it takes a few extra days for the count to be estimated upward, something of a moving average / high frequency signal filter…

2-The description from the rss for each blog post, is just plain wrong (many of the times), it basically tends to include commentaries rather than excerpts from the articles, as if google’s blog-page segmentation didnt do its job very well . For example on the 8th of December, Madoff was still pretty unknown and the ponzi scheme was still a well kept secret (he confessed to his sons on the 9thDec, who then notified the police and he was arrested on 11th), yet for the 8thDec the description includes the ponzi scheme…?? so looking at the page it turns out this is not the description of the article, but from the comments section, a comment from january 2009.

Hence, my preliminary conclusion… Using google-search for automated research purposes may be a tad inacurate )

Martin

It’d have been interesting to work for SIS, I’m quite sure, and I perhaps could have convinced myself that I could have acted as a positive influence within an organisation that seems to have lost its way a little. Deep down, though, I knew that would have just been so much empty rationalising. There’d always have been a nagging little part of my conscience that would have told me that by even being there and accepting a role offered by an organisation that was behaving in a questionable way, I was part of the problem. The fact of the matter is, even in organisations where technical workers aren’t directly involved in front line activities themselves, by being there at all we are lending our skills and abilities to whatever greater efforts the organisations that employ us are engaged in. As recent cases like Wikileaks have shown, technology plays a huge part in the activities of most organisation, private and public sectors alike. We can therefore have a bigger impact than we might think, for good or for bad, in the places where we choose to ply our trade.