From the material available the app looks great and it is also a nice example of what stuff citizen can do when government data is publicly available as Jeff used datasets from the recently started data.gov.uk portal. The remarkable point I want to highlight is shown in the video below when Jeff was asked by Conrad Quilty-Harper if there is a political point to his app:

The answer is a passionate “No“. Can you believe it?

Now I was not totally surprised because some years ago I wrote my MA thesis on “Political Motives of Developers for Collaboration on GNU/Linux“. While I found that a majority of the developers in my sample did actually attribute a political relevance (whatever it might be) to their coding of GNU/Linux, it was also clear from my research and that of others, that most programmers are motivated by the fun of coding. The Free Software community of GNU/Linux might be a bit of an exception given its founding principles but it is certainly not totally defamatory to expect the bunch of Web developers for gadgets like the iPhone to be less principled.

Is there something wrong with this? On the danger of over-generalising a bit too much I would argue yes. A tool like the ASBOrometer which makes transparent how screwed up your area is – this is more than just a funny app because it shows where the government has failed its citizens. Now it won’t come as much of a surprise to most inhabitants of these areas but the app gives you simple and quick access to actual numbers and compare your situation with that of others. This might well be what triggers some people to stop accepting their fate and get up to do something about being let down by society. As much as I hate to refer to the Sun to state a point coverage of the ASBOrometer in broadsheets like these certainly emphasise that is has struck a cord with people – also underscored by the fact that within two days it achieved over 80,000 downloads. Finally, lets not forget the economic dimension (and you don’t have to be a Marxist to know that little could be more political than this) as the newspapers rightly point out that this could have a huge impact on house prices in these areas.

Of course in my opinion all this makes this app all the better but I believe it is important that the people putting stuff like this together have some idea of its potential relevance. Ideally they might have an aim for this as well (I mean one relevant for the public good, not just fun for themselves) but hey, you can’t change the world every day. But the naivety with which software developer go about their work at my best of times amazes me but at other times just outright scares me.

As it says in the Spiderman comics:

“With great power comes great responsibility”

In a world heavily shaped by information and its free flow in the form of bits and bytes, people with computing skills are a small elite, able to manipulate this flow and determine its outcomes. There are many examples in which our current technology vanguards are making use of their skills with the aim to create some benefits to citizens such as mySociety in the UK or Sunlight in the US to name just a few. But we need much more of this.

All of this comes back to the old debate about the ethics of science and to what degree scientists (as just one example of an expert elite) are responsible for what they do – a debate wonderfully illustrated in Dürrenmatt’s The Physicists. Of course the ASBOborometer is no atomic bomb and I’m not in any way suggesting programming should not be fun or Jeff has anything but good (or no) intentions. But I really think it is time that even software developers understand that no tool is neutral and start to take responsibility for their creation – or at least try to understand its relevance. Because code is law, code is political!

To this end we analysed the RSS feeds of the named news aggregators and compared

1. timeliness – how often is the feed updated with new stories (interpretation: the more often, the better)
2. continuity – how well are topics pursued within a medium, that is do stories pick up on topics previously reported or is the news agenda very fragmented and disconnected (interpretation: less fragmentation is better)

As you might have read rather too often on this blog: the results are yet preliminary but hey, this is an academic work in progress. What we can infer from the results so far is:

• The traditional paradigms still outperform all competitors when considering “World News” (ie. international affairs, etc). They update very frequently (completely new content every 24hrs) and have a broad range of topics on their coherent agenda.
• The distributed paradigms do not yet work for news not concerned with technology but for this focus they perform very well. For example Digg receives about 100 submission per hour in the technology section alone but the community applies very hard filters to these submission so that only 1% eventually make it in the popular technology category (that translates to roughly one new 1 article per hour).
• However, what becomes clear from the analysis is that the news agenda of Digg and Slashdot is rather fragmented and in addition also very narrow ie. most stories come up only once and the few topics they really focus on are covered again and again and again (think Google)
• All aggregators, even the distributed ones such as Digg and Slashdot show very clear update cycles with low or none activity at night and very little during weekends.

From these findings so far it seems likely that distributed paradigms can compete with traditional modes of aggregation provided they have a sufficient user base interested in relevant topics. So far, however, relying only on Digg and Slashdot will give you mostly unconnected technology stories (but a lot of them!) and next-to-nothing about what else is going on in the world.

There are still a number of issues remaining including our selection of news aggregators and to optimize the continuity measure but for those of you who are already interested, you can download the slides.

PS: The project did arise out of a greater research project on the Performance of distributed problem-solving networks that the Oxford Internet Institute did in collaboration with the McKinsey Technology Initiative. You can find the relevant reports here.

The presentation was meant to give some overview of the Web 2.0 phenomenon and what implications it has for how government should make its information available. I argued that out of the two main distinctive features of Web 2.0 – collaboration (user-generated content, social web, etc.) and sharing (XML, APIs) – arise a number of challenges to the way governments provide their information:

1. visibility – People do not any longer turn to the government or indeed, any particular trusted information provider or brand. Instead, they use a search engine and in the UK that is usually Google.
   Data from our experiments in which we invite people to our computer lab to find pieces of government information on the Internet shows that 90% of them would go to Google to find the answer to a question. Only 1 or 2 out of 10 people would eventually end up on the government portal direct.gov.uk.
2. competition – What is more, due to this heavy reliance on search there is no guarantee that people will obtain their information from government. Instead, Web 2.0 technologies have been used for a number of non-governmental projects that mix and mash the information available on public sites to add real informational value and enable easier access. Some of these projects have been highly successful, such as UpMyStreet.com for local area information and TheyWorkForYou.com about parliamentary affairs.
   As a matter of fact, in our experiments we tend to find that some questions are answered without any government sites at all despite the fact that all the information we were asking for was essentially produced and provided by the state.
3. control – Now, whether or not you think this is a problem is a slightly different question but evidently there are issues around accuracy and control when public sector information is used and re-presented by third parties (be they social enterprises or businesses). Obtaining school league tables from the BBC website might seem less critical than for example travel advice.
4. license – Last but not least, the rapidly evolving technology has enabled a variety of applications current licensing models for public sector information need yet to find an answer to. A case in point is the recent controversy about the use of UCL CASA Virtual London model in Google Earth that was stopped by Ordance Survey because as Andy Hudson-Smith notes “The OS currently does not have the ability to license models for public usage and this is from a government-funded and approved agency.”

While I am not an expert in this arena, from my point of you there are some basic steps government should take in order to make PSI 2.0:

1. provide structured formats – this increases the value of the information both for the private sector as well as for volunteers who want to play with the data. I am not talking about some laboriously defined standard – anything that provides some unique identifiers and a basic structure will do
2. leverage the wisdom of the crowds – Following on from that I think government should not spend too much money on information visualization themselves. There is simply no way departments can come up with some of the more ingenious ideas of data representation and cross-referencing that the Web community has generated. Instead it makes sense to offer the data, let others make interesting stuff with it and if it works, incorporate it into the state site. There have been some interesting examples of that for the BBC (see BBC Backstage) and the UK parliament (who now gets its post code – MP lookup from UpMyStreet).
3. government certified data stream – one way to leverage the creativity of the Web community while at the same time providing some form of guarantee for the accuracy of the data is a kind of “government certificate” for sites that basically states the project is using the official government data stream and incorporates changes to the official stream in real-time.
4. sort out the licensing – Yes, this is difficult. Does not necessarily mean everything should be available free of charge.

Some of the things that came out of the subsequent discussion will be available via the APPSI meeting notes soon. For now, let me know your opinions on the question of how government should deal with PSI in a Web 2.0 world.

I was quite satisfied with the response so far and it has also really been useful to connect to other people that do similar work. Social network sites as well as geography have been some important topics here. It has been especially enlightening to listen to Mike Thelwall’s analysis of 15,000 MySpace profiles that did help me to qualify some of my findings (such as that girls make better friends and older users have less friends). There are some other insights in his paper so you should check it out on Mike’s website.

There has been quite some interest in this kind of research and I just wanted to highlight a couple of really useful tools for analysis of MySpace and other network sites. First of all, for MySpace I am helping to develop a Perl module (WWW::Myspace) that actually acts as an API that interfaces with MySpace (via screen scraping) to provide access to the (public!) data on people’s profiles. It is now working rather well and can give you friends, comments, location information etc. If you want to give it a quick look, check my simple WWW::Myspace trial.

If you are not familiar to programming, you might want to have a look at dapper.net. It lets you select the sections of a web page that are relevant to you, extracts data and exports it into a format of your choice. I haven’t used it myself so far but it seems really useful.

If you have any comments or question, please leave a comment on this blog or send me an email!

“Last but not least, the subject under public scrutiny does matter as professors might well be made to face personal criticism in their role as public figures while teachers and nurses might have to be treated differently.”

Well, I might have been wrong. A couple of weeks ago the Regional Court of Cologne has decided that teachers too have to tolerate to be rated online by their pupils. In the case a German teacher had sued spickmich.de, a site similar to the British RateMyTeachers, that allows pupils to mark their teachers in nine different categories such as fairness of grades, quality of the lessons and difficulty of exams. So far more than 150.000 kids have used this service (according to the publishers).

The court made two important rulings: First, the ratings of the pupils are clear value judgements and are acceptable under the principles of freedom of expression as long as these are not defamatory. The court has given a lot of freedom and even allows polemic statements as the criticism of the pupils has the necessary relevance (after all, they are talking about people they meet and work with every day) and the evaluation of the school and its teachers is of general interest. Second, the judges did not follow the argument of the claimant that displaying her name next to the rating would constitute a violation of her privacy. They argue that this is information that could be easily obtained from the school’s website.

While this is an important ruling in favour of collaborative group judgements (something my colleague Malte Ziewitz is working on) there are a number of particularities about spickmich.de that merit highlighting to assess the ruling properly. First, there is a need to register on the site. The ratings are not publicly available and pupils have to register on the site in order to rate their teachers. Another difference to other rating sites is that the service does not allow for personal comments about teachers but only for grades between one and six (like the German grading system) in different categories. Finally, looking at the screenshots of the site it seems that what spickmich.de is really after is to establish a new social networking site for pupils that piggybacks on teacher ratings to reach a critical mass. It is telling that spickmich.de refers to itself as a pupil’s network.

Bottom line: I think this has been a good ruling in general as collaborative efforts with general interest goals should be encouraged and not stifled by the law as long as they do not harm anyone. Still I am slightly worried. The creators of the site do not get tired to emphasize that overall marks received by the teachers are rather good (average of 2.9) and that the site is intended to collect fair and objective judgements in order to improve education. But despite their arguably good intentions it remains the secret of the creators of the site how to reach a “fair and objective” judgement in categories such as sexy (or ugly), cool and funny (or embarrassing and boring) or easy (or difficult) exams.

While the association of German teachers made the predictable outcry about this effort one should make no mistakes: the role of teachers is difficult, public demand on them is high while appreciation of their work is low and although for years German teachers have been paid much better wages than their British counterparts, burn out syndroms are common, contributing to the German situation in which only 5% of teachers do actually work until there legal pension age. I yet have to see a fancy Web 2.0 application that would give teachers something in their hands to cope with their everyday work. Are there any social networking sites for teachers?

The opinion of the general public is in high demand as exemplified by a recent announcement of the British National Health Service (NHS) to introduce NHS Choices which among other things will allow patients to comment on their hospital treatment. The move has clearly been inspired by the success of PatientOpinion, a not-for-profit site which has allowed people for more than a year now to voice their experience with treatment received in hospital (and for which this new state-run competition raises some problems of its own).

Commendably recently even the Prime Minister’s Strategy Unit within the Cabinet Office has launched a review that is intended to highlight ways on how to leverage the Power of Information generated by state as well as the public.

However, not all is perfect for this new form of user empowerment. Especially one form of rating site has come under attack: sites which allow to comment on the performance of teachers, in particular a site called RateMyTeachers. These sites have come under fire as teachers complain about cyber bullying:

“Another teacher at the same school is criticised by a pupil because they “cannot speak English” and another as “the worst person ever” and “everyone hates her”.” (BBC, 3rd April 2007)

“Mr Brown said: “Comments are one thing but what about teachers who’ve had images of their heads super-imposed on to gratuitous images or who have had pictures taken and posted of their cleavages or underwear as they bend over, or who have had comments questioning their fidelity to their partner?”" (The Independent, 4th April 2007)

As every reader of newsgroups or forums has learned long ago many people have yet to master the art of making constructive criticisms and it is understandable that teachers take offence at anonymous comments like the ones quoted above. The concerns are serious enough for some to even call for a closure of such sites.

A new court decision in Germany could lead the way in assessing these issues in the future: The German website MeinProf.de which allows students to rate their professors and comment on their performance had been sued. After some nasty comments (including naming the professor a “psychopath”) that were quickly removed by the web site owners the professor in question went to court demanding the operators to pay 3,000 Euros (about £2,000) for any similar comment about him that might appear on the site in the future.

The court has decided that a general “cease and desist” for unacceptable comments is against the law. As a professor one has to face public criticism that cannot be prohibited ex ante. (see full coverage as well as press announcement – all in German I’m afraid).
(btw I wonder what Wendy’s take on the situation is…)

Several things have to be noted: In general this is a positive outcome for web sites that leverage the wisdom of the crowds as it offers some protection for the often not-for-profit operators of these sites. However, this does not justify defamatory comments on those sites and the court has emphasized the operators’ duty to remove those entries as soon as they are recognized. Last but not least, the subject under public scrutiny does matters as professors might well be made to face personal criticism in their role as public figures while teachers and nurses might have to be treated differently.

(update:for a near live coverage see Lex Ferenda)

UPDATE: if you’ve missed the conference, the Berkman Centre has now made videos of the conference available)

I will try to cover what’s on the horizon.

Jonathan Zittrain (OII and Berkman) and John Palfrey (Berkman) introduced a new distributed application that they are devloping. It’s called Herdict – meaning “a verdict from the herd” – and is basically a software application people can install on their machines for the following purposes:

• Detect Badware: collect vital life signs of the computer
• Document Internet filtering: document Internet sites the user cannot access
• Measure Network Neutrality: measure network latency

The data collected by Herdict is sent back to a Berkman server, aggregated and distributed again so that each user can compare the performance of his/her machine/network with the performance of other machines in his neighbourhood or across the world. Eventually the software could help users

• to decide whether or not to install a certain piece of software (based on other user’s experiences with it)
• finding out that there is a problem with your machine as yours is performing much worse than everybody else’s
• produce a near real-time map of Internet filtering around the world that allows for an analysis of where filtering is actually taking place: on state, ISP or institutional (eg. company) level

The setup is likely to remain a centralized client-server architecture (P2P is possible but creates a whole host of security and trust problems) but the ultimate aim is to allow also interfacing with the data via an open API so that 3rd parties can create widgets using the collected information.

While the audience was widely in favour of the capabilities of such a tool, many concerns were raised about whether the privacy and security of Herdict users would be at risk. An Ethan Zuckerman in full constructive-criticism mode raised a whole host of issues but also suggested some potential solutions. Others joined in the discussion which I try to summarize below together with some responses from Jonathan and Steven J. Murdoch who is responsible for some of the technical ideas behind Herdict:

1. Does the collected information about a machine’s (mis)configuration not help malware programmers?
   JZ: These people do already know enough about the weaknesses of other people’s machines (e.g. via their botnets) so this information won’t help them but will make a difference for the good guys
2. Does one maybe draw attention to an insecure configuration on his/her machine so that it can be targeted specifically?
   JZ: Definitely important to strike a balance between openness and the ability to misuse this information. Possible solutions could be to synthesize the collected data. However, the applications installed on your computer give out so much of your information already (e.g. Skype, IM) that one should maybe worry less about Herdict but just start to use the information for a good purpose (if you cannot stop it from being distributed anyway)
3. Doesn’t the centralized client-server architecture make it easier to manipulate and interfere with the data collection?
   JZ: Exactly because the data is centralized it is easier to guarantee its freedom if it is operated by a trusted party with adequate licensing (see Wikipedia example)
   SM: client-server could be more secure than P2P because you only need to make sure that the server is trustworthy, not the whole array of hosts in the network. Also there might be a centralised aggregation of the collected data but a distributed communication to transport the information. The data will be sent encrypted and stripped of personal information.
4. Is a centralized architecture not easily blocked?
   JZ: This would be a sign of success (as governments would apparently take Herdict seriously) but if that would really happen one could think of new ways of working around that.
5. Even with Herdict it would be difficult to detect tempering with pages (e.g. the BBC page is served but with slightly different content, e.g. less critical of your countries government)?
   JZ: One could incorporate a review or user comparison of web pages, something like the ESP Game
6. Do people not draw attention from the authorities to themselves if they start using Herdict and in this way accessing forbidden sites to document filtering?
   Ethan: One idea could be just to report failure of accessing a site back to the Herdict server or a model similar to SETI@home where you download a bunch of URLs for testing. Also, no need to disclose full IP address as class C should be enough to estimate your location.

If you are interested, download the alpha version. which so far is only measuring and comparing the machine’s life signs (e.g. amount of free memory, number of processes running etc.)

Apart from requests from the community, reasons for this development include

• opportunity for firms to let others develop additional functions for own service
• also good for internal use (e.g. Google uses APIs for internal development between different services)
• would allow users an option for easy import and export of their personal data.

Naturally, there is some resistance within the company but surprisingly the first parties that indicated interest were not so much individual developers but big players like IBM and eBay that wanted to incorporate Xing data into their applications.
get data of your contacts.

Xing’s main problem is that it has personal data of its users and the integrity and security of that data is paramount. Although there was some discussion, apart from enabling users to export and import their private data, I cannot see how a useful API could work that does not immediately raises privacy problems. So not surprisingly Gehret was indicating that the public API could eventually be shelved while it would probably still be available for companies that use Xing to organise their human resources.
There also is a legal difference between making data publicy available via the Xing website and to enable automatic data processing (via API for example). While one could argue the moment I put my stuff on Xing on public view it would not matter whether that data is also easily machine-readable via an API. But, legally, in Germany that seems to be a difference. Interesting: the use of screen-scraping tools breaches the contract of Xing – although I’m not sure what kind of contract someone would have with the company if he/she does not have a profile on Xing.

Also on offer a panel about the social dimension of the information society, focusing on the working conditions of notebook manufacturers in China. Interestingly, there seem to be only 11 companies that produce notebooks that are all Taiwanese and produce the hardware for the companies of Sony and the like. These manufacturers produce their hardware in China where one won’t be surprised to hear that working conditions are usually poor. Andreas Manhart from the Oeko-Institut presented a study on the subject that found dangers arise especially from toxic materials in the production process. There was a bit of discussion on whether import duties should be levied on hardware that is produced under such conditions or how the customers’ awareness to this problem could be raised. However, the main problem is simply that at the moment there are almost no manufacturers that produce ethically correct notebooks.

Not too much news from this discussion but Glaubrecht again highlighted that the use of the Google APIs consitute a contract that can potentially be revoked by Google. This has happened in the past already. One thing he made clear was that Google makes sure it got the IP from every user and will not allow the use of its APIs (at least Google Maps) from within company intranets. The justification for this is that Google has a contract with the map providers that only covers the distribution to individuals, not companies.

Interesting as well: there are plans at Google to develop something like an identity container that bundles all the data a company keeps for a customer. The aim would be to enable people to transfer their whole online identity from one place to another, say for example to migrate all your data with Google (e.g. Calendar, Mail, possibly searches) to another provider like Yahoo. Don’t think this will come true soon but at least this would give some more control over personal data back to the user.

Bernhard Reiter took the probably rather unpopular (but important) stance of questioning the transparency of APIs in general. He rightly argued that these APIs create an dependence from the information provider.

btw: a very cool idea are comments to the presentation via SMS that get subsequently displayed on a big video screen next to the speaker. It’s heavily used but rarely for questions or comments to the ongoing presentation. Still, very entertaining

I already mentioned some people that I’ve met here and that have set up German alternatives to del.ico.us and digg. Funnily enough I also met people here that I wanted to meet in the UK but didn’t manage yet: Hello to Tom Steinberg. As a premature resume after the first day: there is a lot more going on here than I anticipated and there is also more collaboration happening than expected.

One of the interesting presentation here was from Jan Schmidt who is working at Forschungsstelle “Neue Kommunikationsmedien” in Bamberg. His aim was to clarify three big myth about the blogosphere – he is in a good position to do so because he is the author of the biggest German blogger survey to date. He identifies three myth about the blogosphere:

1. bloggers are nerds
2. blogs consitute an alternative (counter) public sphere
3. blogs are irrelevant

myth 1: bloggers are nerds

The main stereotype of bloggers is that they are male, overweight and unsocial. However, Jan argued that most of the studies to date are not representative. His own research has shown that there is rather a balance between male and female bloggers with another important study by Harders/Hesse even arguing for a blogosphere that is to two thirds female.

myth 2: blogs are a counter public sphere

Jan argued that blogs cannot be called counter-cultural because they mainly quote established news media and talk a lot about the personal lives of the bloggers. I don’t fully buy his argument because quoting news media does not mean one just mirrors the mainstream opinion. I’m writing a paper that compares the agenda of the mainstream media with the blogosphere that will hopefully contribute some data to this discussion but more on this another time.

myth 3: blogs are irrelevant

Interestingly an argument that also comes from bloggers themselves in order to distinguish themselves other blogs. Jan rightly argues that this myth is created by applying the relevance criteria of the traditional mass media system. However, for most bloggers it is not important to get a big readership but their blogs serve a different purpose: connecting with like-minded people, keeping in touch with friends etc. If their blogs can achieve that goal, they have a personal relevance for these bloggers regardless of the fact that they are irrelevant to 99% of Internet users.

More is to come so stay tuned …