The US startup ambitious journalism projects that have sprung up recently, wholly or partly funded by foundations, in essence share the “steal this story” approach:

Instead of planning how to get the story published before word of it leaked, the excited editors started throwing out ideas for how they could share Johnson’s reporting with a large array of competitive news outlets across the state and around the country. No one would get a scoop; rather, every outlet would run the story at around the same time, customized to resonate with its audience, be they newspaper subscribers, Web readers, television viewers, or radio listeners.

The quote describes California Watch, who also have case-studied themselves.

The huge advantage that “steal this story” in its different versions has over “please pay here”, is that it helps to sustain and modernize a broad, inclusive networked public sphere just at the moment when (some) established media institutions are retreating from their old strong positions (newspapers, broadcasting) due to the technological shifts and from their web strongholds (from free, ad-based websites to subscriber-based services/apps).

In Europe, this trend will play out differently because of the still strong role of license-fee based public service broadcasting. “Steal this story” seems like the perfect chance for these giants to reinvent themselves and secure their legitimacy in the digital era. Their self-image has always been the guarantor of a public sphere based on high quality journalism, culture and entertainment. But it is far from certain that they will succeed. For one thing, they are deeply embedded in a pre-digital culture of copyright regulations. Huge resistance is to be expected to any radical move towards the two mentioned versions of “steal this story” — publish material under Creative Commons licenses and/or let “competing” media organizations re-publish. One symptom is the seemingly endless negotiations needed before historical broadcasting archives can be made available for re-use (or any use at all).

The challenges facing public service broadcasters will however open up possibilities for new initiatives. In a networked public sphere, there is no reason why public service content must be produced by one huge, centralized organization. In the current situation, even small-scale funding (such as the British 4iP) can have great impact. In Europe, this is an argument for redistributing some of the funding for public service broadcasters to new and innovative projects.

Today, it is more than likely that a new “Goethe” would submit the documents to WikiLeaks instead of contacting a book publisher or media organization. The recent strange and consequential case involving the arrest of Bradley Manning and his alleged involvement with leaking the Iraq helicopter attack video which was published by WikiLeaks demonstrates the potential of such a global whistleblower site, though we do not know all the details of Manning’s actions yet.

In April, I heard Daniel Schmitt of WikiLeaks present the concept and thinking behind the site at a conference in Berlin. WikiLeaks will factcheck all submitted material thoroughly in a verification process where they can draw on more than 1000 anonymous experts in different fields. If they conclude it is genuine and of public importance, it will be published — and crucially, all the material a source has sent in will be published, not only edited excerpts. This principle sets WikiLeaks apart and constitutes much of its appeal. A new “Goethe” would not have experienced the same as Daniel Ellsberg did with the Pentagon Papers — indeed, Ellsberg has said that today, he would have posted them on the internet.

WikiLeaks try to build a structure than can withstand attacks from any government or other powerful institutions, using cryptography and different safety mechanisms to ensure that no document first published can be made to disappear from the internet again. Also, Schmitt and founder Julian Assange claim that WikiLeaks is “lawyer-proof” (the new Icelandic media law might help them here).

WikiLeaks guarantees to always protect its sources and in this subscribes to classic principles of the press. But as I interpret them, WikiLeaks claim that their guarantee is more credible than a similar vow from traditional news media. That is because they are not constrained by the ties of traditional media organizations — ties to national power structures, personal ties connecting elites, the need to protect an established position in society.

Schmitt emphasized the neutral character (again, my interpretation) of WikiLeaks: A true, important document will be published in its entirety. Just the facts, make your own interpretations and judgments. He was asked about the Collateral Murder video, which had been published shortly before and was strongly edited to achieve maximum effect on the viewer. Assange went on a promotion tour in the US to present the video. When does WikiLeaks cross the line to editorial comment? Schmitt claimed that Collateral Murder was distinct from the WikiLeaks project, the result of journalists editing the material, but he acknowledged that the line between the publication of the raw source material and the edited version was not drawn clearly enough in that case.

As we now know from the New Yorker report on how Collateral Murder was created, this was very much an Assange/WikiLeaks production. Assange does not seem to be content with the neutral, facts-only approach, being the world’s whistleblower supporter — he crosses over into activism. But on what basis? Is transparency alone enough as ideology? As Raffi Khatchadourian puts it the New Yorker:

But experimenting with the site’s presentation and its technical operations will not answer a deeper question that WikiLeaks must address: What is it about? The Web site’s strengths—its near-total imperviousness to lawsuits and government harassment—make it an instrument for good in societies where the laws are unjust. But, unlike authoritarian regimes, democratic governments hold secrets largely because citizens agree that they should, in order to protect legitimate policy. In liberal societies, the site’s strengths are its weaknesses. Lawsuits, if they are fair, are a form of deterrence against abuse. Soon enough, Assange must confront the paradox of his creation: the thing that he seems to detest most—power without accountability—is encoded in the site’s DNA, and will only become more pronounced as WikiLeaks evolves into a real institution.

News media have often been criticized for being secretive, but WikiLeaks beats them all. It is hard to see how WikiLeaks can become more transparent. In the end, a new “Goethe”, the journalists and audiences of the world will just have to trust them. Is that pact sustainable in the long run?

When the application deadline expired at midnight we had received 135 applications to Nettskap! We are very pleased!

This exuberant tweet was published on the Ministry of Government Administration’s official account on May 10. The idea competition Nettskap 2.0, a Norwegian version of the Apps for Democracy or Show Us a Better Way contests, had been announced just a month earlier, which makes the quantity of applications quite impressive. Nettskap applicants were not required to place government data reuse at the heart of their projects, but the Ministry had encouraged this. It worked – 90 projects, two out of three, are based on the reuse of data.

Winners of the competition are to be announced in June, and will receive funding totaling 2,5 Million Kroner (around 320.000 Euro).

Nettskap is just one of several pieces of good news for government data enthusiasts coming out of Norway lately. In April, the same Ministry unveiled plans for a Norwegian datastore. The URL has been chosen – data.norge.no – and work is now underway on the project details and tender. In the meantime, a blog is set up at data.norge.no where government data questions are discussed.

Two reports on public data reuse have been published already this year, helping to raise interest in the topic. The first was written by a group at the University of Bergen (led by me). The second was produced by an expert group put together by the Norwegian Board of Technology, a consultative office to Parliament.

Norway has long had the potential to become a very interesting place for government data reuse. The public sector is, relatively speaking, considered well-run and efficient, and lots of quality data are collected. There is a deep-rooted tradition of transparency in government, supported by a Freedom of Information Act (FOIA). And the population is more tech-savvy than most, always quick to adopt new communication technology and services.

But despite these favourable conditions, Norway has not been in the data reuse vanguard so far. The accelerating activity and increased attention given to government data issues during the last months might change that. In this article I will consider three factors which I believe will decide the outcome:

• Infrastructure: The coming datastore needs to be supplemented by an open license for data reuse and practical initiatives to encourage data publishing.
• Principles: The PSI directive has been implemented in the FOIA. However, the fundamental principle of division of labour between the public and private sector (and civil society) in data reuse is still not generally acknowledged.
• Reuse in practice: Some public sector agencies are already doing an excellent job making data available, while others haven’t even started. Convincing examples of good reuse are quickly needed for recent positive developments to take hold.

Building the infrastructure
The “data.govs” in the US, UK and other datastores give much-needed ammunition to government data advocates everywhere. In a list of recommendations that concluded our report at the University of Bergen, the swift creation of a Norwegian data.gov was in first place, and many others have voiced the same demand. After first indicating that it was under consideration, the Minister of Government Administration, Rigmor Aasrud, announced the URL data.norge.no in early April.

Not content with waiting for a datastore, several private initiatives were taken during last autumn to start a crowdsourced collection of public sector data sources. This was an important part of our own Bergen project, where we started with an open Googledoc spreadsheet. Some 130 data sources were registered there in a couple of months. And one IT engineer started a wiki called datakilder.no, with essentially the same goal.

The experiences from these experiments are available to the Ministry as it now plans its official datastore, and I believe they could be valuable. Moreover, through a cooperation with the Open Knowledge Foundation, we recently exported the Googledoc data to a Norwegian language instance of the foundation’s Comprehensive Knowledge Archive Network registry (the same software that is used in data.gov.uk).

We are currently considering how to continue this work now that data.norge.no has been announced. Personally, I think an independent datastore run by the open data community has an important role to play in pressing government agencies to release data (here we can be inspired by our German and Canadian colleagues. We have no guarantee that the official datastore will be filled with interesting data instantly – probably it will not. Also, an independent datastore can be used for experiments – such as including international and private sector data sources, and promoting and testing the principles behind linked open data.

Besides the datastore question, the need to define standard licenses for data reuse has been a key request from the open data community. At a conference held by the Ministry in Oslo last December, Opera Software CTO Håkon Wium Lie said that coming up with licenses should be priority number one for the Ministry. Lie, who helped organize a citizen campaign for opening up government data back in 1994, said that in principle there were only two reasons for not publishing data: To protect personal data and national security.

Licenses are certainly sorely needed. Today, practice varies greatly between government agencies. Some attach explicit conditions to their data, others do not inform at all. Some only allow non-commercial reuse. Typically, these differing conditions are confusing to potential re-users. Last December, the Norwegian Mapping Authority released map “tiles” for non-commercial reuse (but not the underlying vector data). It soon emerged that the definition of non-commercial was flawed. For example, would embedding map tiles in a for-profit news website count as commercial use? The Mapping Authority had to edit its terms and conditions.

It remains to be seen how open, or how restrictive, the licenses will be. This is certainly one of the crucial questions now.

During interviews and conversations with civil servants for the University of Bergen report, we noticed great discrepancies in knowledge and interest. We found agencies that had been working professionally with their data sources for years, while others hadn’t given the issue much thought at all. The need is clearly there not only to define licenses, but also to produce a comprehensive and practical guide to opening up data for reuse. We have tried to contribute by translating Ton Zijlstra and James Burke’s government data flowchart into Norwegian.

Adding to the recent positive developments, the Norwegian Ministry has made it clear that it is working on such a guide. I believe the open data community outside government should be involved here as well. This could be done in a number of ways. One possibility is to have everyone comment on a draft of the guide, but the community should also contribute with its own initiatives. Setting up a wiki collecting examples of good practices is one option.

Implementing the principles
Government agencies should make data available for reuse in relevant formats, then leave it to the private sector and civil society groups to present the data in creative and interesting ways (for-profit or not). This assumption of a division of labour is central to the whole idea of government data reuse.

In practice, government agencies will present data as well, and it is hard to see that it would make sense to stop them doing that – as long as the data are released for reuse at the same time. However, the government data debates in Norway show that the fundamental principle is not accepted by everyone. Now and again you hear the argument that it is “unfair” that private companies should earn money on reusing the data collected by a government agency.

Another typical argument against publishing data is that they can be misinterpreted. In a survey of state agencies for the University of Bergen report, 43 percent of respondents agreed that “private businesses and individuals can misunderstand data and disseminate misleading information”. They are probably right – it is impossible to guarantee against misinterprations. However, today it is a very difficult argument to make that information should be kept from the public sphere because of such concerns (how much information would we not have to withdraw if that were a guiding principle?).

An easily discernible tendency that is noted with frustration by those seeking access to government data, is the public sector agency that spends resources on producing its own presentation of data, without releasing the “raw” data. In the worst case, this is the spin doctor’s data presentation.

The good news is that the open data community – from entrepreneurs to journalists to activists – have been handed a new and potentially powerful tool to persuade agencies to make data available.

The EU’s PSI directive was implemented in Norwegian law through the changes in the Freedom of Information Act which came into force January 1, 2009. This is the new section 9:

Right to request access to a collation of information from databases:
Any person may request access to a collation of information that is electronically stored in the databases of an administrative agency insofar as the collation can be done using simple procedures.

In the regulations, the Norwegian Mapping Authority has been permitted to continue its policy of charging for access to map data. Given the importance of map data for so many types of applications, the Mapping Authority’s pricing regime has been heavily criticized for years, dominating the limited discourse on government data.

That debate will predictably continue. Less well known, but now beginning to seep through to the open data community, are the possibilities created by the preparatory documents to the revised FOIA. In the Ministry of Justice’s guide to the Act, the right to copies of digitally stored data is extended to copies in “all existing formats”. It remains to be seen how this will be interpreted in concrete cases. But it is already clear that the unpopular practice in many government agencies of publishing data sets in the pdf format will have to change. At the least, agencies will have to make spreadsheet versions of those data available.

Having data reuse principles “merged” with the FOIA will probably have positive consequences for those of us who work for better access to government data. The FOIA carries a lot of weight in Norway. It gives the open data community a much stronger foundation for seeking access to data than arguments about innovation and economic growth can.

Adding to this is the well-established routines among journalists of using the FOIA as a tool in their daily work. For over a decade, media organizations have had online access to the correspondence register of ministries, and agencies on other government levels have implemented their own public registers. On basis of the information in the register (sender, receiver, letter headline etc), journalists claim to see copies of the correspondence. The press associations follow up by lobbying the ministries and agencies, trying to influence them to adopt liberal practices in releasing documents.

In this environment, claiming access to data just becomes an extension of an existing practice and tradition, not something entirely new and unproven. Civil servants are used to handling FOIA requests. In theory at least, this should work in the open data community’s favour.

Gaining real access
In our research for the Bergen project, we found that only one third of state agencies inform about data sources on their website homepage. Moreover, it is difficult to say if the information is complete (in most cases it probably isn’t). FOIA or not, with no information about data sources it is exceedingly difficult for potential reusers to locate the data. Hence, even with infrastructure and principles in place, there will still be ways to “hide” data.

This means that the open data community will have to continue lobbying and struggling for access to data. Hopefully the government will come up with incentives for agencies to make data available. But pressure from the community will be essential.

That pressure will have much greater chance for success if there are convincing examples of creative reuse to point to. Here contests such as Nettskap are important – they will speed up the production of good reuse cases. The community can do a lot in this area as well, by highlighting intelligent reuse of data at home and abroad.

In the best case scenario, user groups build “coalitions” with government agencies. There is a potential for this organized around location – for example, journalists, IT consultants, scientists and activists can team up with the city government with the common aim to improve a region’s competitiveness and transparency. Or such coalitions can be organized around interests – such as environment/climate data, where there is both commercial and civil society interest.

A government data champion?
Norway and the other Scandinavian countries are potential open data champions. Progress has been made in several key areas during the last year, as I have tried to show in this article – more progress than could have been anticipated a year ago.

The Norwegian open data community that I have mentioned repeatedly, is in fact still emerging and very fragmented. One reason is that the interest for open data unites people with very different backgrounds – in the IT sector (business development and programming), in universities, in the media, in the public sector agencies.

Hence, an important challenge will be to bring people closer together and start building more efficient coalitions around the common interest of reusing data for different purposes. To accomplish this, the power of the convincing example cannot be overestimated.

Even if this succeeds, the community will still need help. Political leadership is essential, as only a clear sign that the government gives the question of data reuse priority will give the great majority of agencies the incentive to act. Unlike in the US and UK, we are still waiting for open data to be promoted from the highest political level.

This text was originally published as a Topic Report at the European Public Sector Information Platform. Also published in the Open Knowledge Foundation Blog.

Time out. Enjoy the weekend!

Summer soon! Another highlight from the Flickr Commons. Photo: Vern C. Gorst.

• no.ckan.net homepage
• List of data sources

Thanks to help from the fast and efficient CKAN community, the translation could be implemented in a short time. In addition, data packages that we had already registered as part of our project could be imported from a Google spreadsheet. With 136 packages already online at no.ckan.net, we feel we have had a good start.

We of course hope that the Norwegian open data community – both inside and outside government agencies – now will seize the opportunity and engage in registering data sources at no.ckan.net. In addition, we believe that the existence of the website will alert more people to the need for a proper “store” of government data. In fact, the Norwegian minister responsible for IT policy recently stated that work on a Norwegian “data.gov” had begun.

In the context of the international open data community, we certainly aim to share experiences made with no.ckan.net as we move forward.

(First published on the Open Knowledge Foundation blog.)

Photo: Estúdio Horácio Novais/Biblioteca de Arte-Fundação Calouste Gulbenkian

The past few months I have led a fact-finding project about open data in Norway at the University of Bergen’s Department of Information Science and Media Studies. In the first phase of the project, we interviewed and surveyed civil servants at the state, regional and local government levels about their opinions on and interest in making datasets available for re-use. Our first project report (see English summary), presented at a seminar in Bergen in January, is mainly based on this work. Among the findings:

• Information scarcity: Banal, but true – to re-use data you first have to know that they exist. This is a very real hurdle. Few state agencies inform well about data sources on their websites, our research showed. Some have data catalogues where you can download datasets, a couple of them even sport APIs. But the majority offer insufficient, if any information. Clearly, a push is needed here, and one obvious tool would be a Norwegian data.gov. To spark some interest in this, we started to list datasets on an open Google spreadsheet. To date, some 130 datasets have been entered there. We have also suggested that informing about data sources – and making them available – should be added as criterions in a yearly evaluation of quality of public sector websites in Norway. This is a well-publicized event with prestigious awards; hence this could be an effective initative in raising awareness about open data.
• Great potential: In our survey among state agencies, two thirds said that their agency possesses data with potential for re-use that is not utilized today. Open data is on the agenda in many agencies; more than six out of ten said they plan to make more data available for re-use during the coming year. Our research generally indicates that knowledge and interest in opening up data varies widely across agencies and even in different departments of the same agency. In some local agencies, the interviewees had barely heard about publicizing data for re-use, whereas others have worked on making data available for years already.

In theory, the Scandinavian countries are potential open data champions. There is a long and well-established tradition of transparency in government, backed by far-reaching freedom of information legislation. In Norway, the legislation was recently amended to include a paragraph on database information (an adjustment prompted by the EU’s PSI directive). Other legislation, notably on the right to access to environmental information, provide additional tools for open data advocates, journalists and others who want access to public sector data.

In practice, however, open data remains a quite obscure question in the public sphere. The political attention generated by open data debates is nowhere near the levels seen in the US and UK. There are some signs of change, though. Recently, the Ministry of Government Administration has indicated that it will soon launch an Apps for Democracy-like competition. The success of some open data iniatitives such as weather data from the Meteorological Institute and (on a smaller scale) of airline traffic data from the state airport company Avinor, has stirred some interest among state agencies.

On the “demand” side, I think it’s fair to say that the legal tools I mentioned are not very well known and little used when it comes to forcing access to data sources. In contrast, claiming access to text documents is routine among Norwegian journalists, so also in this area the potential is there.

At our university department, we plan to continue our project with a different approach – building applications or services, hopefully in cooperation with Norwegian media. This way, we want to demonstrate how government data can be re-used in ways that stimulate public debate. We also have an ambition to strengthen the development of computational journalism. If we succeed in this, we can give a small contribution to what must be a long term goal for the open data community – moving from raw data to real insight.

(First published on the Open Knowledge Foundation blog.)

After some researching, I have only little information about Vivian. Central Camera (110 yr old camera shop in Chicago) has encountered Vivian from time to time when she would purchase film while out on the Chicago streets. From what they knew of her, they say she was a very “keep your distance from me” type of person but was also outspoken. She loved foreign films and didn’t care much for American films.

The Independent has also written about her.

Thanks to document.no for the tip.