I gave a short talk at lunchtime, and the video is here.  It was intended as a provocation – my own ‘Modest Proposal’, as someone said – so don’t hold me to all of it…

You can see all of the videos too.

One of the throwaway remarks I sometimes make at conferences is that “Google knows you’re pregnant before you do”.

I can say this because the things you search for will change as your life changes, and search engine providers may well be able to spot the significance of these changes because they aggregate data from millions of people.

Now Google’s philanthropic arm, google.org, has shown just what it can do with the data it gathers from us all by offering to predict where ‘flu outbreaks will take place in the USA.

It has found that “certain search terms are good indicators of flu activity”, in that they correlate well with reports from the official Centers for Disease Control and Prevention.

And it claims that “across each of the nine surveillance regions of the United States, we were able to accurately estimate current flu levels one to two weeks faster than published CDC reports”, a result that could save people’s lives by alerting them to have ‘flu vaccinations earlier than they might otherwise have done.

This is a really interesting piece of work and clearly demonstrates the power of data mining. Its potential usefulness is not limited to health matters.

As John Naughton pointed out in The Observer, “everyone I know in business has known for months that the UK is in recession, but it’s only lately that the authorities have been in a position to confirm that – because the official data always lag the current reality.”

Perhaps the answer lies buried somewhere in the queries being made online, with company directors or politicians searching for terms that imply a coming recession, like details of redundancy pay or bankruptcy protection.

It isn’t only Google who can do this of course. Its database of queries is vast and fast-growing, but it is only one among many databases that underpin the modern world.

The banking system is really only a collection of collected databases recording who has which assets, while neither government nor business could operate without complex data stores.

Soon the national ID register will store details of everyone in the UK, while the forthcoming Communications Data Bill is likely to include proposals to create a vast system that will record details of every e-mail sent, every website surfed and every file downloaded.

As we have seen with flu trends, sometimes the “interesting” knowledge that can be extracted is well-concealed until comparisons can be made with other sources, as it was the correlation between some search terms and the real-world data that mattered.

Of course Google has not revealed which search terms it analysed because doing so would undermine the model’s effectiveness.

Unfortunately it is being equally reticent about how it has ensured that the data its uses is properly anonymised so that users cannot be identified on the basis of their queries.

A letter from the Electronic Privacy Information Center (EPIC) and Patient Privacy Rights to Google boss Eric Schmidt has not been answered, leaving those concerned with online privacy uncertain over the broader implications of the project.

But as Cade Metz points out in an insightful article in The Register, we may all be happy to know that a ‘flu outbreak is coming, but what happens when the disease involved is more life-threatening and the government asks Google for the names and IP addresses of anyone whose search terms indicate that they are infected?

It’s not that I don’t trust Google. I don’t trust any company, government department or individual without a good reason to do so.

In the case of search engines that claim to protect my privacy I want to know just how they do it and will not accept vague reassurances.

In the case of governments that want to build vast databases, I want strong legal sanctions against their abuse and full disclosure of the technical details.

Those of us living in the west with access to technology and the network have lived through a revolution in the last decade and a half that is as radical in its impact as the industrial revolution, and it has happened a lot faster.

It is hardly surprising that we do not yet know how to operate in a networked world where amazingly detailed data is routinely stored, processed and made available.

We will need to think in new ways, learn to assess risk according to new criteria, and find ways to hold those who have power over us – whether political, social or cultural – accountable in new ways.

The US writer Curt Monash has written about this topic many times over the years, arguing that since we clearly cannot halt the move towards data capture and use we should put legal and regulatory frameworks in place as a matter of urgency.

We have made a start in Europe with data protection legislation which could be strengthened and reinforced if politicians were willing to make the effort.

But first we need an active press and an engaged population, one that asks hard questions, forces those who want to develop new databases to be accountable and open, and makes the boundaries of acceptable surveillance a matter of public debate.

And perhaps we should ask google.org to start work on “Privacy Trends”, hoping to spot privacy disasters before they happen by looking at searches for “compromised data”, “hacked database” and “lost USB stick”.

FACT is a fabulous building, and there’s  a real buzz in the atmosphere. And since there’s free wifi I’ve been Qiking:

and

Of course, it’s all a great breach of privacy…

• Al Gore Places Infant Son In Rocket To Escape Dying Planet – the only sensible option…
• Reporters sans frontières – Beijing Games 2008 – Advice from RSF
• Test: Silicon Swings and Silicon Roundabouts – read and digest before taking any action
• Cory Doctorow: Filesharing deal will drive swapping underground | Technology | guardian.co.uk – Cory speaks the truth, but underestimates just how painful, protracted and destructive the battle will be.
• Media face web censorship at Beijing Olympics | Sport | guardian.co.uk – Indeed

• Thingology (LibraryThing's ideas blog): Build the Open Shelves Classification –
• 2gether08 | Spirited Away – what next for 2gether – Call for ideas from Steve Moore for all at 2gether08
• Freedom to Tinker » Blog Archive » Could Too Much Transparency Lead to Sunburn? – "As technology evolves, the same public information laws create novel and in some cases previously unimaginable levels of transparency"
• All Nvidia G84 and G86s are bad – The INQUIRER – Nvidia chips seem to have a problem. A big problem

• Lose Your Wikipedia Crutch: 100 Places to Go for Good Answers Online – Distance Degrees.com – Useful list of places to look
• BBC NEWS | Politics | Voters' data 'should not be sold' – Another sensible report to be ignored by the UK government…
• Personal Internet Security: follow-up report – Richard offers a clear-sighted view on the new personal internet security report from the House of Lords Science and Technology committee
• London Games Fringe – be there.

• Slashdot | The Internationalization of Malware – Nice attempt to take a multicultural view of malware
• Why Identi.ca needs to look further afield than open source. Strange Attractor – excellent analysis from Suw, as usual.
• Wendy?s Blog: Legal Tags » Privacy Falls into YouTube?s Data Tar Pit – excellent analysis from Wendy Seltzer of the YouTube data disaster in the making
• Status-Q » Blog Archive » Internet independence – want one

• 2gether no more | DavePress – I?m absolutely convinced that for events, conferences or whatever, doing fun stuff is more important that sitting and listening to people on stages.
• Judge Orders YouTube to Give All User Histories to Viacom | Threat Level from Wired.com – And this is why data retention is a bad idea. Even if we trust Google, who would trust Viacom?
• Chaos Communications Congress 25 call for participation – Boing Boing – Should be fun
• Slashdot | RMS and Clipperz Promoting Freedom In the Cloud – Interesting initiative – wonder if it will get any traction

• Bad science: Suicides, Aids, and a masts campaigner | Comment is free | The Guardian – Surely there's a link to wifi too…
• ICO slaps TfL over Oyster data hoard | The Register – So if you want to get your child's discount you have to hand over their identity details to TfL so they can track them – must not stand.
• Heavyweight physics prof weighs into climate/energy scrap [printer-friendly] | The Register – Interesting (and long) consideration of David MacKay's work in progress
• Charter delays NebuAd rollout after outcry – Over here the relevant authorities don't seem to care what Phorm gets up to, while NebuAd, a comparable system in the US, gets scrutinised by Congress…

These are the notes I spoke from, slightly tidied up.

Good morning

My name is Bill thompson

My NI number is [redacted]

My mum’s maiden name is [redacted]

And my birth date is [redacted]

I’m telling you this because my kids’ child benefit goes to their mum, and I wanted to feel a connection to the other parents here this morning.

The exposure of personal information by HMRC in its interactions with the NAO, and the other cases that have come to light since, may well be a defining moment in the complex dance between the state and the citizen that characterises modern democracy.

And indeed in the dance between the private sector and the customer that characterises digital capitalism.

The specifics of the data involved are less important than the revelation that the offices of state consider my personal information less important than the lunch menu in the treasury.

The contempt that is now being revealed is breathtaking, and the sense of shock can only grow as we realise the inadequacy of any regulatory response – and of course efforts to strengthen the Information Commissioner’s powers which may result are to be welcomed.

The debate now begun will extend to cover the children’s database, the identity database and of course the myriad of private sector databases held by large companies from Tesco to Google.

We can attempt to opt out, of course.

I have here my Tesco Clubcard…

[at which point I took out the card and tore it in two. Not with one deft movement, of course – they’re stiff plastic.  But the gesture was effective].

But we cannot easily opt out entirely. They’ll send me a new card – I’m still on their database, and indeed I’d probably have to go to Richard for a consent order to persuade them to delete my records, just as with Facebook.

I live a lot of my life online, and I expose a great deal of personal information there.

Anyone who cares to ‘friend’ me on Facebook – and yes, it is a verb now – will know that I was in Leicester yesterday, that I’m tired now and that I’m off to Newcastle later today.

This ‘personal’ use of personal data occupies a relatively new zone, subtly different from the ways private companies and the public sector store and use data.

It deserves closer analysis because while businesses and ministries have been collecting and collating data about citizens, subjects and customers for centuries it is only recently that most of us have assembled ‘databases’ – whether christmas card address lists or Google-indexed mailboxes – of information about our friends and family, or published our own structured data on social network sites.
Doing this creates difficulties.

One is the risk of data loss.

I have personal details of a few of you here on this unsecured storage device [waving my iPod Touch around] and there is not even a password to stop them finding Becky’s mobile number. I suspect many of you have the same sort of data storage devices on you right now.

Another is the risk that published data will be used against us.

Unruly Oxford students have been tracked down by the university authorities, a beauty queen in the USA has been blackmailed over supposedly private photos, and employees have been told that their employers may own any profiles or contacts lists they create using work computers.

And recently Facebook users have discovered that the new ‘Beacon’ service is advertising their activities on other websites to Facebook and hence to their networks, although here public pressure seems to have prompted a rethink.

We’ll need to keep a close eye on Mark Zuckerberg, though, since his instincts are clearly all wrong.

A big part of the problem is apparently that we are all giving away too much information that should remain secret, like our date of birth, address and even details of which schools we have attended or where we have worked.

This information should apparently be carefully protected because criminals can use it to fill in applications for credit cards or loans, stealing our identities and causing all sorts of problems.

This seems to be entirely the wrong way around.

I have never kept my birthday secret from my friends, partly because I like to get cards and presents, and I do not see why I should have to keep it secret from my online friends.   If that means that other people can find out about it then the systems that assume my date of birth is somehow ‘secret’ need to adapt, not me.

Some things really are private and should be kept confidential, but modern society relies on data sharing, data mining and data use, and building systems which rely on the secrecy of easily available information seems to be foolhardy, as the report points out.

Partly this is about systems design. Any good computer system will ‘degrade gracefully’ – if it stops working then it will collapse elegantly and not simply fall over and display the Blue Screen of Death.  The systems – public, private, and personal – that acquire, store, process and transmit data must be designed in the same way.
That means extra work on all sides.

It means extra work for those building hardware and software, for those developing systems and those creating regulatory environments.

It will mean more work for the Information Commissioner, but I’m sure he’s up to the task.

And it will mean more work for all of us, too. I’ll start by putting a password on my iPod.

Peter and Niamh have done us all a service today – now we need to ensure that we take full advantage of the opportunity their skills and HMRC’s ineptness offers us.
For me the key recommendation of this excellent report is that we “lead open discussions and debate to help build more secure, effective and appropriate technology for personal information”.

I’d like to see the Information Commissioner’s Office work with the Department for Children, Schools and Families to take this debate into schools as part of a broader IT literacy strategy, so that children grow up aware of these issues and able to take control of their own data destinies.