I wrote and published this on my blog in September 2003, over three years ago, and came across it while rebuilding my home page today. I think it is even more relevant now.
In February I wrote a column for BBCi’s News Online in which I called for the creation of ‘OfSearch’, a UK government agency charged with the regulation of Web search engines.
This proposal generated considerable online debate, much of it from US correspondents and little of it positive. As so often in the past, I was chastised for my interventionist socialist proposals and criticised roundly for believing that government and regulation can solve any of the world’s problems.
My column was prompted by Google’s purchase of Pyra, the company that built and runs the Blogger weblogging community. This takeover, which generated a great deal of mostly positive coverage, prompted me to look more closely at how Google operates.
I had written before about how Google seemed willing to self-censor its database without any legal basis for doing so. And I often commented on the lack of effective data protection legislation in the US and the pathetic inadequacy of the EU/US Safe Harbor scheme.
I had also been aware of the rumbling discontent over the ways that Google’s core technology, its Page Rank algorithm, operated, and had noted many occasions where Google was accused of underhand action in changing the ranking algorithm.
The Google Watch website gave me more information about Google, and drew my attention to the ongoing debate about the duration of its cookies (til 2038), the lack of detail in its privacy policy and the absence of any means through which Google users can delete or control the data that has been collected as they search.
I was also reminded that the Google toolbar reports all visited web pages back to Google in order to rank the pages you are visiting. This disturbed me greatly because I use the Toolbar, and was one of the earliest people to download it. As one correspondent informed me, the Toolbar installation process does announce this and give you a chance to opt out, yet when I installed it I had obviously clicked through the relevant screen of privacy information without either noting or registering it.
If I, a journalist, critic and commentator on this area, hadn’t noticed then how could I believe that the other millions of users were aware of what the Toolbar was doing? It was clear that people were not aware of what was happening, and had no mechanism for asking Google to delete the data trail matching their computer to a comprehensive log of all web pages requested,
In the end, I concluded, we could best resolve the difficulties by regulating Google and other search engines as public utilities, just as we regulate power companies, telecommunications service providers and restaurants.
When my modest proposal was published, I was accused of stupidity and worse for daring to suggest that Google, the most blessed of all search engines, could be less than perfect. Nobody, it seems, was to imply that Larry and Sergey and the other nice people who built the company could ever abuse the trust we all place in them when we send them our search terms, accept their cookies or report our web surfing to them.
But still, as Galileo might have said, Google tracks.
For a typical user sitting at their own machine with a fixed IP address and the Toolbar installed to show Page Rank, Google will have somewhere in its vast database a complete record of everything searched for, every link followed and every URL typed directly in to the address bar of their browser.
Google may not know who I am, but it knows that I recently visited Venice, that I fly EasyJet and that I stayed at the Hotel Monte Carlo. It knows that I read Need to Know and The Register. It knows that I tend to visit Bill Thompson’s website rather more frequently than seems necessary.
If it tried – and I accept that it has not done this and has no plans to do this and that the people running Google are decent and honourable – then it could easily map my google identity to my real identity and I would have no way to stop it.
Its use of this data is governed by a self-prescribed privacy policy which could be changed without notice. Abuse of this data would, it is claimed, be punished by ‘the market’ rather than the law, but experience of the US approach to data protection gives us little hope that this is a genuine protection.
I still use Google every day. I’m happy to infringe its trademark by using the term ‘googling’ without the “tm” (I do the hoovering with a Dyson, too). I have the toolbar (although now with page rank disabled) and I love the service it provides. Google is as near to a complete compendium of all human knowledge and opinion as we have yet achieved. I do not know how I could do my job without it.
Which is why it is time to recognise that the time has come to bring this important resource under proper control.
If we reflect on the obvious power which Google has, the degree to which millions of other sites rely on the traffic driven through Google searches, and the complete lack of accountability which the developers of the technology seem to enjoy, then it is obvious that something should be done.
I would argue that all the evidence demonstrates clearly that search sites are now properly public utilities, and that regulation is not only possible but desirable.
The provision of gas, electricity, water and telephony are all controlled, both to ensure that natural monopolies are not abused and to guarantee public safety and utility. Yet to date few aspects of the Internet have attracted the same regulatory attention.
If I change my mobile phone network I am legally entitled to keep my old number, but if I change ISP I am not entitled to retain an email address I may have been using for significantly longer.
Apart from Oftel’s vain attempts to limit BT’s monopoly power, little has been done by governments or regulators to ensure that the Net develops in the wider public interest, and the ‘hidden hand’ of the free market has been allowed to shape the development of all key aspects of the Internet, from core protocols to search engines.
We may debate whether the existing array of organisations involved in the the development of Internet standards and services, from the Internet Society through the IETF, IAB, W3C and even ICANN, are appropriate to guide the network’s future development. But at the service level – when it comes to search – the situation is clearer.
Google – and other search tools and directories – are now so important that their future development can no longer be left to the market. They have reached a degree of maturity that merits proper consideration from the regulatory structures which are a vital aspect of modern industrial society.
While nationalising Google might have been the solution preferred fifty years ago, when steel, airlines and coal were all brought into the public sector, it is no longer the preferred way to do this sort of thing. Instead, we should consider the creation of an agency with regulatory responsibility for search engines. I do not wish to burden OfCom with even wider responsibilities, and so I propose ‘OfSearch.’
It is not the first time we have done this.
The motor car was invented in the 1880’s. In 1904 registration was introduced, despite the opposition of those who were already driving around the countryside and felt they were being branded like convicts. Today drivers are licensed and regulated, cars must be registered, insured and checked for roadworthiness, speed limits are enforced and those who drive in central london must pay a tax for the privilege.
This has not, it may be observed, made car ownership or driving a rare activity carried out only by those who enjoy living in a completely regulated state. And drivers’ licenses are even, I believe, issued in the United States of America, despite the populace’s distaste for such communistic practices as privacy laws, gun control and the Kyoto protocol.
The reasons for regulation are simple:
- The Internet – and especially the Web – has become a vital resource, used by the majority of the UK populationFinding things online relies on a limited number of metasites (search engines, directories, portals)
- The way a metasite operates directly affects the online experience of the user
- Metasites, by definition, are provided with information about the user far wider in scope that individual destination sites
- There is an emerging monopoly in that Google dominates the search space. This monopoly may be abused
Something, as they say, must be done. It took twenty-five years for motor cars to become established to the point where the balance between freedom and some degree of regulation shifted in favour of control. The Internet is twenty years old, the commercial Net about ten and the first serious search engine, Alta Vista, was launched in 1995. It is time to regulate search,
What regulation?
I am not going to try to draft the OfSearch remit here. Instead, I have some suggestions.
To begin with, google.co.uk and other UK-based search engines could be required
- to list all sites without favour,
- to mark any sites that have been removed from its cache on the basis of a cease and desist letter,
- to provide at least four weeks notice of significant changes to the ranking algorithm,
- to provide a more precise mechanism for indicating pages and directories that are not be to indexed or cached,
- and to provide users with a means of inspecting and deleting the data held on them by the operator
They could also be required to disclose details of their algorithms and business practices to the regulator who would be charged with ensuring that they were not anti-competitive and remained fair and reasonable.
All search engines would be regulated, but as the market leader Google would expect to take up most of the regulator’s time.
The Global Internet
A likely objection to this proposal, as to so many proposals to regulate the Net, is that the global scope of the Internet and the lack of effective online borders makes such regulation unenforceable and irrelevant.
This is not the case – even before the adoption of trusted computing platforms allows us to create
these borders. We can, and do, regulated online activity at a national level, even if we accept that enforcement is sometimes difficult.
However such regulation does not have to be heavy-handed. While it would be possible to call on Google US to deny UK users access, just as French courts sought to force Yahoo! to block French citizens’ access to auctions, this would be unreasonably restrictive.
As a commercial company Google will seek to obey the law of those countries within which it operates – look at the deal struck with the Chinese government to see how keen they are to cooperate.
If UK law requires google.co.uk to operate in a certain manner then this will be done. It is then up to individual users to use the regulated Google – and we should hope that regulation improves the service and ensures that more comprehensive and usable results are produced.
BBC and Search
If search is becoming a public utility then we can reasonably ask whether it should be provided as a public service.
One argument sees the provision of Web search by the BBC as an unjustifiable interference in what should be a free market service.
Another plays with language to argue that Web search is no different from the Radio Times offering channel listings for ITV, satellite and cable.
Both are bad arguments made by those seeking particular outcomes.
However we can look at the issue from another viewpoint.
If search is a vital utility in the network age, then it is reasonable for government to promote good, accountable and open search facilities, just as it promotes the provision of clean water, reliable electricity supply, safe gas and competitive telecommunications.
A ‘good’ search engine is not just one which indexes a large proportion of the Web in a way which satisfies user expectations.
It is also one which is accountable to its users, which is amenable to appropriate regulation, which respects the privacy and personal data of its users and which returns results based on objectively assessable criteria rather than for payment or on the basis of factors which are not directly correlated with the relevance of the site to the needs of searchers.
It is hard to see how a commercial site can do this when its primary goal is to make money for its owners. We have already seen, with etoys.com and boo,com, how customer data is traded as an ‘asset’ for a failed company, and even within EU Data Protection Law data-sharing can take place to a degree which would surprise most site users.
However the BBC’s public service remit can easily extend to cover the provision of search, just as it can stretch to cover the production of educational programmes, religious broadcasting and the reporting of Parliament.
Conclusion
Metasites are now important enough to be considered public services just as gas, electricity and water are.
Just as markets are regulated – to ensure no collusion, cartels or price-fixing – for the benefit of consumers, so search should be regulated for the benefit of net users.
Google, as the dominant player, can no longer be allowed to operate purely on the basis of its own commercial interests: the public interest is also important and must now be served too.
The BBC is in a position to be the search provider of choice for the UK public, either by developing its own technology or contracting with one or more existing players to provide a trusted service which respects personal privacy.