5 Feb 2012

REVIEW: David Weinberger's 'Too Big to Know'


The title of David Weinberger’s ‘
Too Big to Know’ refers to the overwhelming complexity and volume of information flow of the networked world. (The book has a keen sense of his
tory and Weinberger points out that writers have been complaining about information overload since at least the Romans.) A post-Modernist stance is summed up in the playfully over-long sub-title: ‘Rethinking Knowledge Now the Facts Aren’t the Facts, Experts Are Everywhere, and the Smartest Person in the Room is the Room’. The nub of Weinberger’s argument is that the nature of books (and more generally paper publishing) has defined what we have understood as facts, expertise and knowledge in the three centuries since the Age of Enlightenment. Now that paper publishing is yielding to the Net what we understand as  facts, expertise and knowledge is also changing.

The paradigm shift
The world of books dealt with complexity by restricting access to publishing. Publishers put out a fraction of what was submitted to them, you couldn’t be an expert without being published, most people who had something to contribute were put off by the high hurdles, and the system amounted to a series of filters that kept output to a manageable level for academic journals, libraries and other media. Weinberger adopts a Darwinian tone to explain how it all worked:

Our system of knowledge is a clever adaptation to the fact that our environment is too big to be known by any one person. A species that gets answers and can then stop asking is able to free itself for new inquiries. It will build pyramids and eventually hadron colliders and Oreos. This strategy is perfectly adapted to paper-based knowledge. Books are designed to contain all the information to stop inquiries within the book’s topic.”


But it came at a cost. Good ideas got left behind. Many experts never got published. And the system made it hard to root out false beliefs. Now we’re in the world of networks where there are no such publishing restrictions. Information has broken out of books and is reorganising itself around the new means of sharing knowledge: the shapeless, shifting networks of the web. In the process, the comforting solidity of book-based knowledge is being lost. And in its place comes a form of knowledge in which facts increasingly look like social constructs, the notion of expertise is morphing from an elite concept to something much broader, and becoming knowledgeable appears to require networking skills.

“Knowledge lives not just in libraries and museums and academic journals. It lives not just in the skulls of individuals. Our skulls and institutions are simply not big enough to contain knowledge. Knowledge in now a property of the network, and the network embraces businesses, governments, media, museums, curated collections, and minds in communication.”


Weinberger believes that for the first time we have a medium big enough to carry ‘knowledge’ -- there are no practical limits to how much information the Web can hold. The beauty of the system is that it is infinitely scalable. But it’s messy.

“... the Net can only scale that large because it doesn’t have edges within which knowledge has to squeeze. No edges mean no shape. And no shape means that networked knowledge lacks what we have long taken to be essential to the structure of knowledge: a foundation.”


That’s the essence of why so many of us, as individuals and as members of organisations, find this era unsettling.

For those unconvinced that terms as key to our culture as something like ‘fact’ could undergo suchchange, Weinberger patiently explains how the word has already changed its meaning twice in the past 400 years. I'm the early 17th century, when it first appeared in English, it meant deed. When Bacon developed the scientific method in the 18th century, it meant something like the opposite of theory. And with Bentham and the development of political and social thought it came to mean something along the lines of free from the bias of interested parties.

“Push on a fact hard enough, and you’ll find someone contradicting it. Try to use facts to ground an argument and you’ll find links to those who disagree with you all the way down to the ground. Our new medium of knowledge is shredding our optimism that we could all agree on facts and, having done so, all agree on conclusions.”


Traditional knowledge as an accident of paper
Weinberger accepts that there’s no agreement over what constitutes knowledge. But he proceeds anyway to define three key elements:  
  1. Knowledge is a subset of belief. We believe many things but only some of them are knowledge
  2. Knowledge consists of beliefs we have some good reason to believe, whether it’s because we’ve done experiments, because we’ve proved them logically, or because God revealed them to our people
  3. Knowledge consists of a body of truths that together express the truth of the world.

It’s this third element, under attack for a couple of generations,  that is looking outmoded now as  the Web seals the fate of notions like the idea that “the news” could be fitted into the pages of a newspaper, or that a 65,000-strong collection of articles could properly constitute an encyclopedia.

Five properties of the social web that create new forms of knowledge
Weinberger identifies five new things human beings can do that couldn’t be done before the advent of the social web:

1. Mobilise bigger groups than ever before
In 1994’s ‘The Wisdom of Crowds’ James Surowiecki identified the conditions under which a ‘crowd’ could outperform ‘experts’ -- diversity, independence, decentralisation, and a means to establish a collective decision. The simplest form of this is ‘crowd-sourcing’ -- the manner in which the Guardian mobilised more than 30,000 citizens to ‘Investigate your MP’s expenses’ by  trawling through official documents on what they’d claimed for. (NB Weinberger glosses over the fact that this approach only got half the job done).

As in geo-political events like the Arab Spring, social media is helping larger and larger groups to assemble. They almost seem old-fashioned now but the two most obvious examples of projects requiring many minds to work are Linux and Wikipedia. These just wouldn’t have happened under top-down heirarchical set-ups.

2. Find ‘experts’ at the most granular level
Innocentive, a problem-solving platform created by Eli Lilley to channel external expertise, regularly hosts prize competitions to solve long-standing problems. Wired magazine describes it as ”the research world’s version of iStockphoto.”  Ed Melcarek won $25,000 when he solved Colgate-Palmolive’s challenge of injecting fluoride powder into toothpaste tubes without contaminating the surrounding air. He’s a physicist and worked out straight away that the answer was to ground the tube and add an electronic charge to the powder.

MIT researcher Dr Karim Lakhani studied Innocentive’s competitions and concluded, “the further the problem was from the solver’s expertise, the more likely they were to solve it,” often by applying specialized knowledge or instruments developed for another purpose. He also discovered that participants were motivated by extrinsic benefits. This is one of the standard results of network theory -- what Mark Granovetter originally diescribed as “the strength of weak ties” -- the most efficient networks being those that connect the broadest range of information and expertise.

Weinberger cites Scott Page, author of The Difference, on his view that on and off the Net. “diversity trumps ability’:

“The best problem solvers tend to be similar; therefore, a collection of the best problem solvers performs little better than any one of them individually. A collection of random, but intelligent problem solvers tends to be diverse. This diversity allows them to be collectively better.”


3. Facilitate clusters of expertise
While the Net connects nearly everyone and is incredibly diverse one of its essential features is much denser sub-networks.  YourEncore, a network of former Procter & Gamble employees, mobilises the skills and knowledge of thousands of older workers to solve problems from a variety of organisations via an online community. CompanyCommand.com is a network of West Point graduates, which allows them to swap notes outside the formal power structures of the army. It later spawned the official CALDOL online expertise community. Meanwhile
PatientsLikeMe.com enables patients to share their experiences of symptoms, treatments and responses. But it also takes that data, which is from real experts -- those that are actually experiencing the disease and the treatment -- makes it anonymous and sells it to researchers.

An interesting combination of 2) and 3) is Primary Insight -- a consultancy specialising in connecting those organisations looking for specialist advice with a diverse range of experts.

4. Accumulate and disseminate knowledge in real-time
This is perhaps the most obvious. For the first time you can see networked expertise coming together in real-time. Almost everyone who has downloaded new software will have experienced one dimension of this. New software invariably has bugs -- part of the ‘perpetual beta’ culture of the Web. But almost immediately workarounds are shared by enthusiasts and fans. [Quote from Zittrain?]

5. Scale infinitely
Twitter scales perfectly whether you are using it with a few friends as an intimate communications channel or if you have a million followers and are using it in broadcast mode. The only other medium with this kind of scalability is paper. But paper lacks interactivity. It may not be possible for Ashton Kutcher to converse with his million plus followers but they can talk to one another. Conversation scales horizontally on the web if not vertically.

IBM’s edgy social media strategy includes company-wide jams in which the global workforce swaps notes on an issue of strategic importance. It started with the corporate values statement in 2003 but led on to an ‘innovation jam’ which spawned Smart HealthCare Payment Systems and Intelligent Utility Networks that have become central to the companys ‘Smarter Planet’ initiative.

The smartest person in the room is the room itself
If expertise is more widely distributed than we had realised, and knowledge requires a social dimension before it can be realised, then it follows that the most important factor is the design of the social networking that brings together the diversity of people and mobilises knowledge-sharing.

The many examples cited by Weinberger illustrate the complexity of this -- from the completely open Twitter to the professionals-only companycommand.com. From free Facebook to charged-for Primary Insight. From networks operating across an organisation like IBM’s ‘Jams’ to networks operating across an interest such as patientslikeme. Each is designed to mobilise new sources of human intelligence. And each is configured differently.

‘Too Big to Know’ is an entertaining guide to the new world of networked knowledge, but it’s also a useful reminder of the sheer diversity of network types necessary to meet the challenge that the smartest person in the room be the room itself.