Wisdom of Crowds

James Surowiecki, "The Wisdom of Crowds" (Anchor Books, 2005)

This book shows dramatic examples of how averaging the results of a large number of individuals gives more accurate numerical answers than experts. Frances Galton who is known for his measuring of IQ looked at the average guess for the weight of an ox after it was "slaughtered and dressed." The crowd, including many who had no knowledge of butchering, entered a contest where they entered the number of pounds for the ox. The average was 1,197 pounds and the actual weight was 1,198 pounds.

And Mr. Surowiecki said that to harness "the wisdom of crowds," the individuals should make their conclusions independently and we need a way to aggregate or combine them. Often this is a simple average.

More practically, in 1968, the United States Navy submarine Scorpion disappeared. There was little information and many plausible guesses as to what caused it to sink. He assembled a group of experts. They individually bet on relevant information such as why the submarine crashed, it speed as it headed to the ocean bottom, and its rate of descent. He combined all these bets and came up with a collective estimate of where the submarine would have hit bottom. But these were guesses, perhaps slightly educated guesses. That estimate was 660 feet from accuracy.

Then the book went to a less useful scenario, "Who wants to be a millionaire" and here the contestant could choose between asking a person that the contestsant said was "the smartest person they knew." Or the contestant could poll the studio audience. The latter knew the correct answer 91% of the time wheile the "expert" was only right 65 per cent of the time.

Other experiments showed the same thing, with the classic of guessing the number of jelly beans in a jar, the temperature in the room, ranking items by weight. Again, the groups were very accurate and more accurarte than individual.

We all know of the Random Walk theory of finance. The estimate of stocks given by the many traders in the market is better than any of the traders. For example, ninety percent of mutual fund managers underperformed the Wilshire 5000 index and 95 percent of bond traders underperfomred the bond market.

After the Challenger Explosion, there were four firms that could have been rsponsible: Rockwell International, Lockheed, Martin Marietta and Morton Thiokol. At the end of the day, the latter was down twelve per cent. The remainder were down three per cent. At this point, noone could really tell which company was the one liable. We know now that Morton Thiokol was the party responsible for the guilty O-Rings.

The theory that Mr. Surowiecki holds is that bad information cancels out and the real information surfaces if one aggregates the result of a crowd.

James Surowiecki, and more importantly myself, speculate on how to use the "wisdom of the crowd" to improve democracy and he has a later chapter on Peter Fishkin's Deliberative Democracy. I will talk more about these in later posts.