The New Gatekeepers, Part 5: The Problem of Crowds

Fifth in the series on The New Gatekeepers.

It’s been over two weeks since the that last part of this series. This gap in time can be partly rationalized by my hoping to build up some anticipation for this next part. We’re going to look at epidemics, cascades and the problem of crowds.

A mini-epidemic of interest a week ago brought in a bunch of new readers: a regular reader in Portugal, Luis Santos, reflected on gatekeepers and announced that he recomendo vivamente this series; Stephen Downes of New Brunswick posted that he agrees with my hypothesis; someone emails the piece to a mailing list; by the end of the day in Los Angeles, Rafat Ali offered it as “worth reading” for the weekend, along with four other pieces. Independently, on Saturday, Seth posted several links, as we have been discussing this series for weeks now; Jay Rosen read Seth’s post and noted the part relevant to him, and not to this. Independently, Walt Crawford, whom I discovered through Seth, devoted five pages (3300 words!) of his 22-page monthly Cites & Insights newsletter to this series and five other Civilities articles. In the last week an additional 220 visitiors have come to the article, versus 70 in the previous 18 days.

Thus, to use Gladwell’s term: an epidemic, though a small one. Douglas Rushkoff, one of my old mentors in this space, had suggested the term Media Virus in his book of ten years ago by that name; in 2001 Seth Godin had gotten around to Unleashing the Idea Virus. Depsite the grisly connotations (and Gladwell does mention actual diseases outbreaks), the viruses and epidemics were generally portrayed as things to be harnessed. The introduction to The Tipping Points signals: “Why is it that some ideas or behaviors or products start epidemics and others don’t? And what can we do to deliberately start and control positive epidemcs of our own?”

Let’s bear in mind the negative epidemics. I’ll use a more neutral term, information cascades, as profferred by James Surowiecki, Gladwell’s equally talented and highly celebrated colleague on the New Yorker. Both writers have excelled at synthesizing research on social psychology into popular writing– Gladwell into narratives articles, Surowiecki into crisp one-page columns on business and finance. Surowiecki assembled his writings into a bestselling book this past year, The Wisdom of Crowds.

It’s a book that many are tempted to simply quote the title without having read the book (I once did that). Surowiecki recognizes the danger, for on page 75 he cautions: “Understanding when decentralization is a recipe for collective wisdom matters because in recent years the fetish for decentralization has sometimes made it seem like the ideal solution for every problem.” The wisdom of crowds refers to the way in which individuals, acting with their own information are able to collectively make intelligent. This is the easy lesson: a good modern example are futures markets, such as the Iowa Electronic Markets, in which people wage real money to predict the outcome of events like U.S. elections. The hard lesson is understanding the precise conditions where the crowds simply fail.

An information cascade is one such pitfall. It’s discussed in the chapter 3, titled “Monkey See, Monkey Do: Imitiation, Information Cascades, and Independents.” Surowiecki presents the research of economists Sushil Bikhchandani, David Hirshleifer, Ivo Welchers, who in the last decade studied decision-making behavior. These researchers had suggested the example of a pair of new restaurants trying to attract for customers: for some reason customers choose one, it looks busy, and others simply follow the “judgment” of these early adopters. Surowiecki lays on top of this the rapid adoption of plank roads to smoothe out wheeled transport in the 19th century. The plank roads really seemed like the solution, so every town invested in them. But the wood did not last as long as was thought; in the end it was a huge expensive mistake. Why did people keep buying into them? “The fundamental problem with an information cascade is that after a certain point it becaomes rational for people to stop paying attention to their own knowledge.”

It is quite natural for humans as social animals to imitate; our own knowledge is often limited, and we trust others to do the right thing, and to make the right judgments. The downside of innate trust is that we are susceptible to manipulation. This is even more pronounced in group dynamics, in which people interact with each other directly. On page 186, Surowiecki summarizes the problem thusly:

On juries, for instances, two-thirds of all foremen—who leads and structures deliberation—are men, and during deliberations men talk far more than women do, even though no has suggested that men as a gender have better insight into questions of guilt or innocence. In groups where the members know each other, status tends to shape speaking patterns, with higher status people talking more and more often than lower-status people. Against, this wouldn’t matter as much if the authority of higher status people was derived from their greater knowledge. But oftentimes it doesn’t. Even when higher status people don’t really know what they’re talking about, they’re more likely to speak.

Surowiecki presents the research evidence to back this up; I can corroberate from my observations of a group of fifty people at the blogging/journalism conference.

The warnings from The Wisdom of Crowds couldn’t be more evident. And neither are they more elusive than in the blogosphere.

That’s all the more unusual since Surowiecki was invited to speak at the O’Reilly Emergency Technologies conference (“Etech”) in March. He gave a talk “Independent Individuals and Wise Crowds, or Is It Possible to Be Too Connected?” As is common these days with these sorts of conferences on information technology, a good number of people sketch out their impressions on their blogs. So we not only can read what was spoken at the conference, but a sense of how people perceived it.

I did a search on Technorati for “information cascades.” I found 40 references to the term (whereas there are 800 to Surowiecki). Several of these referenced Surowiecki’s talk:

Wade Roush transcribed it. (32 people have bookmarked it on Howard Rheingold’s SmartMobs references the transcript and quotes it directly, without reflection. One blogger named “Reemer” cribbed the speech as well, in the end taking down this lesson: “you can be too connected if the connections are the wrong kind, especially if the connections reinforce your knowledge and prejudices.” Of course, by framing the danger as being “too connected,” Surowiecki may have asked for too much from his audiences (in the business technology world, one can never be too rich or too connected.) Perhaps he should have more clearly entitiled the speech “The Problem of Crowds.”

I could find only one person who truly got the point of Surowiecki’s message. That would be Gary Jones, who reviewed the transcribed talk on his blog, mostly to say that he had addressed this in three previous essays. He also raised a question about the context of a particular quote used in the speech, and Surowiecki himself posted to the blog to provide some an explanation. I asked Jones how it was that he was without peer in connecting these ideas. He admitted only to being an “an old farmer with a net connection but no education, credentials, or accomplishments.” In his probing essays, Jones has challenged the celebrated science writers like Jared Diamond and Michael Pollan. In May 2004 essay entitiled Intoxicated Swarms, Jones wrote:

Increasing the speed and scale of individual interactions using ICT [Internet Communications Technology] doesn’t change humans, doesn’t enable new kinds of association. It doesn’t make mobs smarter. If anything, it makes them dumber, the intelligence of the intoxicated swarm declining monotonically with its increase in scale.

After Surowiecki’s book was published that August, Jones observed in Situation Normal:

Perhaps the greatest impediment to improved social structures will be resistance from those dedicated to exploiting information cascades to achieve power and skew social behavior for gain. Activists of all stripes work to develop manipulative skills to cause cascades. They aren’t interested in wisdom or good governance, they just want to make the sale, stampede the herd, win. They don’t seek to inform, they seek to persuade. They don’t value dissent, they demonize dissenters and try to marginalize them.

Jones assigns blame to individuals. This series aims to exonerate them somewhat by finding fault in the technology itself — the architecture of the blogosphere, and the values of those who drive the technology.

When it comes to information cascades, blogs are a big improvement to what came immediately before– email forwarding. With email chains it is virtually impossible for the average person to trace the source; all context is stripped off; there is no way to respond in a way that everyone will see it. So blogs are a step up the food chain of constructive media.

How does the blog technology influence behavior? I had suggested In her September 2000 essay Weblogs: a history and perspective, Rebecca Blood described the “original style” of weblogs: “Their editors present links both to little-known corners of the web and to current news articles they feel are worthy of note.” Most were positive reviews of sites or articles they wanted to promote. On occasion the blogger took the opportunity to demote the piece: “An editor with some expertise in a field might demonstrate the accuracy or inaccuracy of a highlighted article or certain facts therein; provide additional facts he feels are pertinent to the issue at hand; or simply add an opinion or differing viewpoint from the one in the piece he has linked.”

The idea from the original source has now been morphed, or even replaced. How many readers will read the original source– and form their own opinion– before reading the derivative piece? Perhaps this is only natural, since we check a review for a movie or a theater before we go see the movie itself. As I explained in the last section, this reinforces the role of a gatekeeper.

And which will be read by more? Perhaps the shorter, derivative piece, the one in the so-called conversational style; the one which calls for immediate action. This is the values clash I raised in part three. I don’t think this is necessarily a good thing.

Here’s a recent example. In the middle of February, Roll Call ran an article about the upcoming debate about whether the FEC would regulate Internet campaigning as it does other media. Michael Bassik, the Vice President of Interrnet Advertising and contributor to the Personal Democracy Forum, thought it over in a 1200-word rumination on the PDF website. His conclusion showed the good sense open-mindedness:

Although I really haven’t made up my mind, Congress might be right in wanting to monitor coordination and regulate the funding of online communication. Getting the nod from Congress that the Internet is no different than any other medium might actually work to our advantage. It could go a long way in legitimizing our space…

This debate got a little more attention two weeks later when Declan McCullagh interviewed Bradley Smith of the Federal Election Commission for CNet in a story titled The Coming Crackdown on Blogging. (Seth’s response: "Declan McCullagh has a history of hype-filled, yellow, ‘journalism’… [he’s] a dogmatic Libertarian proselytizer.") The story was apparently cited, according to Google, some 3800 times around the web. On PDF, Richard L. Hasen, a professor specializing in election law at Loyola contributed two articles . On the 23rd the FEC’s draft was leaked, and Bassik’s response was initially optomistic. He grew a bit skeptical after reading the report (posting an 1100-word follow-up in the comments), yet got around to saying that “it looks like the FEC is on the right path.” Searching Google for references to the title, only one person cited it.

Bassik’s final word for the National Journal was very positive on the whole experience “It just goes to show that we as a community helped to wake up the FEC.” (Apparently cited 20 times, but one of them was in the highly popular Instapundit).

This week Bassik got a tip that San Francisco’s Board of Supervisors was considering an ordinance which would regulate electioneering through any media. He interpreted this as hinting at a regulation of blogging, and signaled the alarm. His post to PDF announced an immediate commencement of an email campaign to Sophie Maxwell of the Board of Supervisors, and asked bloggers to copy their emails to the forum. This story, by its title, shows 426 references on Google, including the popular conservative bulletin board Free Republic, the liberal IndyMedia site, and many others.

The next day Chris Nolan responded to Bassik on PDF and reminded readers that she was in fact the group’s San Francisco correspondent. She pointed to her own article, describing the background to the legislation, and explained that it had nothing to do with blogging. Still, she asserted, it was poorly written, and ought to be straightened out.

By Sunday this story had reached some of the more influential voices in social media, and was referenced by Dan Gillmor and by Slashdot. The responsible thing for them to do would have been to link directly to the better-reported piece by the Nolan. They did not; they Gillmor would later post on the 5th that Nolan was heading over to City Hall to report on this (though only after an infrequent reader reminded him about). After the Slashdot post people started leaving messages on PDF that they had joined the email campaign (BL Ochman, quoted earlier in this series, made an appearance here to title her letter “Dear Herr Maxwell.” [note correction] If she wanted to accuse the Supervisor of being a Nazi, she should have used Frau). In all, 600-700 emails poured in, mostly from outside San Francisco, Maxwell aide Greg Asay told me. He did feel that the email campaign did encourage them to have a second look at the language about Internet activity (to say nothing of Chris Nolan’s diligent reporting.)

I didn’t have the luxury of time then to see the big picture then: why had Bassik lost his confidence in the power of deliberative discourse. Matthew Hirsch put together a thorough analysis of the whole affair in the San Francisco Bay Guardian News which offfers some clues. Bassik never checked his facts; he also “failed to disclose that a prime target of the ordinance, the California Urban Issues Project” was one of his company’s clients.

Hirsh’s piece, incidentally, was linked to by only 30 websites, according to Google. For every fourteen reactors who promoted the apparent scandal, only one promoted its correction. The McCullagh interview was nineteen to one over the National Journal summary. By sheer luck another test was suggested by PDF Editor Micah Sifry, searching for the terms wendy and finger on Technorati . He found 1300 references by March 25; a month later I see 3300. If I add arrested I only see 386. That’s seven to one. [After note: this datapoint is flawed. Adding any unrelated word would reduce the number of hits.]

I would be very curious to find some more complete research on this, if only that the average person can stop quoting Goebbels on propaganda. Thus here is Garfunkel’s hypothesis: People who blog have a much greater tendency to pass along incomplete quick impressions than balanced analyses written later, by a ratio of greater than seven to one. Or, the blogosphere breeds propoganda better than the corrections. I doubt that any serious person in civil society would be proud of that ratio. And I doubt that the traditional media is anywhere close to that ratio. [After note: again, assuming that this number is valid. I do not have the confidence since I originally wrote this.]

If you’ve followed along from the beginning, you should now be ready to correct a mistake I’ve made. The old gatekeepers and the new gatekeepers are not the same. Both, after all, influence what we watch and read. If we fear that the old gatekeepers can be restricting information, than we should also have reason to fear that the new gatekeepers can be amplifying selective information. [After note: the wording of this last sentence has been significantly changed to give the intended meaning.]

Who do you trust?

Consider the names in this last story: Michael Bassik, Chris Nolan, Dan Gillmor, Michael Hirsh. Who do you trust? I have a lot of respect for Chris, because her writing is absolutely the sharpest there is (she happens to be a career journalist). Curiously, when I listed names in the Online Political Writers four months ago, I hadn’t even heard of Chris (we correspond on occasion). Most people haven’t who should, because people pay attention to those who are cited, and those are cited more are those who write more frequently, and Chris doesn’t write more than twice a day.

What of the the big names I dropped who’ve offered the theoretical groundwork: Malcolm Gladwell, James Surowiecki, Douglas Rushkoff, Seth Godin, Gary Jones. Is Jones, the uneducated farmer, qualified to stand with this group? I would suppose yes, because he writes full essays and demonstrates independent thinking.

And how about the kind folks who have linked to Civilities this past week– Luis Santos, Rafat Ali, Stephen Downes, Seth Finkelstein, Walt Crawford, Cesarra Lamanna. What basis do you have for evaluating their writings? It should be no surprise, that I admire Walt and Seth for the same reasons. Stephen Downes I look forward to reading.

To get a quick measure the veracity of a person or a story you have to track its citations, its influences as I have done in this exercise above. But because of the nature of information cascades, we find ourselves trusting people and stories simply because other people appear to trust them at all. There are enough independent thinkers out there to offer a useful amount of doubt. We just need to find them. And that’s where the new architecture will come in.

Note 5/10: One name I didn’t drop in the last section was Declan McCullagh. Worse, I had mispelled it above as "MacCollough" mangling just about every part of it. Though somehow I managed to get "Surowiecki" right every time. Also Stephen Downes is from New Brunswick, not Vancouver (he gave a talk in Vancouver. I’ve been to Seattle. The Canadian Dollar is now 80 cents. And other non sequitors.)