I’m a big believer in the power of functional (or wise) groups to make better decisions than even the most talented individuals. I learned this first-hand years ago when I took LDP at the Center for Creative Leadership and we did a survival exercise similar to the one detailed in table 4 of this document. In our exercise, every individual — including a Brigadier General — was outperformed by the group in prioritizing a list of items necessary for wildnerness survival.

So I believe that groups guess jellybean jar counts better than individuals, that PageRank generally works for finding web pages, that feedback (used to) work on eBay (until they said sellers can only say positive things), that Diggs are useful way to identify interesting content, that Wikipedia is a great way to build an encyclopedia (particularly a technology one), and generally most of the other stuff I’m supposed to believe as good, web 2.0, Silicon Valley guy.

I believe this so much that we invited James Surowiecki, author of The Wisdom of Crowds, to keynote our user conference coming soon on May 12-14, 2009. So I’m on board with the program.

But I also wonder about the opposite, what I’ll call The Madness of Mobs. From financial bubbles to looters to Spring Breakers to a dozen other examples, we can all find examples of where everything cuts exactly the opposite way: where a wise crowd transforms to a mad mob.

So I was quite interested to find this article, Swine Flu: Twitter’s Power to Misinform, which talks precisely about how the “mass brain” of Twitter appears to be shorting out when it comes to the topic of swine flu. Excerpt (edited for brevity, and bolding mine):

Thus, Unlike basic internet search — which has been already been used by Google to track flu trends — Twitter has introduced too much noise into the process: as opposed to search requests which are generally motivated only by a desire to learn, too many Twitter conversations about swine flu seem to be motivated by desires to fit in, do what one’s friends do, or simply gain more popularity.

In such situations this, there is some pathological about people wanting to post yet another status update containing the coveted most-searched words – only for the sake of gaining more people to follow them. And yet the bottom line is that tracking the frequency of Twitter mentions of swine flu as a means of predicting anything thus becomes useless. (However, there are plenty of non-Twitter options summed up nicely on Mashable)

Hum. I should probably cop a maybe-guilty plea on blogging on swine flu. Like moths to a flame, we bloggers are drawn to hot topics.

The article continues:

If you think that my concerns about context are overblown, here are just a few status updates from random Twitter users:

I’m concerned about the swine flu outbreak in us and mexico could it be germ warfare?

In the pandemic Spanish Flu of 1918-19, my Grandfather said bodies were piled like wood in our local town….SWINE FLU = DANGER

Good grief this swine flu thing is getting serious. 8/9 specimens tested were prelim positive in NYC. so that’s Tx, Mexico and now Nyc.

Be careful of the swine flu!!!! (may lead to global epidemic) Outbreak in Mexico. 62 deaths so far!! Don’t eat pork from Mexico!!

Swine flu? Wow. All that pork infecting people….beef and chicken have always been meats of choice

Be careful…Swine Flu is not only in Mexico now. 8 cases in the States. Pig = Don’t eat

If my reading list on Twitter was only restricted to the individuals who had produced the posts above, by now I would be extremely scared … In moments like this, one is tempted to lament the death of broadcasting, for it seems that the information from expert sources should probably be prioritized over everything else.

Now, I’m pretty sure the counter-arguments to The Madness of Mobs goes like this:

• Not all groups are wise. The Wisdom of Crowds relies of wise groups.

• You can’t cherry-pick the scariest contributions to argue that The Wisdom of Crowds doesn’t work. Much as the abortion page on Wikipedia is the result of a rugby scrum of passionate, oppositional forces, so will be the mass brain of Twitter on swine flu. You need to look at the whole picture.

In fact, Surowiecki outlines failures of crowd intelligence and finds root causes which include groups that are too homogeneous, too emotional, too centralized, too divided, and too imitative.

Hopefully, we’ll hear more from Jim on this topic at the user conference and, in the meantime, before enslaving yourself to The Wisdom of Crowds, ponder if your crowd is a wise one, and whether you’re actually dealing with The Madness of Mobs.

Related Information / Stories

• Follow the CDC on Twitter. (A nice government 2.0 example, though they might have picked a better username.)
• Google Helps to Track Swine Flu, Twitter Catches Criticism
• Twitter Catches Swine Flu, But Don’t Shoot the Messenger
• Trying to Track Swine Flue Across Cities in Realtime (O’Reilly Radar)

Swine Flu Tracker Map

View H1N1 Swine Flu in a larger map

Here’s this year’s version of her speech. My favorite slide is #15, which does a simple least-squares regression of GDP growth vs. ad spending growth. For more commentary, see this post on Silicon Alley Insider or see this post on the ReadWriteWeb.

In many ways, I think it’s true.

• Many of the previously-unconstrained-by-revenue web 2.0 startups are in for a reality check.

• I attended the now-famous Sequoia “Rest in Peace Good Times” meeting and indeed sat in the front row. While I concur with Sequoia that this is a big structural problem that will take years to unwind, I think it will impact different companies in different ways. (And you have to love the Whiner Jenkins parody in response.)

• Heck, even the father of Web 2.0, Tim O’Reilly, urged startups to get serious and stop delivering silly software.

• Because web 2.0 startups were less capital-intensive, it seems that there are way more of them. And, oh, are some of the names painful. I won’t miss Weebly, Zlio, and Yoono.

• Heck, there was even the famous web 2.0 bubble video by The Richter Scales, a sure sign that things were coming to a head.

However, in many ways, I think the Web 2.0ver assertion is not true at all. In fact, it almost misses the point. While a swarm of eyeball-catching, oddly-named, twenty-something-led startups may get obliterated, that wasn’t the point of web 2.0 (outside venture circles, at least). To me, web 2.0 was, is, and will remain, an important collection of concepts that will endure:

• A read/write web, where we can participate, update, annotate, comment, link, tag, etc
• A social web, where there is awareness of relationships that can be leveraged appropriately
• User-generated content, which is here to stay and, in fact, always has been (think: radio call-in shows, Kids Say the Darndest Things, or America’s Funniest Home Videos)
• The use of the web for communication and entertainment. People are natural communicators. We will always adapt our tools to that fundamental need.
• A personalized web, that understands what we like and how we like to get it

These concepts — and others — came with web 2.0, and perhaps despite the illness of the hosts who brought them, they are most certainly not web 2.0ver.

What are you creating?

How will you make money?

What tools will you use?

(I’d love to see XQuery in the toolset, but happy to see that database, server, and XML are already there.)

And who says you can’t do interesting analytics on content? I thought this was fascinating. Check out Seedcamp’s blog post about the exercise, here.

You can listen to the interview here. Because the sound quality isn’t great, I suggest listening to the interview while reading along with the full transcript here.

To me the themes remain the same:

• It’s about a machine processable web as opposed to simply a human readable one
• It’s about structuring data from pages so it can be used by programs
• It’s then about integrating data from across multiple sites and/or inferencing across information from one or more sites
• He’s a big believer that people should publish information (e.g., a catalog) in both HTML format for human viewing and RDF format for machine processing
• RDF is all about triples, which go something like: object-1 property object-2 (e.g., Dave is-brother-of Fred, Dave is-son-of Judy).
• Creating new knowledge then involves inferencing using these triples (e.g., knowing the two triples above you can induce that Fred is-son-of Judy)

Note that the whole “social graph” captured by Facebook or LinkedIn could easily be dynamically recreated if everyone had some universal profile that listed a bunch of friend-of-a-friend triples (Dave is-a-friend-of Tim, Dave is-a-friend-of Joe, …)

When I think “semantic web” I think about three things:

• The concept: to make the web machine interpretable as opposed to simply machine deliverable.

• A set of core technologies (e.g., RDF, OWL, SPARQL) which by and large have not taken off in the market.

• Inferencing to create new information from the web itself. For example, if site A says that Bandit Kellogg is a Bernese Mountain Dog and site B says that Bernese Mountain Dogs eat socks, then you can induce that fact that Bandit Kellogg eats socks (which he does, voraciously).

Perhaps mine is a traditional or outdated view of the semantic web vision, but I’m not sure. See this post by Nitin Karandikar entitled The Promise of the Semantic Web for his take, which is similar to mine. (The post is based on an interview with Nova Spivack, CEO of Radar Networks, makers of Twine.)

When I look at Twine, I see more Web 2.0 technologies

• Tagging
• Collaboration
• Wikis
• Diggs

Yes, it appears that Twine does automatic entity extraction (which they call Smart Tags) against things that are bookmarked. And they say they use a bunch of semantic web technologies inside the system to figure out relationships between the tags and between people and tags. Excerpt from the Read/WriteWeb:

Where Twine is differentiated from the likes of wikipedia is that its underlying data structure is entirely Semantic Web. Spivack told me that the following Semantic Web technologies are being used: RDF, OWL, SPARQL, XSL. Also he said that they plan to use GRDDL in the near future. Spivack had an interesting term for what Twine is doing with Semantic Web technologies, riffing off the Facebook Social Graph. Spivack is calling Twine a “Semantic Graph”, which he says will map relationships to both people and topics. So Twine’s Semantic Graph actually integrates the Social Graph.

I’d like offer more commentary on Twine but it seems their newfound popularity is impacting their website — I couldn’t successfully register for the invite-only Beta. When I clicked “register” it just hung forever. I’ll try again in a few weeks and if it works and if I get invited to the Beta, I’ll share some first-hand feedback.

Meantime, see the two posts cited above or check out Nitin Karandikar’s email interview with Nova Spivack on his Software Abstractions blog.

The voting was originally supposed to be done using Mozes, a text messaging platform run by a fellow I recently met, named Dorrian Porter. But — believe it or not — they actually manage to hold the Web 2.0 Summit in a dungeon of a room whose thick walls block all cellphone coverage, so instead of using Mozes, we used audience noise-making for the voting instead.

The six finalists invited to present were:

• Cleverset
• G.ho.st
• Realius
• ClickForensics
• Tripit
• Spiceworks

The audience vote awarded best-in-show to Cleverset, a personalization company. My personal favorite was Spiceworks, a “free,” ad-supported application targeted at IT in small and medium businesses. The company says that half of all IT spending is done by small and medium companies, the challenges these jack-of-all-trades IT managers face are significant, and while they typically don’t have much staff, they do influence quite a bit of purchasing and should be both an attractive and otherwise hard-to-reach target for advertisers.

The service I’m most likely to try personally is Tripit. While I don’t know if I’d invest in Tripit, I do think its a useful little app that helps you integrate your travel agenda.

“You’re just 3 years old, and we’ve got you in there playing basketball with a 12-year-old,” Ballmer gushed and gesticulated, nearly popping out of his seat. “You’re growing up quick and getting better every day, and you’ve got all the potential in world, and it may take you ’til you’re 7, 8, 9 or 10, but you’re gonna dunk and you’re gonna dunk on the other guy some day, Johnny.”

But the surprise for me thus far has been Evan Williams of Twitter who, in a very brief presentation, had quite a bit to say on the merits of definition by removal and focus, instead of addition.

• I want to discuss learning by the aggressive application of constraints

• Our decision to use SMS as a messaging vehicle in Twitter meant that we had to support short (140 character) format-less messages. This was a huge constraint.

• We didn’t define Twitter as Blogger less comments, tags, template editors, titles, etc. But it is an interesting way to look at it. (We defined it as a ubiquitous friend status network.)

• What else can we define by taking away?

• In my prior life at Blogger, I spent most of my time trying to add things.

• What would happen if you had Flickr without tags and with a one photo/day limit? You’d get higher quality photos and more and better comments on them … and you’d be Fotolog which recently sold for $90M.

• What would happen if you had Blogger without titles, tags, comments, and a 140-character limit? You’d get Twitter.

• What would happen if you had Yahoo! without the home page and just a blank screen, with a search box? You’d get Google.

• What would happen if you had MySpace but you could only use it if you were in college? You’d get Facebook.

There’s fantastic stuff in here. I’d take the time to read it. You can download the PDF via SlideShare, or go get it directly at www.morganstanley.com/techresearch.

• Pay $3.2K to attend a conference and not be able to find a seat (and then be encouraged to go upstairs to watch the conference on a video feed in the “overflow” room)

• Sit down and hear the guy behind you say something like ” … well … I was interviewed by the New York Times … and they asked me if I thought there was another bubble … and I wanted to be measured [in my reply] …” (It turned out to be Roelof Botha from Sequoia Capital who, among other things, led the partnership’s investment in YouTube and is quoted in the the article I blogged about here.)

• Sit down and have the guy sitting to you next doing a real-time “Twittercast” of the conference. (Rafe Needelman from Cnet.) I’d quaintly inquired if he was doing real-time blogging. How passé!

• Find an event where a moderately disheveled, professorial book publisher is regarded as an icon, seemingly trailed by groupies? (Tim O’Reilly)

• Have such a weird agglomeration of the following people: venture capitalists, founders, startup-up CEOs, PR handlers, journalists, photographers, old media executives, alpha geeks, and a smattering of large company executives.