Internet Search: The Reality of Link-Buying and Comment Spam

Google search today has, in my opinion, degenerated roughly to the point of keyword search a decade ago.  Most searches, particularly those with commercial intent, have been search-engine-optimized, spammed, link-farmed, or content-farmed to the point of uselessness.

As Michael Arrington succinctly put it:  Search Still Sucks.  I’d actually quibble with the “still” — it’s taken a decade of cat-and-mouse to make Google as bad today as AltaVista was in 2000.

One of the many reasons search has degenerated is link-buying.  One of the benefits of running a blog is that you get to see tactics like link-buying and comment spam first-hand.  In this post, I thought I’d share that first-hand look.

Here is an email I received today which is an example of link buying.

That’s it.  If you write a post and link to my client, I’ll pay you.  It can’t be easy for Google to algorithmically figure out which links I’ve put in naturally and which ones I’ve been paid to insert.  It’s not obviously even possible, though getting close probably is.  But it can’t be easy.

For comment spam, here is what the comment dashboard looks like in my blog, which is powered by WordPress.

Since Google is all about inbound links, comment spammers either load their comments up with links (see last entry above) or enter a seemingly innocuous text comment with a blog/web address that is the link they’re promoting (see Minh’s entry).

The amazing thing about comment spam is the volume.  My blog has had 4600 spam comments in the past 60 days.   While I believe these are much easier to detect than purchased links — particularly for the blogging platform if not the search engine — the volume is certainly impressive.  Note that since WordPress bundles Akismet all of these spam comments were picked off before Google had to deal with them.  But I’m sure for plenty of blogs that’s not the case.

If you look at the history of search and spam, it’s pretty simple:

Phase 1:  keyword frequency.  Rank pages by the TF/IDF of search keywords.   Spammers then quickly discover how to load pages and/or tags with keywords to inflate their rank.

Phase 2:  inbound link frequency and authority.  Rank pages by the number and authority of inbound links.  Pages that themselves have lots of inbound links have higher authority than those that don’t.  Spammers slowly discover the aforementioned techniques to eventually beat this as well.

I believe the world is strongly in need of a phase 3 approach and I suspect it will involve curation.  Consider some more of Arrington’s comments:

Yes, search is very hard. But Silicon Valley is really good at doing hard things. The real problem right now is that there’s a perception that Google is untouchable in search. When a venture capitalist sees a pitch from a new search startup all they can think about is the Cuil debacle. And since venture capitalists are just about the most risk averse people in Silicon Valley, the funds just don’t flow.

But all the evidence suggests otherwise. Demand Media is worth $1.6 billion, and their entire business is based on pushing cheap, useless content into Google to get a few stray links. If Google was good at search, Demand Media wouldn’t exist. And Bing wouldn’t be making solid gains in search market share. And JC Penney wouldn’t be able to massively game search results for a few months, during the holiday season, without getting caught until months later.

We need to see a real competitor emerge in search. If only because it will make Google up its game, and make all of us a lot happier.

This is one reason I’m watching Blekko.  While I’m not in love with the way they currently do curation (i.e., slashtags), I do believe that they are focusing on the right core concept.  For more information on Blekko, you can read this TechCrunch article to which, I should probably say, I linked by choice and not for profit.

7 responses to “Internet Search: The Reality of Link-Buying and Comment Spam

  1. Unfortunately, despite Google generally being pretty good and Bing getting slightly better, search still sucks in a lot of ways for actually ranking stuff. Google’s Page Rank is the best thing so far, but I think there will be other innovations in other realms that will help out with finding stuff. As social networks mature and more and more people are integrated with these things in their daily lives, I think you’re going to see more and more of an emphasis on all kinds of recommendations and searches on stuff like Twitter and Facebook. May marketers that would have previously tried link-buying campaigns are now trying to buy Facebook fans at PopularFans and other similar sites. I think that social recommendations will be a lot more important in the future compared to traditional search engines and Google needs to come up with something soon because Facebook is going to blow by them within a short time period at their current rate of momentum.

  2. I generally agree. Though it doesn’t necessarily have to be Google who comes up with the next innovation. I do believe Facebook might. I do believe it will involve curation at some level, but even curation is gameable, particularly with low-cost labor available thru crowdsourcing.

  3. Dave,

    Great content as always. How does Google’s Panda update alter the playing field? I was under the impression that it had significantly impacted SEO-shenanigans.

    • Yes and no. While from the web provider perspective, the most recent was a big deal (I have heard stories of traffic drops of 30%) from my personal consumer viewpoint it didn’t do much. Plus, to my knowledge it was more aimed at “content farms” (e.g., Demand Media) than at link buyers and other link gamers, but I’m not sure. What I can say is that from my personal perspective, not much changed — if you do some of my favorite “junk searches” on things like “nutritional information on ___” or “how do I do ____” you still end up getting a lot of content-farmed stuff (e.g,. eHow) at the top.

  4. How does Google’s Panda update alter the playing field? I was under the impression that it had significantly impacted SEO-shenanigans.

  5. Though it doesn’t necessarily have to be Google who comes up with the next innovation. I do believe Facebook might.

  6. We need to see a real competitor emerge in search. If only because it will make Google up its game, and make all of us a lot happier.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s