Google search today has, in my opinion, degenerated roughly to the point of keyword search a decade ago. Most searches, particularly those with commercial intent, have been search-engine-optimized, spammed, link-farmed, or content-farmed to the point of uselessness.
As Michael Arrington succinctly put it: Search Still Sucks. I’d actually quibble with the “still” — it’s taken a decade of cat-and-mouse to make Google as bad today as AltaVista was in 2000.
One of the many reasons search has degenerated is link-buying. One of the benefits of running a blog is that you get to see tactics like link-buying and comment spam first-hand. In this post, I thought I’d share that first-hand look.
Here is an email I received today which is an example of link buying.
That’s it. If you write a post and link to my client, I’ll pay you. It can’t be easy for Google to algorithmically figure out which links I’ve put in naturally and which ones I’ve been paid to insert. It’s not obviously even possible, though getting close probably is. But it can’t be easy.
For comment spam, here is what the comment dashboard looks like in my blog, which is powered by WordPress.
Since Google is all about inbound links, comment spammers either load their comments up with links (see last entry above) or enter a seemingly innocuous text comment with a blog/web address that is the link they’re promoting (see Minh’s entry).
The amazing thing about comment spam is the volume. My blog has had 4600 spam comments in the past 60 days. While I believe these are much easier to detect than purchased links — particularly for the blogging platform if not the search engine — the volume is certainly impressive. Note that since WordPress bundles Akismet all of these spam comments were picked off before Google had to deal with them. But I’m sure for plenty of blogs that’s not the case.
If you look at the history of search and spam, it’s pretty simple:
Phase 1: keyword frequency. Rank pages by the TF/IDF of search keywords. Spammers then quickly discover how to load pages and/or tags with keywords to inflate their rank.
Phase 2: inbound link frequency and authority. Rank pages by the number and authority of inbound links. Pages that themselves have lots of inbound links have higher authority than those that don’t. Spammers slowly discover the aforementioned techniques to eventually beat this as well.
I believe the world is strongly in need of a phase 3 approach and I suspect it will involve curation. Consider some more of Arrington’s comments:
Yes, search is very hard. But Silicon Valley is really good at doing hard things. The real problem right now is that there’s a perception that Google is untouchable in search. When a venture capitalist sees a pitch from a new search startup all they can think about is the Cuil debacle. And since venture capitalists are just about the most risk averse people in Silicon Valley, the funds just don’t flow.
But all the evidence suggests otherwise. Demand Media is worth $1.6 billion, and their entire business is based on pushing cheap, useless content into Google to get a few stray links. If Google was good at search, Demand Media wouldn’t exist. And Bing wouldn’t be making solid gains in search market share. And JC Penney wouldn’t be able to massively game search results for a few months, during the holiday season, without getting caught until months later.
We need to see a real competitor emerge in search. If only because it will make Google up its game, and make all of us a lot happier.
This is one reason I’m watching Blekko. While I’m not in love with the way they currently do curation (i.e., slashtags), I do believe that they are focusing on the right core concept. For more information on Blekko, you can read this TechCrunch article to which, I should probably say, I linked by choice and not for profit.
