DBMS in the Cloud: Amazon SimpleDB

Continuing to steadily and patiently execute on their Amazon Web Services vision, Amazon recently announced SimpleDB, a web service for running queries in real time against structured data.

It’s the first instance of which I’m aware of someone offering DBMS-level services in the cloud. Arguably, GoogleBase is a competitor, but I’ve always viewed that as more aimed at eBay and Craigslist and less about cloud computing.

While most SaaS-type applications are indeed applications (e.g., NetSuite, Salesforce), Amazon has been coming at cloud computing from an infrastructure-up, rather than an application-down, perspective. Previously Amazon launched lower-level services including EC2 (elastic compute cloud) and S3 (simple storage service) in the same “pay as you go to use our infrastructure” manner.

I’m told Amazon got into cloud computing because, due to the spikey nature of retail, they have built a massive infrastructure to handle demand peaks (e.g., Christmas) that goes largely unused most of the time. AWS is their attempt to monetize it.

For more on SimpleDB, see this post on the ProgrammableWeb blog, or check out the developer’s guide here.

Finally, here’s the pricing for SimpleDB:

Machine Utilization – $0.14 per Amazon SimpleDB Machine Hour consumed (normalized to the hourly capacity of a circa 2007 1.7 GHz Xeon processor).

Data Transfer

    $0.10 per GB – all data transfer in
    $0.18 per GB – first 10 TB / month data transfer out
    $0.16 per GB – next 40 TB / month data transfer out
    $0.13 per GB – data transfer out / month over 50 TB

Structured Data Storage – $1.50 per GB-month

The Demise of Closed-Source RDBMSs?

A friend pointed me to this interesting post by Allan Packer of Sun entitled Are Proprietary Databases Doomed? Overall, I think it’s a well done analysis of the DBMS market and well worth reading.

First, a nit. When I was a lad, “proprietary” didn’t mean “closed source“, it meant proprietary (i.e., vendor controlled) interface. For example, Ingres originally spoke a query language called Quel. SQL then emerged as the standard and any DBMS that spoke a language other than ANSI standard SQL was deemed proprietary. While I know that some people in the open source community view the opposite of “open source” as “proprietary,” I think that’s a misnomer. I think the correct antonym is closed source.

First, I think Allan makes an excellent point about stagnation:

By the turn of the millenium, relational databases had already pretty much met the essential requirements of end users, and proprietary database companies were either pointing their vaccuum cleaners toward other interesting money piles, or losing the plot entirely and sailing off the edge of the world. Today, database releases continue to tout new features, but they’re frosting on the cake rather than essentials. No-one issues a tender for a database unless they have unusual requirements. No-one loses their job because they chose the wrong database. And it’s been that way for years.

As a general rule I am shocked by the lack of innovation returned by the R&D budgets of most technology companies. As I mentioned yesterday, despite billions of R&D investment, Google has yet to come up with another big business. And what does Microsoft get for the billions they spend each year on R&D? An incompatible version of Office with irritating “ribbons” that takes four years to make.

Silicon Valley startups create new categories with $10s of millions in venture capital. It seems that once they become “real companies” they forget how to innovate at all, let alone on a shoestring.

Specifically in the DBMS market, I think the lack of innovation — enabled by the oligopolistic structure of the market — creates a soft underbelly for focused, innovative companies to carve our niches. (And remember “niches” of $10B market can be pretty big.)

Allan goes on to do some interesting pricing analysis, and then poses the question:

Why, then, is proprietary database software becoming more expensive while everything else reduces in price? End users normally expect to benefit from the cost savings resulting from improvements in technology. I am writing this blog, for example, on an affordable computer that would easily outperform expensive commercial systems from just 10 years ago.

It seems difficult to resist the conclusion that proprietary database companies have managed to redirect a good chunk of these savings away from end users and into their own coffers. Successful as this strategy has been, though, it could ultimately backfire. The more expensive proprietary databases become, the more attractive lower cost alternatives appear.

I think the short answer to his question is (1) the market is an oligopoly and (2) there is a lot of inertia when it comes to database management systems. So change will happen, but it will happen slowly. And, ironically, the force that drives the market change will be overpricing on the leaders’ part. Were RDBMSs not so expensive, there would be less impetus to move to open source.

Now, the RDBMS vendors probably argue they should “milk” the market until the real threat emerges and then “wave a wand” to reduce price, but that is a risky strategy because they could very easily wave the wand too late, which is what I think they are doing.

The only point I think Alan misses in his analysis is that some powerful vendors like SAP and EMC don’t like the fact that their applications run on top of lower-level DBMS technologies from competitors. For example, SAP has been trying to get itself off Oracle for about a decade, and I’m told they fund developers to work on MySQL towards that end. I know that EMC/Documentum is not comfortable that the vendors who provide the DBMSs they run on are all now challenging them in content management (e.g., Oracle/Stellent, IBM/FileNet, Microsoft SharePoint).

He then speculates on what he thinks will happen going forward:

My vote for the Strategy Most Likely To Succeed is a tie between Revenue Pull-Through and Reduce Prices. Oracle is arguably becoming the most successful proponent of the pull-through strategy. Oracle wants to supply you with a full software stack, including an OS, virtualization software, a broad range of middleware, a database, and end user applications. The largest component of Oracle’s revenue currently still comes from database licenses, but the company is working hard to reduce that dependency. Until that happens, reducing prices across the board will be challenging for Oracle. If Oracle succeeds with a pull-through strategy, it doesn’t mean that OSDBs will fail, of course. It simply means that Oracle is less likely to sustain major damage from their success.

He concludes:

Are proprietary databases doomed, then? Not at all. Even if proprietary database companies pull no surprises, they won’t fade away anytime soon … Make no mistake, though, open source databases are coming. For established companies it’s more likely to be an evolution than a revolution.

I believe there are two major trends in the DBMS market today: (1) open-source chipping away at the closed-source oligopoly, and (2) special-purpose DBMSs innovating and carving out niches in the soft underbelly. I actually think point 1 provides powerful “air cover” for vendors pursuing strategy 2, because point 1 is a direct attack on the existing business.

Google as Publisher: The Grassy Knol

On December 13th Google took its first step from organizer and indexer of the world’s knowledge to supporting-creator of it with the announcement of a new free tool called “knol” (a cutesy-ism which stands for unit of knowledge).

The folks at publishing industry watcher Outsell were quick to use the announcement as validation of their predictions that Google would eventually enter the publishing market:

While it was debatable in the past whether or not Google’s actions constituted those of a publisher, there can be no doubt about it today.

Outsell takes a broader view of the announcement than most, who generally see it as a clear, direct shot at Wikipedia. (For an example of the consensus viewpoint, see this Newsfactor story entitled Death Knell Sounds for Wikipedia, About.com. I’d add that lesser known and poorly named Freebase seems squarely in the cross-hairs as well.)

Outsell points out that once a large knol-base (phrase coined by me, you heard it here first!) is created, then Google can tweak its search algorithms to favor its content over competing sites such as Wikipedia, which currently enjoys great organic search rankings; About.com, which doesn’t; and Answers.com which was a casualty of an algorithm change in August, resulting in a 28% traffic drop and a nearly 20% drop in their stock price.

There has been plenty written about knol so I won’t add a deep analysis here. For more, I’d go to the official Google blog post that launched knol and scroll down to see the list of blog postings that refer to it.

Techcrunch has a great write-up here:

Google is moving away from simply indexing the worlds content to being a content provider itself. Of course Google in response would argue that it is simply facilitating user generated content (like with Blogger), that ultimately they are the host as opposed to the creator, but it still competes with existing content providers, many of whom rely on Google search results for their living.

If you thought publishers were uncomfortable partners with Google before, things just got a lot frostier.

To me, despite billions of R&D investment and boatloads of hype, Google remains, as Kris Tuttle at Research 2.0 says, “a one-trick pony (but it’s one darn good trick.)” So no new initiative can be presumed successful simply because Google is behind it. Consider the defunct Google Answers, or the perennially weak comparison shopping service, Google Products, nee Froogle.

Is knol a gimme just because Google’s its dad? No way. The poor choice of name will hinder it as will Wikipedia’s entrenched position, positive karma, and what I sense is a growing Google fatigue in the market.

(Like a boyish 40-year-old suffering from Peter Pan Syndrome, I think Google is increasingly out of touch with its perception. They’re not cute and cuddly techies who everybody loves anymore, so they should stop trying to do cute and cuddly things.)

So, should this make publishers uncomfortable? Yes.

Is it (another) warning shot for the information industry? You betcha.

Do I have three words of advice for publishers regarding Google? Watch your back.

Your Web Search History: Worth a Look

Frankly, I don’t spend much time worrying about my web search history — but I’m starting to wonder if maybe I should.

Sure, I blogged about the topic once here (You Are What You Search). If you’ve never looked at the rather famous anonymized AOL search log of user 672368 then you should, just to give yourself a concrete idea of how much can be revealed by your search history. As John Battelle says, search creates a database of intentions. Looking at 672368’s reveals a lot about hers.

So what does your database of intentions look like?

Well, if you regularly login to Google (e.g., Gmail, Blogger) then Google has been creating your own — hopefully private — web search history. In theory, they’re using it to personalize and improve your web search results. But the questions are:

  • Do you want them keeping this information?
  • What problems would you face if it was accidentally exposed?
  • Or if it was subpoenaed?

Well, there’s no better way to know than go look at yours. If you have a personal web search history on Google, here’s how you can go look at it.

  • Login to your account (click sign-in on the Google.com homepage)
  • Click on web history
  • Start browsing

Enjoy the stroll down search memory lane. And then think about search privacy.

Frankly, after looking at my own and weighing the perceived upside (none, as far I can tell) vs. the possible downside, it wasn’t a difficult decision to turn it off.

Research 2.0 Software Trends Update

Just a quick post to direct you to this interesting presentation, entitled Software Trends Update by Kris Tuttle and Dennis Byron of Research 2.0.

I know Kris from his days at SoundView, and I always found him a particularly astute financial analyst.

I particularly enjoyed this slide, which does a high-level analysis of the top players. I love the line about Google: “still a one-trick pony, but one damn great trick.”