Without a doubt, the most controversial post I’ve written on this blog was last month’s The Database Tea Party: The NoSQL Movement. I know this both from the comment stream, but also from the volume and tenor of emails I received from friends and colleagues over the past few weeks.
The first thing I learned is that standing in the middle is a great way to get attacked from all sides.
- My database buddies blasted me, treating me like a reckless turncoat: how dare you endorse these BASE people? (Humorous double entendre intended.)
- The NoSQL folks blasted me, generally misunderstanding the protest march metaphor I was using (see below).
- The above-it-all crowd blasted me for oversimplifying the issue, suggesting that I was endorsing simplistic views such as it’s about batch vs. online or it’s about scaling vs. not scaling.
All of which, of course, only confirmed the religious nature of the movement and that was indeed a movement, regardless of whether any given participant identified himself as such.
(Several folks also blasted me for using the Tea Party Movement as a metaphor. Let me clarify that this is not a political blog so I won’t debate politics. I intended the metaphor to cover only concepts like “rebellion” and “grass roots,” both of which I do believe apply to NoSQL.)
First, I’ll clarify the protest march metaphor which was easily the most misunderstood aspect of the post. Let me share my first rule of protests: not everyone is there for the same reason. Some people are there for the stated cause, which I’ll call cause A. But others participate on behalf of a group; their signs will say “Group 1’s for Cause A.” Others are there for cause B which lacks enough support to generate its own march, so they tag along with signs saying “Cause A and Cause B.” If you’ve ever been at a rally, you’ve invariably winced when speakers attempted to hijack the agenda, turning to some personal cause, all while pretending to speak on behalf of the group.
It was this sense of chaos and disorder that I was trying to portray. When I made the list of reasons why I thought people were on the NoSQL march, it was neither to say that I agreed with them or that all people were there for all reasons. I was doing the equivalent of asking protesters on the UC Berkeley Anti-Grenada March why they were there. To which the replies might have been:
- To protest the Grenada invasion (cause A)
- To remind people about [insert group here] rights, all while protesting against Grenada
- To protest about UC budget cuts, all while protesting about something the Government did, which I’m pretty sure was bad.
- Because I go to every march; what’s this one about?
- Because I was at Top Dog when it went by and had nothing better to do
So hopefully, the intent of my NoSQL-reasons-list is now clear. Some folks are there because they don’t want ACID transactions. Some folks are there because they are dealing with Internet scale. Some folks are there because they hate the SQL impedance mismatch. Some folks are there because they’re tired of paying oligopoly prices to Oracle. (I particularly liked the comment that said Oracle was free because they had an enterprise license that most certainly wasn’t, ignoring the possibility that recent enterprise/agency directives to look at open source could result directly from the size of the last Oracle check.)
One irony is that I actually agree with one of the fiercest commenters:
A very interesting write-up with one little oversight: you’re wrong.
I am part of a large program to write a NoSQL database for military applications. [It’s not about …] It’s [about] the fact that RDBMSs are built in a different space in the CAP trades.
Google, Amazon, Facebook, and DARPA all recognized that when you scale systems large enough, you can never put enough iron in one place to get the job done (and you wouldn’t want to, to prevent a single point of failure). Once you accept that you have a distributed system, you need to give up consistency or availability, which the fundamental transactionality of traditional RDBMSs cannot abide. Based on the realization that something fundamentally different needed to be built, a lot of very smart people tackled the problem in a variety of different ways, making different trades along the way. […]
So – the NoSQL databases are a pragmatic response to growing scale of databases and the falling prices of commodity hardware. It’s not a noble counterculture movement (although it does attract the sort that have a great deal of mental flexibility), it’s just a way to get business done cheaper.
To respond to the commenter:
- Thank you for the clear definition of why you moved to NoSQL.
- Your comment was picked up by the Otaku blog in an post called NoSQL Explained Correctly (Finally), so congrats and I’m glad I could help facilitate “the conversation.”
- Disorder and chaos is what I was trying to portray in the protest march metaphor, not hippies or nobility
- You are clearly using NoSQL for two of the reasons on my list: scalability and bloatware (i.e., perhaps not the best word choice, but the idea was undesired, included functionality — e.g., ACID transactions)
- You did exactly what I said to do: consider all alternatives and do what’s right for your business
- So, why are we disagreeing again?
I think some people didn’t like my putting “coolness” on the table as a factor and the notion of a “movement.” I believe those are both very real and ironically those who disagreed with me loudest were effectively screaming: it’s not a movement and I’m not doing it to be cool; I’m doing it because it’s right for my business. If so, great. But why does it hit such a nerve?
In the end, when it comes to NoSQL I am trying to:
- Provide an overview of why I think people are considering and/or using NoSQL solutions
- Provide good background references and readings (see bottom of my first post)
- Remind mangers to keep an eye out for the “bad reasons” to go NoSQL — i.e,. coolness and Google wannabeism
- Remind people not to confuse NoSQL with NoDatabase. Special-purpose databases (e.g., MarkLogic) are optimized for specific applications (e.g., semi-structured data) and handle them far better than a general-purpose RDBMS. So in your haste to move off Oracle, don’t advance directly to an open source key-value store; there might be alternative DBMSs that meet your needs more effectively.
- Remind people not to confuse NoSQL with NoCommercialSoftware. While people seem to dislike when I say it, the RDBMS market is an oligopoly and the big vendors’ pricing, margins, and heavy-handed customer relationships are all consistent with that market structure. But you can find other classes of commercial software where the vendors are hungrier and more customer centric.