Early in my career, I interviewed at Gartner for a job as an analyst. During the meetings, I met with Mike Braude, then head of research. Mike had made himself famous by declaring the death of IBM’s SAA about two days after it was launched.
I remember when we talked about research quality. “Do you know what great research is, Dave?” he asked.
I can’t remember exactly what I said, but I probably used words like “fair, factual, and customer-centric.”
“Dave, great research is research that sells.”
At the time, I think I saw little horns sprouting from his head as he said it, but today I know what he meant. He assumed research would be fair, factual, and customer-centric. But to be great, it needed more. It needed to have sizzle and controversy. It needed to make you think.
I’m happy to report that the tradition of sizzle and controversy lives on today. While I didn’t attend Gartner’s recent Symposium conference, I have heard a lot about a presentation by Mark Beyer and Donald Feinberg entitled: “The Death of the Database.”
From what I’ve read of the session, here’s the key point: some types of data don’t really need to be in databases, especially in an RFID world.
In fact, this is a question I’d wondered about ever since I first started using databases in the 1980s. The inventory table is a model of a reality. Reality is what’s in the warehouse. And for lots of reasons (e.g., error, theft, or damage) the reality and the model don’t always match.
Companies do a lot of work to minimize discrepancies between reality and the model. They take great pains to ensure that all additions and subtractions are correctly reflected in the database. And because they know that even painstaking processes won’t guarantee synchronization, they periodically audit the inventory and update the database.
You can even get philosophical in thinking about this problem. At Ingres I always wondered about this epistemological question — if my record in the employee database were flagged “terminated,” which was more likely:
- That there was an error in the database. (Reality was right and the model was wrong.)
- That I’d been fired and no one had actually gotten around to telling me.
My guess was the world would increasingly find itself in the second case. When, in effect, does the model become reality?
So if the question you’re asking is “what’s in the warehouse right now” then I agree that it’s infinitely better to just go ask the warehouse via RFID. However, if the question is “what was in the warehouse last week” or “how have inventory levels changed over time” then you’re back in the database business. (And the Gartner guys both address and concede this example.)
I believe they’re really arguing that real-time databases about physical objects need not exist in an RFID world. I think it’s a great point. But what happens in this world is not the death of databases, but the replacement of databases with data warehouses. (The former typically focus on the present and the latter on the past.)
I’m almost giddy that, per this blog, Feinberg and Beyer also apparently said:
- Only 20% of the data that’s stored will be structured anyway
- That XML and XQuery will be useful for accessing the other 80%
- That searching unstructured information will be important
Since the blog doesn’t double-quote them, I can’t be sure they said ‘SQL will take a back seat to XQuery,’ but one can dream. Either way, this is not business-as-usual for Gartner who, just a few years ago, answered most database inquiries with “take DB2, Oracle, or SQL Server and call me in the morning.”
BI was not spared in the presentation, with the analysts arguing that it “wasn’t an application anymore” and would be embedded into operational applications. I both agree and disagree. In my nearly 10 years in BI, I came to believe there were three segments:
- Contextual BI. The use of BI to produce standard reports that enable everyone to develop a common understanding of “what’s normal” and “how things work around here.” This is the most popular use of BI and it’s basically ignored by the market.
- Operational BI. The embedding of intelligence into applications. What’s better — a data mining tool that produces a list of the top 50 leads or a telemarketing application that sorts the leads automatically by sales-value and presents them to the telemarketers in that order? Operational applications will get smarter over time and this will intrude on the traditional BI market.
- Analytical BI. This is heavy lifting with stats tools and data mining. This will remain the domain of the “lab coat crowd” and will remain an important segment of the market.
So I’m happy to see that Gartner is producing some thought-provoking database research, mixing it up, and generating some controversy. But, to paraphrase Twain, I do think the rumors of the database’s death have been at least somewhat exaggerated.
Author’s Notes (1/17/06)
Since the original posting, I have learned the following things via emails from Mark Beyer and Donald Feinberg of Gartner
- The presentation was done by Donald Feinberg and Mark Beyer (not Ted Friedman, as I had originally said). Ted and Daniel Sholler co-contributors to the materials. Apologies for the mistake.
- Mark Beyer says they said that XQuery would challenge and eventually overwhelm SQL.
- Donald Feinberg says that XQuery wouldn’t replace SQL but the two would work together.
I don’t take the last two bullets as inconsistent, but interpret them as meaning that they believe XQuery will gradually increase in uptake over time, be better than SQL for accessing XML data (yes, I’m reaching here), and that two will need to live together for a long time.
