—-

The Cloud in Action

Big Data, Cloud Computing and Industry Perspectives with Dave Kellogg

BY Darren Cunningham

I had the pleasure of working with Dave Kellogg early in my marketing career and continue to learn from him as a regular subscriber to his popular blog, Kellblog. A seasoned Silicon Valley executive, Dave has been a board member (Aster Data), CEO (MarkLogic), CMO (Business Objects) and VP of Marketing (Versant and Ingres). I recently sat down with Dave to discuss industry trends. As always, he didn’t hold back.

Dave, you’ve written a lot about “Big Data” on your blog. Why is it such a hot topic in the world of data management?

First I think Big Data is a hot topic because it represents the first time in about 30 years that people are rethinking databases. Literally, since about 1980 people haven’t had to think much about databases. If you were an SMB, you went SQL server; if you were enterprise, you’d go Oracle or IBM depending on your enterprise preferences. But in terms of technology, to paraphrase Henry Ford: any color you want, as long it’s relational.

Overall, I think Big Data is hot for three reasons:

• Major new innovation is finally happening with databases for the first time in three decades.
• Hardware architectures have changed — people want to scale horizontally like Google.
• We are experiencing a serious explosion in the amount of data people are analyzing and managing. Machine-generated data, the exhaust of the Web, is driving a lot of it.

I think Big Data is challenging on many fronts from the cool (e.g., analytics and query optimization), to the practical (e.g., horizontal scaling), to the mundane (e.g., backup and recovery).

What’s the intersection with Cloud Computing?

I think when people say cloud computing, they mean one of several things:

• SaaS: The use of software applications or platforms as services.
• Dynamic scaling: My favorite example of this is Britain’s Got Talent, which uses Cassandra. Most of the time they have nothing to do. Then one night half the country is trying to vote for their favorite contestants.
• Service orientation: The ability to weave together applications by calling various cloud services — in effect using a series of cloud services as a platform on which to build applications.

I think Big Data intersects with cloud in several ways. First, the people running cloud services are dealing with Big Data problems. They are hosting thousands of customers’ databases and generating log records from hundreds of thousands of users. I also think Big Data analytics are very dynamic loads. One minute you want nothing, then suddenly you need to throw 100 servers at a complex problem for several hours.

How do you see these trends changing the role of IT?

I think corporate IT is constantly evolving because smart corporations want their internal resources focused on activities that they can’t buy elsewhere and that generate competitive advantage for the business.

IT used to buy and run computers. Then they used to build and run applications. Then they focused on weaving together packaged applications. Going forward, they will focus on tightly integrating cloud-based services. They will also continue to focus on company-proprietary analytics used to gain competitive advantage.

The other trend driving IT is consumerization. The Web sets expectations for functionality, user interface and quality that corporate IT must meet with internal systems. The bar has gone way up – people won’t tolerate old-school ERP-style interfaces at work when they’re used to Facebook or Yelp.

What does that mean for technology sales and marketing?

If Mr. McGuire in The Graduate were dishing out advice today, instead of saying “plastics,” he’d say “data science.” More and more companies will use data scientists to analyze their business and drive tactical operations. First you need to gather a whole bunch of data about your operations and customers. Then you need to throw world-class data analysts at it to get business value and to be sure you don’t draw false conclusions – e.g., mixing causality with correlation.

Today, most companies have their sales departments on salesforce.com. Leading marketing departments are on Marketo or Eloqua, but most marketers still don’t have much technology backing them. Going forward you will see a whole class of analytics applications vendors providing advanced analytics for Salesforce (e.g., Cloud9, Good Data) and the marketing automation vendors will move beyond lead incubation into providing overall marketing suites. I expect Marekto or Eloqua to try to do for the chief marketing officer what SuccessFactors did for the chief people officer – and if they don’t, then there’s a real opportunity for someone else.

Speaking of all things cloud, you often write about Silicon Valley trends. How would you characterize what’s going on in the market right now?

From my perception, the Silicon Valley innovation engine is running full out. Top VCs are raising new funds. I meet a few new startups every day. Of late, I’ve met fascinating companies in next-generation business intelligence, analytics, Big Data, social media monitoring and exploitation and Web application development. One of the more interesting things I’ve found is a VC fund dedicated to big data - IA Ventures (in New York). When I heard about them, I thought: oh, lots of Big Data infrastructure and platform technologies. Then I spent some time and realized that most of their portfolio is about exploiting new Big Data infrastructure technologies via vertical applications. That was really interesting.

People will debate whether we’re in a mini tech bubble or a social networking-specific bubble. Who knows? I just read an article in the The Wall Street Journal that argues $140B valuation for Facebook is realistic, and it was fairly convincing. So you can debate the bubble issue but you can’t debate that the IPO market has been closed for a long time. Now it is starting to open, and that’s a huge change in Silicon Valley.

Entrepreneurs have historically dreamed of creating $1B independent companies. I’d say for most of the last decade they’ve dreamed of getting bought for 5-10x revenues. Michael Arrington had a great quote a while back saying that “an entire generation of entrepreneurs [has been lost] building dipshit companies that sell to Google for $25M.” I think those days are over. When the IPO window opens, people dream of building stand-alone companies.

What advice do you have for both entrepreneurs and IT veterans?

Don’t build or run things that you can buy or rent. If you follow that mantra, you will follow market trends, and always stay at the right stack-layer to ensure that you are adding value as opposed to leveraging old skill sets. While you may know how to run a Big Data center, you can now rent time in one more cost-effectively. So either go work for a company that runs data centers (e.g., Equinix) if that’s your pleasure, or go leverage the people who do. Put differently, don’t be static. If you’re still using skills you learned 10 years ago, make sure that you’re not teeing yourself up to get left behind.

As always, great advice, Dave! Thank you.

Darren Cunningham is VP of Marketing for Informatica Cloud.

—

[Notes:  Minor changes made from the SandHill post.  I added emphasis via bolding and I corrected the attribution of the famous lines "plastics" from The Graduate.  It was not Mr. Robinson, but Mr. McGuire, who said it.]

• It struck me as unduly and anecdotally negative on the future of information technology. It reeked of a “the party’s over because I’m leaving” mentality.

• Some of it felt like pure spin. When talking about his new venture, C3 (which I think stands for carbon-concious consumer), Siebel seemed to go miles out of his way to position it as an energy technology company, not an information technology company. But, as far as I could tell, it was going to make a software application to help enterprises manage their carbon footprint, kind of an ERP for carbon. Quoting Siebel: “an application to monitor, monetize, and mitigate the carbon footprint.” To me, that’s clearly information technology.

Basically, it felt like his thesis was “IT is dead, long live ET” and any, uh, “facts” that contradicted it were overlooked or spun away. So I was happy to see this piece in the New York Times this morning, Are The Glory Days Long Gone for IT?, which challenges some of Siebel’s claims.

The chart above shows the peril’s of defining the world by one’s own experience. While Siebel characterized 1980-2000 — Siebel’s prime years — as the go-go years of IT, in reality, the fastest period of IT growth was between 1961 and 1971.

Timothy Bresnahan, a Stanford economist, similarly does not accept Mr. Siebel’s contention that the decline in growth rates this decade, which encompasses two recessions, signals a permanent end to I.T.’s record of growing faster than the larger economy. “It is early days to say the game is over,” he said.

When the economy recovers, there is no dearth of unfinished projects for I.T., he said, like “automating white-collar work and automating buying and selling in markets.”

To me, the latter point is critical. Rather than defining the future of the industry by its recent growth rate or by one person’s successes, we should define it in terms of work left to do. And from that perspective, there is plenty of work left and plenty of growth associated with that work.

In my estimation, it’s not that 1980-2000 were the two big decades of IT, they were the two decades of data. The growth that Siebel refers to was all driven by the relational database, the applications layered on it, and the analytics on top of those applications.

One key reason I joined Mark Logic was that, in some way, I agree with Siebel — we are hitting diminishing marginal returns — not on IT overall — but instead on what we can do with data. While we have seen great strides in what we can do with data over the past 30 years, content, by contrast, still lives in the stone ages.

Documents are stored redundantly. They’re not centralized in databases. They’re not re-used. They’re not controlled and managed by applications. They’re not analyzed. Compared to data, content is the Wild West. It’s my belief that this will change, and change radically, over the years to come.

The full story, written by San Jose State business school professor Randall Stross, is here.

• Siebel now runs something called First Virtual Group, a diversified holding company that runs Siebel’s real estate, agribusiness, financial investments, and philanthropic activities. “Think of it as a private equity firm with one investor.”

• He did a long riff contrasting the period from 1980 to 2000 with what he anticipates in the period from 2010 to 2030.

• The 1980 to 2000 period was a paradise of government policies, efficient capital markets, and a free flow of capital to information technology that ultimately created a $1T information technology industry.

• IT growth was 17% CAGR from 1980 to 2000. From 2000 it grows, he says, with GDP at a rate more like 3%. “The party here is over.”

• “It’s done. Tell me the next step that’s a replacement technology. Right now it’s all bells and whistles.”

• The big picture from 2010 to 2030, he says, is (1) increased government regulation, (2) exponential population growth, (3) aging population, (4) increased demand for healthcare (of which 85% of an individual’s lifetime consumption is spent in the last year of life), and (5) energy scarcity.

• It took from 8000 BC to 1750 AD to grow the world population to 1B. In 2008, it’s 6.5B. In 2028 it will be 9B. (Says the guy next to me: “and they’re all going to need to buy things — how is this bad?”)

• These trends make for the following opportunities: (1) food, (2) water, (3) energy, and (4) healthcare. He also mixes in some Malthusian FUD with an overtone of Limits to Growth.

• Per-capita energy population is increasing exponential. So if you combine exponential population growth with exponential per-capita energy consumption, you end up with energy demand that is — pardon the expression — exponential squared.

• All energy on Earth comes from the Sun. (I was waiting for him to add “and it’s burning out” at this point, but he didn’t.)

• He then cited some interesting charts and graphs from a book called Fundamentals of Renewable Energy, which I think is this book though it might be this one.

• He then discussed the concept of peak oil, an idea that I’d heard of but that I hadn’t known was postulated in the 1950s by an engineer at Shell. By 2020/2030, says Siebel, this gets to be a real problem.

• He then said we have two choices: drill / drill / drill or invent / invent / invent. (There are times when I wonder if I shouldn’t have exploited my geophysics degree more.)

• He then discussed two initiatives he’s working on: (1) an Energy-Free Home Challenge that is soon to be formally announced, and (2) a new company called C3 that he was involved in founding.

• The Energy-Free Home Challenge is a contest with $20M in prizes paid for by the Siebel Foundation (2007 annual report here). The goal is to find a way to build houses that consume net zero energy at the end of a year, built with no more expense than traditional construction methods. (This, by the way, sounds definitionally impossible, but who I am to nitpick a billionaire).

• $5M will be awarded to enabling technologies (e.g., glazing, appliances). Ten finalists will be chosen and $2.5M will be awarded to them in prizes and $2.5M will be spent to build their homes. A grand prize of $10M will go to the winner. They will then build 90 additional energy-free homes (using whose money is unclear) in order to demonstrate the viability of an energy-free community.

• C3 (which I think is related to the acronym carbon-conscious consumer) is a new company, run by Siebel veteran Pat House, that will make enterprise software to help companies manage their carbon footprint. The company started by calling together a panel of 29 experts during the summer of 2008. “Deliberations were concluded 12/08.” C3 was founded last month, in 1/09. Operations will begin in 2/09. The product spec will be completed by summer 09. And — if I heard correctly — they will have demonstrable product one quarter later in fall 09. (And one heck of a development team if they can actually build a product in a quarter.)

• C3′s goal is to help companies “monitor, mitigate, and monetize” their carbon footprint. I tried for about 15 minutes to find a website for the firm and failed. If you find one, let me know via a comment and I’ll link it here.

• Almost to the point of comedy Siebel strained to not position C3 as an information technology company, despite the fact that it will sell enterprise software. “C3 is an energy tech company.” “IT is incidental to C3.” “C3 will not have an IT rate of growth.” (How quickly they turn.)

• And the capper: “My closest relation to information technology in the last 4 years has been selling Oracle stock every quarter.”

Overall, it was an interesting speech and I thank Tom for taking the time to give it. I like the idea behind C3, though I think it *is* an information technology company and am not afraid to say so.

I think the Energy-Free Home Challenge is interesting and it begs the question: given $20M, what is the most effective way to stimulate innovation? Contests with prizes? Grants to scientists? Venture capital investments? I don’t know the answer and since it’s not my money, I don’t need to. But I am happy Siebel’s giving it a try and I’m sure that if if produces even a few innovations that the money will have been well spent.

I’d also add that while Siebel didn’t talk about it at the event, that I think what he’s doing with The Meth Project is both quite creative and sadly quite necessary.