Category Archives: Gartner

How To Get Your Startup a Halo

How would you like your startup to win deals not only when you win a customer evaluation, but when you tie — and even sometimes when you lose?

That sounds great.  But is it even possible?  Amazingly, yes — but you need have a halo effect working to your advantage.  What is a halo effect?  Per Wikipedia,

The halo effect is a cognitive bias in which an observer’s overall impression of a person, company, brand, or product influences the observer’s feelings and thoughts about that entity’s character or properties

There’s a great, must-read book (The Halo Effect) on the how this and eight other related effects apply in business.  The book is primarily about how the business community makes incorrect attributions about “best practices” in culture, leadership, values, and process that are subsequent to — but were not necessarily drivers of — past performance.

I know two great soundbites that summarize the phenomenon of pseudo-science in business:

  • All great companies have buildings.” Which comes from the (partly discredited) Good To Great that begins with the observation that in their study cohort of top-performing companies that all of them had buildings — and thus that simply looking for commonalities among top-performing companies was not enough; you’d have to look for distinguishing factors between top and average performers.
  • “If Marc Benioff carried a rabbit’s foot, would you?”  Which comes from a this Kellblog post where I make the point that blindly copying the habits of successful people will not replicate their outcome and, with a little help from Theodore Levitt, that while successful practitioners are intimately familiar with their own beliefs and behaviors, that they are almost definitionally ignorant of which ones helped, hindered, or were irrelevant to their own success.

Now that’s all good stuff and if you stop reading right here, you’ll hopefully avoid falling for pseudo-science in business.  That’s important.  But it misses an even bigger point.

Has your company ever won (or lost) a deal because of:

  • Perceived momentum?
  • Analyst placement on a quadrant or other market map?
  • Perceived market leadership?
  • Word of mouth as the “everyone’s using it” or “next thing” choice?
  • Perceived hotness?
  • Vibe at your events or online?
  • A certain feeling or je ne sais quoi that you were more (or less) preferred?
  • Perceived vision?

If yes, you’re seeing halo effects at work.

Halo effects are real.  Halo effects are human nature.  Halo effects are cognitive biases that tip the scales in your favor.  So the smart entrepreneur should be thinking:  how do I get one for my company?  (And the smart customer, how can I avoid being over-influenced by them?  See bottom of post.)

In Silicon Valley, a number of factors drive the creation of halo effects around a company.  Some of these are more controllable than others.  But overall, you should be thinking about how you can best combine these factors into an advantage.

  • Lineage, typically in the form of previous success at a hot company (e.g., Reid Hoffman of PayPal into LinkedIn, Dave Duffield of PeopleSoft into Workday).  The implication here (and a key part of halo effects) is that past success will lead to future success, as it sometimes does.  This one’s hard to control, but ceteris paribus, co-founding (even somewhat ex post facto) a company with an established entrepreneur will definitely help in many ways, including halo effects.
  • Investors, in one of many forms:  (1) VC’s with a strong brand name (e.g., Andreessen Horowitz), (2) specific well known venture capitalists (e.g., Doug Leone), (3) well known individual investors (e.g., Peter Thiel), and to a somewhat lesser extent (4) visible and/or famous angels (e.g., Ashton Kutcher). The implication here is obvious, that the investor’s past success is an indication of your future success.  There’s no doubt that strong investors help build halo effects indirectly through reputation; in cases they can do so directly as well via staff marketing partners designed to promote portfolio companies.
  • Investment.  In recent years, simply raising a huge amount of money has been enough to build a significant halo effect around a company, the implication being that “if they can raise that much money, then there’s got to be a pony in there somewhere.” Think Domo’s $690M or Palantir’s $2.1B.   The media loves these “go big or go home” stories and both media and customers seem to overlook the increased risk associated with staggering burn rates, the waste that having too much capital can lead to, the possibility that the investors represent “dumb money,” and the simple fact that “at scale” these businesses are supposed to be profitable.  Nevertheless, if you have the stomach, the story, and the connections to raise a dumbfounding amount of capital, it can definitely build a halo around your company.  For now, at least.
  • Valuation.  Even as the age of the unicorn starts to wane, it’s undeniable that in recent years, valuation has been a key tool to generate halos around a company.  In days of yore, valuation was a private matter, but as companies discovered they could generate hype around valuation, they started to disclose it, and thus the unicorn phenomenon was born.  As unicorn status became increasingly de rigeur, things got upside-down and companies started trading bad terms (e.g., multiple liquidation preferences, redemption rights) in order to get $1B+ (unicorn) post-money valuations.  That multiplying the price of a preferred share with superior rights by a share count that includes the number of lesser preferred and common shares is a fallacious way to arrive at a company valuation didn’t matter.  While I think valuation as a hype driver may lose some luster as many unicorns are revealed as horses in party hats (e.g., down-round IPOs), it can still be a useful tool.  Just be careful about what you trade to get it.  Don’t sell $100M worth of preferred with a ratcheted 2 moving to 3x liquidation preference — but what if someone would buy just $5M worth on those terms.  Yes, that’s a total hack, but so is the whole idea of multiplying a preferred share price times the number of common shares.  And it’s far less harmful to the company and the common stock.  Find your own middle ground / peace on this issue.
  • Growth and vision.  You’d think that industry watchers would look at a strategy and independently evaluate its merits in terms of driving future growth.  But that’s not how it works.  A key part of halo effects is misattribution of practices and performance.  So if you’ve performed poorly and have an awesome strategy, it will overlooked — and conversely.  Sadly, go-forward strategy is almost always viewed through the lens of past performance, even if that performance were driven by a different strategy or affected positively or negatively by execution issues unrelated to strategy.  A great story isn’t enough if you want to generate a vision halo effect.  You’re going to need to talk about growth numbers to prove it.  (That this leads to a pattern of private companies reporting inflated or misleading numbers is sadly no surprise.)  But don’t show up expecting to wow folks with vision. Ultimately, you’ll need to wow them with growth — which then provokes interest in vision.
  • Network.  Some companies do a nice and often quiet job of cultivating friends of the company who are thought leaders in their areas.  Many do this through inviting specific people to invest as angels.  Some do this simply through communications.  For example, one day I received an email update from Vik Singh clearly written for friends of Infer. I wasn’t sure how I got on the list, but found the company interesting and over time I got to know Vik (who is quite impressive) and ended up, well, a friend of Infer.  Some do this through advisory boards, both formal and informal.  For example, I did a little bit of advising for Tableau early on and later discovered a number of folks in my network who’d done the same thing.  The company benefitted by getting broad input on various topics and each of us felt like we were friends of Tableau.  While sort of thing doesn’t generate the same mainstream media buzz as a $1B valuation, it is a smart influencer strategy that can generate fans and buzz among the cognoscenti who, in theory at least, are opinion leaders in their chosen areas.

Before finishing the first part of this post, I need to provide a warning that halo effects are both powerful and addictive.  I seem to have a knack for competing against companies pursuing halo-driven strategies and the pattern I see typically runs like this.

  • Company starts getting some hype off good results.
  • Company starts saying increasingly aggressive things to build off the hype.
  • Analysts and press reward the hype with strong quadrant placements and great stories and blogs.
  • Company puts itself under increasing pressure to produce numbers that support the hype.

And then one of three things happens:

  1. The company continues delivering strong results and all is good, though the rhetoric and vision gets more unrelated to the business with each cycle.
  2. The company stops delivering results and is downgraded from hot-list to shit-list in the minds of the industry.
  3. The company cuts the cord with reality and starts inflating results in order to sustain the hype cycle and avoid outcome #2 above.  The vision inflates as aggressively as the numbers.

I have repeatedly had to compete against companies where claims/results were inflated to “prove” the value of bad/ordinary strategies to impress industry analysts to get strong quadrant positions to support broader claims of vision and leadership to drive more sales to inflate to even greater claimed results.  Surprisingly, I think this is usually done more in the name of ego than financial gain, but either way the story ends the same way — in terminations, lawsuits and, in one case, a jail sentence for the CEO.

Look, there are valid halo-driven strategies out there and I encourage you to try and use them to your company’s advantage — just be very careful you don’t end up addicted to halo heroin.  If you find yourself wanting to do almost anything to sustain the hype bubble, then you’ll know you’re addicted and headed for trouble.

The Customer View

Thus far, I’ve written this post entirely from the vendor viewpoint, but wanted to conclude by switching sides and offering customers some advice on how to think about halo effects in choosing vendors.   Customers should:

  • Be aware of halo effects.  The first step in dealing with any problem is understanding it exists. While supposedly technical, rational, and left-brained, technology can be as arbitrary as apparel when it comes to fashion.  If you’re evaluating vendors with halos, realize that they exist for a reason and then go understand why.  Are those drivers relevant — e.g., buying HR from Dave Duffield seems a reasonable idea.  Or are they spurious —  e.g., does it really matter that one board member invested in Facebook?  Or are they actually negative — e.g., if the company has raised $300M how crazy is their burn rate, what risk does that put on the business, and how focused will they stay on you as a customer and your problem as a market?
  •  Stay focused on your problem.  I encourage anyone buying technology to write down their business problems and high-level technology requirements before reaching out to vendors.  Hyped vendors are skilled at “changing the playing field” and trained to turn their vision into your (new) requirements.  While there certainly are cases where vendors can point out valid new requirements, you should periodically step back and do a sanity check:  are you still focused on your problem or have you been incrementally moved to a different, or greatly expanded one.  Vision is nice, but you won’t be around solve tomorrow’s problems if you can’t solve today’s.
  • Understand that industry analysts are often followers, not leaders.  If a vendor is showing you analyst support for their strategy, you need to figure out if the analyst is endorsing the strategy because of the strategy’s merits or because of the vendor’s claimed prior performance.  The latter is the definition of a halo effect and in a world full of private startups where high-quality analysts are in short supply, it’s easy to find “research” that effectively says nothing more than “this vendor is a leader because they say they’re performing really well and/or they’ve raised a lot of money.” That doesn’t tell you anything you didn’t know already and isn’t actually an independent source of information.  They are often simply amplifiers of the hype you’re already hearing.
  • Enjoy the sizzle; buy the steak.  Hype king Domo paid Alec Baldwin to make some (pretty pathetic) would-be viral videos and had Billy Beane, Flo Rida, Ludacris, and Marshawn Lynch at their user conference.  As I often say, behind any “marketing genius” is an enormous marketing budget, and that’s all you’re seeing — venture capital being directly converted into hype.  Heck, let them buy you a ticket to the show and have a great time.  Just don’t buy the software because of it — or because of the ability to invest more money in hand-grooming a handful of big-name references.  Look to meet customers like you, who have spent what you want to spend, and see if they’re happy and successful.  Don’t get handled into meeting other customers only at pre-arranged meetings.  Walk the floor and talk to regular people.  Find out how many are there for the show, or because they’re actual successful users of the software.
  • Dive into detail on the proposed solution.  Hyped vendors will often try to gloss over solutions and sell you the hype (e.g., “of course we can solve your problem, we’ve got the most logos, Gartner says we’re the leader, there’s an app for that.”)  What you need is a vendor who will listen to your problem, discuss it with you intelligently, and provide realistic estimates on what it takes to solve it.  The more willing they are to do that, the better off you are.  The more they keep talking about the founder’s escape from communism, the pedigree of their investors, their recent press coverage, or the amount of capital they’ve raised, the more likely you are to end up high and dry.  People interested in solving your problem will want to talk about your problem.
  • Beware the second-worst outcome:  the backwater.  Because hyped vendors are actually serving Sand Hill Road and/or Wall Street more than their customers, they pitch broad visions and huge markets in order to sustain the halo.  For a customer, that can be disastrous because the vendor may view the customer’s problems as simply another lily pad to jump off on the path to success.  The second-worst outcome is when you buy a solution and then vendor takes your money and invests it in solving other problems.  As a customer, you don’t want to marry your vendor’s fling.  You want to marry their core.  For startups, the pattern is typically over-expansion into too many things, getting in trouble, and then retracting hard back into the core, abandoning customers of the new, broader initiatives.  The second-worst outcome is when you get this alignment wrong and end up in a backwater or formerly-strategic area of your supplier’s strategy.
  • Avoid the worst outcome:  no there there.  Once in awhile, there is no “there there” behind some very hyped companies despite great individual investors, great VCs, strategic alliances, and a previously experienced team.  Perhaps the technology vision doesn’t pan out, or the company switches strategies (“pivots”) too often.  Perhaps the company just got too focused on its hype and not on it customers.  But the worst outcome, while somewhat rare, is when a company doesn’t solve its advertised problem. They may have a great story, a sexy demo, and some smart people — but what they lack is a core of satisfied customers solving the problem the company talks about.  In EPM, with due respect and in my humble opinion, Tidemark fell into this category, prior to what it called a “growth investment” and what sure seemed to me like a (fire) sale, to Marlin Equity Partners.  Customers need to watch out for these no-there-there situations and the best way to do that is taking strong dose of caveat emptor with a nose for “if it sounds too good to be true, then it might well possibly be.”

The New Split CPM Magic Quadrants from Gartner

This week Gartner research vice president John Van Decker and research director Chris Iervolino took the bold move of splitting the corporate performance management (CPM), also known as enterprise performance management (EPM), magic quadrant in two.

Instead of publishing a single magic quadrant (MQ) for all of CPM, they published two MQs, one for strategic CPM and one for financial CPM, which they define as follows:

  • Strategic Corporate Performance Management (SCPM) Solutions – this includes Corporate Planning and Modeling, Integrated Financial Planning, Strategy Management, Profitability Management, and Performance Reporting.
  • Financial Corporate Performance Management (FCPM) Solutions – this includes Financial Consolidation, Financial Reporting, Management Reporting/Costing/Forecasting, Reconciliations/Close Management, Intercompany Transactions, and Disclosure Management (including XBRL tagging)

You can download these new CPM magic quadrants here.

What do I think about this?

  • It’s bold.  It’s the first time to my recollection that an MQ has included product from different categories.  Put differently, normally MQs are full of substitute products — e.g., 15 different types of butter.  Here, we have butter next to olive oil on the same MQ.
  • It’s smart.  Their uber point is that while CPM solutions are now pretty varied, that you can pretty easily classify them into more tactical/financial uses and more strategic uses.  Highlighting this by splitting the MQs does customers a service because it reminds them to think both tactically and strategically.  That’s important — and often needed in many finance departments who are struggling simply to keep up with the ongoing tactical workload.
  • It’s potentially confusing.  You can find not just substitutes but complements on the same MQ.  For example, Host Analytics and our partner Blackline are both on the FCPM MQ.  That’s cool because we both serve core finance needs.  It’s potentially confusing because we do one thing and they do another.
  • We are stoked.  Among cloud pure-play EPM vendors, Host Analytics is the only supplier listed on both MQs.   We believe this supports our contention that we have the broadest pure-play cloud EPM product line in the business.  Only Host has both!
  • In a hype-filled world, I think Gartner does a great job of seeing through the hype-haze and focusing on customers and solutions.  They do a better job than most at not being over-influenced by Halo Effects, and I suspect that’s because they spend a lot of time talking to real customers about solving real problems.

For more, see the Future of Finance blog post on the new MQs or just go ahead and download them here.

Free Download of the Gartner 2015 Magic Quadrant for Corporate Performance Management Suites

Just a quick post to let you know that my company, Host Analytics, is offering a free copy / free download of the Gartner 2015 Magic Quadrant (MQ) for Corporate Performance Management (CPM) Suites.

You need to give about six fields of basic contact information to get the report, which can be downloaded here.  CPM is also known as enterprise performance management (EPM) and financial performance management (FPM) and includes corporate financial planning, scenario planning, budgeting, consolidations, financial reporting, profit modeling and optimization, and analytics.

Thoughts on the Gartner Magic Quadrant for Corporate Performance Management Suites 2015

“Mirror mirror on the wall, who’s the fairest EPM vendor of them all?”

It’s that time of the year again.  Gartner just released their Magic Quadrant for Corporate Performance Management (CPM) Suites 2015.  At Host Analytics, we call the category Enterprise Performance Management (EPM), so I’ll refer to the market henceforth as EPM.

In this post, I’ll share some of my commentary on the most recent Gartner Magic Quadrant (MQ) for EPM.

  • Get the big picture — and that picture is cloud.  All four cloud EPM vendors are in the Visionaries quadrant.  The first two trends in the market overview are about cloud.  The word “cloud” appears 113 times in the document, more than twice as much as the word “performance” at 46.   Cloud EPM is a huge trend in the market.
  • Use their critical capabilities research for short lists.   The MQ is a lot of fun to look at, but remember that Gartner analysts generally don’t recommend using the MQ for making vendor short lists.  Because markets are broad (and suite markets particularly so), they generally recommend using their critical capabilities documents to create a short list of which vendors are most likely to meet your specific needs.
  • Avoid dot-vector analysis.  Remember that Gartner analysts view the quadrant as re-created from scratch every year.  Vendors tend to obsess with dot-movement vectors, but when I ask the Gartner team about this issue, they tell me that the CPM MQ is effectively recreated from scratch each year and thus all about positions, not movements.  I know it’s hard to resist the temptation (heck one year at Business Objects I made an animated GIF of the MQ where the Brio dot even exploded in the end), but the folks there tell me it’s not meaningful, so I won’t do it.
  • Analyze via the football.  If you look at one-hundred Gartner MQs, you’ll see the pattern that the MQ placement algorithm generally lays out the vendors in a football shaped way.  I believe that inside the football you find the standard leaders, visionaries, challengers, etc.  Since most vendors are in the football, it’s more interesting to me to look who’s outside it and understand why.

football

  • Beware the impacts of the customer satisfaction survey.  I’ve seen and conducted customer satisfaction research on EPM vendors and what I see generally doesn’t jibe with the results that Gartner uses as an important data source in making the MQ.  By analogy, if you were trying to decide where to have dinner and I gave you the choice of two surveys, which would you pick?  Survey 1 — a random sample of diners leaving the restaurant.  Survey 2 — ask each restaurant to provide 10 customers.  You can argue that Survey 2 is “fair” because every restaurant had the same opportunity to prime, coach, offer free meals, etc., to the respondents.  But for my dinner investment, I’d pick survey 1.  The Gartner MQ survey is more like Survey 2.
  • Factor in degree of difficultly.  Another factor that influences the Gartner customer satisfaction survey is more subtle:  low-end vendors trying to solve easier problems should generally get better marks than high-end vendors trying to solve harder ones:  imagine a diving contest where we scored dives on execution but forgot to multiply by degree of difficulty.  Note that I do believe that the analysts understand this issue and try to mentally correct for it.  But the point is should I care how much QuickBooks users like QuickBooks if I’m in the league where only Oracle or SAP can solve my problem?
  • Read the text.   I sometimes think Gartner MQs are like old Playboy magazines, full of good journalism but where nobody actually reads the articles.  The MQ document is 1/35th pictures and 34/35th text.  Read the text. There’s a lot of great insight in there — where they even touch on some of the issues I’m raising above.
  • My favorite quote:  “Although the base of cloud CPM applications is still larger for small and mid-market organizations, the number of larger organizations (more than $1 billion in annual revenue) turning to the cloud for CPM continues to increase.”  This is why I joined Host Analytics 2.5 years ago.  EPM is an amazing market.  The cloud is still only scratching the surface of the market.  We are the leader in taking up-market EPM applications to the cloud.  And cloud EPM is increasingly coming up-market.

You can download a free copy of the Gartner Magic Quadrant for Corporate Performance Management Suites 2015 via Host Analytics, here.

ZL vs. Gartner: If At First You Don’t Succeed, Try, Try Again

As previously covered, email archiving vendor ZL Technologies of San Jose, California sued leading IT market researcher Gartner for $132M earlier this year for its treatment in one of Gartner’s vaunted magic quadrants.

On November 4th, the case was dismissed, but have no worries. Armed with a seemingly endless legal budget and apparent certainty in its position, ZL is back at it. One month later, on 12/4/09, ZL filed an amended complaint (PDF) in US District Court of Northern California, naming both Gartner as well as the lead analyst for email archiving, Carolyn DiCenzo, as defendants.

After a quick read, it appears primarily to be more of the same story, just better and more clearly argued.

They assert that:

  • ZL has superior products and services
  • Gartner dominates IT research with make-or-break power over vendors
  • Placement in the niche vendor quadrant is basically a fate worse than death
  • Gartner’s marketing creates the impression that analyst statements and reports are facts rather than opinion. Recall that freedom of opinion was Gartner’s defense, so this claim strikes me as pivotal.
  • Gartner’s business model contains an inherent conflict of interest and therefore its reviews are not unbiased third-party opinions
  • Gartner labeled ZL a “niche player” every year from 2005-2009. (Ouch. Now we see where the anger comes from!)
  • Magic quadrants are not based on objective, verifiable fact but on subjective opinion. (Here ZL seems to be arguing against itself. As I understand the law, we are all entitled to our opinions and we have the right to be wrong. What we can’t do is assert known falsity as truth — that’s defamation. But if the magic quadrants are opinion, then Gartner’s allowed to be wrong.)

English majors will enjoy some of the language used later in the document — I’m mixing and matching some words and phrases just to show the language:

… statements were false, malicious, fraudulent, oppressive, vile, base, contemptible … made with malice, hatred, ill-will, improper and malevolent purpose, reckless disregard, and knowledge of falsity … exposed ZL to hatred, contempt, ridicule, and obloquy

OK. What the heck is obloquy?

Part of me likes this offbeat little company trying to buck the system. Gartner is very influential, no doubt. As someone who’s worked with analysts for 20 years, I can say they’re not always the easiest people with whom to work. Discussions about magic quadrants (and imitations thereof) and the factors that drive them can be frustrating. But every company, including Gartner, has its flaws and every industry has its movie critics.

So a bigger part of me thinks that ZL’s just plain nuts. They are breaking glass all around them and spending real money to do it. If their software really is as good as they claim, then if they’d simply done a better job at marketing then they probably could have avoided all of this.

Because — and I agree with Gartner here — the best product / technology doesn’t always win. It takes a great business, too. And suing Gartner is, frankly, not something that I’ve seen many great businesses do.

Put differently, you can catch more flies with honey than vinegar. And ZL is dropping vinegar on Gartner right now like a helicopter dropping water on a forest fire.

For those who enjoy reading source documents, I’ve embedded the complaint below.

ZLFirst Amended Complaint

Judge Does Not Decide on Dimissing ZL Technologies Complaint Against Gartner

(Revised: confirmed that the source, MS&L, is ZL’s PR firm.)

While I don’t have official verification of this, I did learn the following this afternoon, regarding the lawsuit filed against Gartner by ZL Technologies over its treatment in their magic quadrants about which I posted earlier this week and which is also covered here, here, and here.

“In today’s hearing on Gartner’s motion to dismiss ZL’s complaint, the court did not come to a decision.”

Precisely because I couldn’t find any reference to this online (yet), I figured it was breaking news and should share it via the blog. I learned this information via an email from David Schraeder of MS&L, the PR firm representing ZL Technologies.

Given my limited understanding of the law, a non-decision is a decision. That is, by not deciding to throw out the case, I take to mean that the case is on and will proceed. More information will undoubtedly come out later, but I wanted to share this while it was hot.

It’s not every day I can break a story on the Mark Logic CEO blog!

Gartner Sued Over Magic Quadrant for Alleged Damages of $132M plus Punitives of $1.3B

I found this story today, entitled Gartner’s Magic Quadrant Goes to Court, about ZL Technologies who, citing damages of $132M, has decided to sue Gartner over its Magic Quadrants. From ZL’s web page on the suit:

ZL Technologies, a San Jose-based IT company specializing in … enterprise software solutions for e-mail and file archiving, is challenging Gartner Group and the legitimacy of Gartner’s “Magic Quadrant.” In a complaint … ZL claims that Gartner’s use of their proprietary “Magic Quadrant” is misleading and favors large vendors with large sales and marketing budgets over smaller innovators such as ZL that have developed higher performing products.

The complaint alleges: defamation; trade libel; false advertising; unfair competition; and negligent interference with prospective economic advantage.

“Sour grapes” spring to mind as an immediate reaction. In fact, ZL concedes that they’ve been ranked in the “niche” segment of every email archiving quadrant since 2005. (Ouch!) But they nevertheless argue that bigger stakes are in play and that this is not only about ZL, but Gartner itself, technological innovation, and very nearly preservation of the American way of life. Excerpt, edited for brevity:

Regardless of how the court may decide the First Amendment arguments, ZL hopes to achieve the following …

  • Fair Disclosure on Conflicts of Interest. Gartner generates its revenues from payments made by the same vendors whose products it evaluates. …
  • Fair Disclosure on Evaluation Scores. The tech industry would benefit if Gartner were required to disclose more data in its evaluation process and disclose component scores so vendors know exactly where they are lacking and by how much and take corrective action. …
  • Better Oversight. Gartner currently has an employee act as ombudsman to handle disagreements. The conflict of interest is self-evident in the way ZL’s concerns were summarily dismissed with little supporting evidence. ZL believes that Gartner’s immense heft and power in the marketplace necessitate careful checks and balances against abuse of power. ZL believes that if IT innovation is to remain a driver for the US economy, there must be assurances that ratings agencies such as Gartner do not subvert the competitive forces which drive innovation.

I remember a long time ago CA boycotted all Gartner research after some research-related dispute. It certain did nothing to help them: picking a fight with the movie critics always seems a risky strategy for a producer.

But it is hard to argue that Magic Quadrants are good for competition. They are inherently subjective in their assessments, they two-dimensionalize an N-dimensional problem, they encourage mental laziness on the part of customers, and –- heck –- some of us work in sectors that don’t even have a magic quadrant. What’s worse, ZL? Getting a poor ranking on an existing quadrant, or selling in a software category that Gartner doesn’t even recognize?

Since I think it’s fun to read court filings (when I have the time), let’s dig down a little deeper. The court documents are here, and I’ll embed them along the way as well. Here’s the initial complaint.

One of the many arguments made in the complaint is that Gartner doesn’t do “a single minute of independent testing of the products it purports to evaluate.” When I was younger in my career, I used to buy that argument. As I gotten older, I now realize (think: Wisdom of Crowds) that it is indeed possible to get a pretty good picture of
a product’s strengths and weaknesses simply by talking to lots of people who use it.

And that’s what Gartner does. Yes, there are no guys in lab coats doing Consumer Reports style testing. But, sometimes the guys in the lab coats measure the wrong things anyway. So while Gartner does not, to my knowledge, do hands-on testing, they neither claim to do so nor, in my estimation, is such testing strictly necessary to develop an informed opinion.

That said, a pathological case of that research model is when a vendor has very small market share. If research is done primarily through talking and there’s no one to talk to, then you’re not going to get on the map very easily.

On the other hand, I love their brass tacks description of the reality behind being labeled a “niche player”:

These MQ placements were, and are, derogatory because they are understood by technology purchasers as a warning, by Gartner, that ZL and ZL products are not good choices for enterprise email archive applications.

Yep.

Also of interest was this statement by Gartner’s ombudsman:

My sense is that there has been a relationship issue for many years with [archiving analyst] Carolyn DiCenzio and at this point it’s come down to level of trust and respect.”

I suppose there’s some logical consistency at least — if you’re going to declare war on the #1 analyst firm, well, why not make it personal as well. :-)

Let’s move on. Here’s Gartner’s response, a motion to dismiss, which is much tougher reading and more techno-legal:

Clearly, Gartner’s response is based on opinion and freedom of speech. Excerpt:

Whether plaintiff’s opinions about its product are correct, comprehensible or sincere has no legal significance; what matters is that the Complaint fails to state a claim because it attacks opinions expressed by Gartner, Inc. These opinions are constitutionally protected, in part to discourage lawsuits like this one, which are aimed at chilling the free expression of ideas and opinions.

While Gartner marketing may not love that response (imagine: “could we please defend the research as well as our right to have opinions?”) it’s not a terribly surprising one.

If nothing else, you can being to see why lawsuits cost so much money. Bear in mind the legal meters are probably running at $600/hour and they’re still debating whether the case should be immediately thrown out: it’s like dropping $20K standing on the starting line fighting about the start time for the race.

Here is ZL’s opposition to Gartner’s motion for dismissal, another 32-pages:

If you didn’t jump into the document above, let me pull out the first zinging paragraph (bolding mine):

This is a commercial case about a dominant industry player’s baseless defamation of an independent startup whose growth prospects have been crushed by the defendant’s unfair business practices. Defendant Gartner, Inc. (“Gartner”), which advises businesses on information technology decisions, exercises hegemonic control over the purchases made by a wide swath of the international corporate and governmental market. The technology Gartner says to buy is bought; what Gartner says not to buy languishes unsold, leaving its developers scrambling for the leftover market share Gartner does not dictate. The problem arises when Gartner exercises its market power recklessly, maliciously or—because of its tremendous influence—negligently. When that occurs, as it has here, innovation and competition are stifled, to the detriment of small companies who lack the resources to challenge Gartner, and to the consuming public at large.

Wow, someone turned up the rhetoric meter! At this point things are quickly getting over my legal head. There arguments seem to be largely about what is fact vs. opinion. Since I’m unable to comment on the legal issue, I’ll move on.

Finally, for the strong of legal stomach, here is Gartner’s reply to the opposition t
o the motion to dismiss. (Say that ten times fast.)

Here’s a nice summary of the counter-argument:

Try as it might, ZL cannot create a dispute where there is none. ZL alleges at great length in its Complaint (and recapitulates in its Opposition) that it has a strong product and satisfied customers. The Magic Quadrant reports do not say otherwise; the real point of contention here is not the quality of ZL’s product, but instead the subjective analytical model Gartner used to assess ZL’s market position and prospects. ZL does not contest Gartner’s basic assessments of ZL—that it has a good product but needs to expand its sales and marketing—but ZL challenges its placement on the Magic Quadrant Report because Gartner uses a “misguided analytical model” that gives “undue weight to sales and marketing.”

I have no idea how this will end. Will it be quickly thrown out of court? Will it a long drawn-out case? I don’t know. I would say that Gartner’s quadrants wield enormous power and that vendors go to great lengths to maxmize their position on them. And I’d say that you can’t judge a vendor by the quality of its technology alone. While Ingres arguably had the best database technology in the 1980s, Oracle’s sales and marketing prowess caused it to win the market and any analyst who — focused solely on the technology — would have recommended Ingres at that time would have done his customers a disservice.

I don’t know how the movie here ends, but I at least expect it to be interesting.