Category Archives: Gartner

I found this story today, entitled Gartner’s Magic Quadrant Goes to Court, about ZL Technologies who, citing damages of $132M, has decided to sue Gartner over its Magic Quadrants. From ZL’s web page on the suit:

ZL Technologies, a San Jose-based IT company specializing in … enterprise software solutions for e-mail and file archiving, is challenging Gartner Group and the legitimacy of Gartner’s “Magic Quadrant.” In a complaint … ZL claims that Gartner’s use of their proprietary “Magic Quadrant” is misleading and favors large vendors with large sales and marketing budgets over smaller innovators such as ZL that have developed higher performing products.

The complaint alleges: defamation; trade libel; false advertising; unfair competition; and negligent interference with prospective economic advantage.

“Sour grapes” spring to mind as an immediate reaction. In fact, ZL concedes that they’ve been ranked in the “niche” segment of every email archiving quadrant since 2005. (Ouch!) But they nevertheless argue that bigger stakes are in play and that this is not only about ZL, but Gartner itself, technological innovation, and very nearly preservation of the American way of life. Excerpt, edited for brevity:

Regardless of how the court may decide the First Amendment arguments, ZL hopes to achieve the following …

• Fair Disclosure on Conflicts of Interest. Gartner generates its revenues from payments made by the same vendors whose products it evaluates. …

• Fair Disclosure on Evaluation Scores. The tech industry would benefit if Gartner were required to disclose more data in its evaluation process and disclose component scores so vendors know exactly where they are lacking and by how much and take corrective action. …

• Better Oversight. Gartner currently has an employee act as ombudsman to handle disagreements. The conflict of interest is self-evident in the way ZL’s concerns were summarily dismissed with little supporting evidence. ZL believes that Gartner’s immense heft and power in the marketplace necessitate careful checks and balances against abuse of power. ZL believes that if IT innovation is to remain a driver for the US economy, there must be assurances that ratings agencies such as Gartner do not subvert the competitive forces which drive innovation.

I remember a long time ago CA boycotted all Gartner research after some research-related dispute. It certain did nothing to help them: picking a fight with the movie critics always seems a risky strategy for a producer.

But it is hard to argue that Magic Quadrants are good for competition. They are inherently subjective in their assessments, they two-dimensionalize an N-dimensional problem, they encourage mental laziness on the part of customers, and –- heck –- some of us work in sectors that don’t even have a magic quadrant. What’s worse, ZL? Getting a poor ranking on an existing quadrant, or selling in a software category that Gartner doesn’t even recognize?

Since I think it’s fun to read court filings (when I have the time), let’s dig down a little deeper. The court documents are here, and I’ll embed them along the way as well. Here’s the initial complaint.

One of the many arguments made in the complaint is that Gartner doesn’t do “a single minute of independent testing of the products it purports to evaluate.” When I was younger in my career, I used to buy that argument. As I gotten older, I now realize (think: Wisdom of Crowds) that it is indeed possible to get a pretty good picture of
a product’s strengths and weaknesses simply by talking to lots of people who use it.

And that’s what Gartner does. Yes, there are no guys in lab coats doing Consumer Reports style testing. But, sometimes the guys in the lab coats measure the wrong things anyway. So while Gartner does not, to my knowledge, do hands-on testing, they neither claim to do so nor, in my estimation, is such testing strictly necessary to develop an informed opinion.

That said, a pathological case of that research model is when a vendor has very small market share. If research is done primarily through talking and there’s no one to talk to, then you’re not going to get on the map very easily.

On the other hand, I love their brass tacks description of the reality behind being labeled a “niche player”:

These MQ placements were, and are, derogatory because they are understood by technology purchasers as a warning, by Gartner, that ZL and ZL products are not good choices for enterprise email archive applications.

Yep.

Also of interest was this statement by Gartner’s ombudsman:

My sense is that there has been a relationship issue for many years with [archiving analyst] Carolyn DiCenzio and at this point it’s come down to level of trust and respect.”

I suppose there’s some logical consistency at least — if you’re going to declare war on the #1 analyst firm, well, why not make it personal as well. :-)

Let’s move on. Here’s Gartner’s response, a motion to dismiss, which is much tougher reading and more techno-legal:

Clearly, Gartner’s response is based on opinion and freedom of speech. Excerpt:

Whether plaintiff’s opinions about its product are correct, comprehensible or sincere has no legal significance; what matters is that the Complaint fails to state a claim because it attacks opinions expressed by Gartner, Inc. These opinions are constitutionally protected, in part to discourage lawsuits like this one, which are aimed at chilling the free expression of ideas and opinions.

While Gartner marketing may not love that response (imagine: “could we please defend the research as well as our right to have opinions?”) it’s not a terribly surprising one.

If nothing else, you can being to see why lawsuits cost so much money. Bear in mind the legal meters are probably running at $600/hour and they’re still debating whether the case should be immediately thrown out: it’s like dropping $20K standing on the starting line fighting about the start time for the race.

Here is ZL’s opposition to Gartner’s motion for dismissal, another 32-pages:

If you didn’t jump into the document above, let me pull out the first zinging paragraph (bolding mine):

This is a commercial case about a dominant industry player’s baseless defamation of an independent startup whose growth prospects have been crushed by the defendant’s unfair business practices. Defendant Gartner, Inc. (“Gartner”), which advises businesses on information technology decisions, exercises hegemonic control over the purchases made by a wide swath of the international corporate and governmental market. The technology Gartner says to buy is bought; what Gartner says not to buy languishes unsold, leaving its developers scrambling for the leftover market share Gartner does not dictate. The problem arises when Gartner exercises its market power recklessly, maliciously or—because of its tremendous influence—negligently. When that occurs, as it has here, innovation and competition are stifled, to the detriment of small companies who lack the resources to challenge Gartner, and to the consuming public at large.

Wow, someone turned up the rhetoric meter! At this point things are quickly getting over my legal head. There arguments seem to be largely about what is fact vs. opinion. Since I’m unable to comment on the legal issue, I’ll move on.

Finally, for the strong of legal stomach, here is Gartner’s reply to the opposition t
o the motion to dismiss. (Say that ten times fast.)

Here’s a nice summary of the counter-argument:

Try as it might, ZL cannot create a dispute where there is none. ZL alleges at great length in its Complaint (and recapitulates in its Opposition) that it has a strong product and satisfied customers. The Magic Quadrant reports do not say otherwise; the real point of contention here is not the quality of ZL’s product, but instead the subjective analytical model Gartner used to assess ZL’s market position and prospects. ZL does not contest Gartner’s basic assessments of ZL—that it has a good product but needs to expand its sales and marketing—but ZL challenges its placement on the Magic Quadrant Report because Gartner uses a “misguided analytical model” that gives “undue weight to sales and marketing.”

I have no idea how this will end. Will it be quickly thrown out of court? Will it a long drawn-out case? I don’t know. I would say that Gartner’s quadrants wield enormous power and that vendors go to great lengths to maxmize their position on them. And I’d say that you can’t judge a vendor by the quality of its technology alone. While Ingres arguably had the best database technology in the 1980s, Oracle’s sales and marketing prowess caused it to win the market and any analyst who — focused solely on the technology — would have recommended Ingres at that time would have done his customers a disservice.

I don’t know how the movie here ends, but I at least expect it to be interesting.