Category Archives: Magic Quadrant

The New Gartner 2018 Magic Quadrants for Cloud Financial Planning & Analysis and Cloud Financial Close Solutions

If all you’re looking for is the free download link, let’s cut to the chase:  here’s where you can download the new 2018 Gartner Magic Quadrant for Financial Planning and Analysis Solutions and the new 2018 Gartner Magic Quadrant for Cloud Financial Close Solutions.  These MQs are written jointly by John Van Decker and Chris Iervolino (with Chris as primary author on the first and John as primary author on the second).  Both are deep experts in the category with decades of experience.

Overall, I can say that at Host Analytics, we are honored to a leader in both MQs again this year.  We are also honored to be the only cloud pure-play vendor to be a leader in both MQs and we believe that speaks volumes about the depth and breadth of EPM functionality that we bring to the cloud.

So, if all you wanted was the links, thanks for visiting.  If, however, you’re looking for some Kellblog editorial on these MQs, then please continue on.

Whither CPM?
The first thing the astute reader will notice is that the category name, which Gartner formerly referred to as corporate performance management (CPM), and which others often referred to as enterprise performance management (EPM), is entirely missing from these MQs.  That’s no accident.  Gartner decided last fall to move away from CPM as a uber category descriptor in favor of referring more directly to the two related, but pretty different, categories beneath it.  Thus, in the future you won’t be hearing “CPM” from Gartner anymore, though I know that some vendors — including Host Analytics — will continue to use EPM/CPM until we can find a more suitable capstone name for the category.

Personally, I’m in favor of this move for two simple reasons.

  • CPM was a forced, analyst-driven category in the first place, dating back to Howard Dresner’s predictions that financial planning/budgeting would converge with business intelligence.  While Howard published the research that launched a thousand ships in terms of BI and financial planning industry consolidation (e.g., Cognos/Adaytum, BusinessObjects/SRC/Cartesis, Hyperion/Brio), the actual software itself never converged.  CPM never became like CRM — a true convergence of sales force automation (SFA) and contact center.  In each case, the two companies could be put under one roof, but they sold fundamentally different value propositions to very different buyers and thus never came together as one.
  • In accordance with the prior point, few customers actually refer to the category by CPM/EPM.  They say things much more akin to “financial planning” and “consolidation and close management.”  Since I like referring to things in the words that customers use, I am again in favor of this change.

It does, however, create one problem — Gartner has basically punted on trying to name a capstone category to include vendors who sell both financial planning and financial consolidation software.  Since we at Host Analytics think that’s important, and since we believe there are key advantages to buying both from the same vendor, we’d prefer if there were a single, standard capstone term.  If it were easy, I suppose a name would have already emerged [1].

How Not To Use Magic Quadrants
While they are Gartner’s flagship deliverable, magic quadrants (MQs) can generate a lot of confusion.  MQs don’t tell you which vendor is “best” because there is no universal best in any category.  MQs don’t tell you which vendor to pick to solve your problem because different solutions are designed around meeting different requirements.  MQs don’t predict the future of vendors — last-year’s movement vectors rarely predict this year’s positions.  And the folks I know at Gartner generally strongly dislike vector analysis of MQs because they view vendor placement as relative to each other at any moment in time [2].

Many things that customers seem to want from Gartner MQs are actually delivered by Gartner’s Critical Capabilities reports which get less attention because they don’t produce a simple, dramatic 2×2 output, but which are far better suited for determine the suitability of different products to different use-cases.

How To Use A Gartner Magic Quadrant?
In my experience after 25+ in enterprise software, I would use MQs for their overall purpose:  to group vendors into 4 different bucketsleaders, challengers, visionaries, and niche players.  That’s it.  If you want to know who the leaders are in a category, look top right.  If you want to know who the visionaries are, look bottom right.  You want to know which big companies are putting resources into the category but who thus far are lacking strategy/vision, then look top-left at the challengers quadrant.

But should you, in my humble opinion, get particularly excited about millimeter differences on either axes?  No.  Why?  Because what drives those deltas may have little, none, or in fact a counter-correlation to your situation.  In my experience, the analysts pay a lot of attention to the quadrants in which vendors end up in [2] so quadrant-placement, I’d say, is quite closely watched by the analysts.  Dot-placement, while closely watched by vendors, save for dramatic differences, doesn’t change much in the real world.  After all, they are called the magic quadrants, not the magic dots.

All that said, let me wind up with some observations on the MQs themselves.

Quick Thoughts on the 2018 Cloud FP&A Solutions MQ
While the MQs were published at the end of July 2018, they were based on information about the vendors gathered in and largely about 2017.  While there is always some phase-lag between the end of data collection and the publication data, this year it was rather unusually long — meaning that a lot may have changed in the market in the first half of 2018 that customers should be aware of. For that reason, if you’re a Gartner customer and using either the MQs or critical capabilities reports that accompany them, you should probably setup an appointment to call the analysts to ensure you’re working off the latest data.

Here are some of my quick thoughts the Cloud FP&A Solutions magic quadrant:

  • Gartner says the FP&A market is accelerating its shift from on-premises cloud.  I agree.
  • Gartner allows three types of “cloud” vendors into this (and the other) MQ:  cloud-only vendors, on-premise vendors with new built-for-the-cloud solutions, and on-premises vendors who allow their software to be run hosted on a third-party cloud platform.  While I understand their need to be inclusive, I think this is pretty broad — the total cost of ownership, cash flows, and incentives are quite different between pure cloud vendors and hosted on-premises solutions.  Caveat emptor.
  • To qualify for the MQ vendors must support at least two of the four following components of FP&A:  planning/budgeting, integrated financial planning, forecasting/modeling, management/performance reporting.  Thus the MQ is not terribly homogeneous in terms of vendor profile and use-cases.
  • For the second year in a row, (1) Host is a leader in this MQ and (2) is the only cloud pure-play vendor who is a leader in both.  We think this says a lot about the breadth and depth of our product line.
  • Customer references for Host cited ease of use, price, and solution flexibility as top three purchasing criteria.  We think this very much represents our philosophy of complex EPM made easy.

Quick Thoughts on the 2018 Cloud Financial Close Solutions MQ
Here are some of my quick thoughts on the Cloud Financial Close Solutions magic quadrant:

  • Gartner says that in the past two years the financial close market has shifted from mature on-premises to cloud solutions.  I agree.
  • While Gartner again allowed all three types of cloud vendors in this MQ, I believe some of the vendors in this MQ do just-enough, just-cloud-enough business to clear the bar, but are fundamentally still offering on-premise wolves in cloud sheep’s clothing.  Customers should look to things like total cost of ownership, upgrade frequency, and upgrade phase lags in order to flesh out real vs. fake cloud offerings.
  • This MQ is more of a mixed bag than the FP&A MQ or, for that matter, most Gartner MQs.  In general, MQs plot substitutes against each other — each dot on an MQ usually represents a vendor who does basically the same thing.  This is not true for the Cloud Financial Close (CFC) MQ — e.g., Workiva is a disclosure management vendor (and a partner of Host Analytics).  However, they do not offer financial consolidation software, as does say Host Analytics or Oracle.
  • Because the scope of this MQ is broad and both general and specialist vendors are included, customers should either call the Gartner for help (if they are Gartner customers) or just be mindful of the mixing and segmentation — e.g., Floqast (in SMB and MM) and Blackline (in enterprise) both do account reconciliation, but they are naturally segmented by customer size (and both are partners of Host, which does financial consolidation but not account reconciliation).
  • Net:  while I love that the analysts are willing to put different types of close-related, office-of-the-CFO-oriented vendors on the same MQ, it does require more than the usual amount of mindfulness in interpreting it.

Conclusion
Finally, if you want to analyze the source documents yourself, you can use the following link to download both the 2018 Gartner Magic Quadrant for Financial Planning and Analysis and Consolidation and Close Management.

# # #

Notes

[1] For Gartner, this is likely more than a semantic issue.  They are pretty strong believers in a “post-modern” ERP vision which eschews the idea of a monolithic application that includes all services, in favor of using and integrating a series of cloud-based services.  Since we are also huge believers in integrating best-of-breed cloud services, it’s hard for us to take too much issue with that.  So we’ll simply have to clearly articulate the advantages of using Host Planning and Host Consolidations together — from our viewpoint, two best-of-breed cloud services that happen to come from a single vendor.

[2] And not something done against absolute scales where you can track movement over time.  See, for example, the two explicit disclaimers in the FP&A MQ:

Capture

[3] I’m also a believer in a slightly more esoteric theory which says:  given that the Gartner dot-placement algorithm seems to try very hard to layout dots in a 45-degree-tilted football shaped pattern, it is always interesting to examine who, how, and why someone ends up outside that football.

The New 2017 Gartner Magic Quadrants for Cloud Strategic CPM (SCPM) and Cloud Financial CPM (FCPM) – How to Download; A Few Thoughts

For some odd reason, I always think of this scene — The New Phone Book’s Here – from an old Steve Martin comedy whenever Gartner rolls out their new Magic Quadrants (MQ) for corporate performance management (CPM). It’s probably because all of the excitement they generate.

Last year, Gartner researchers John Van Decker and Chris Iervolino kept that excitement up by making the provocative move of splitting the CPM quadrant in two — strategic CPM (SCPM) and financial CPM (FCPM). Never complacent, this year they stirred things up again by inserting the word “cloud” before the category name for each; we’ll discuss the ramifications of that in a minute.

Free Download of 2017 CPM Magic Quadrants

But first, let me provide some links where you can download the new FCPM and SCPM magic quadrants:

Significance of the New 2017 FPCM and SCPM Magic Quadrants

The biggest change this year is the insertion of the word “cloud” in the title of the magic quadrants.  This perhaps seemingly small change, like a butterfly effect, results in an entirely new world order where two of the three megavendors in the category (i.e., IBM, SAP) get displaced from market leadership due to the lack of the credibility and/or sophistication of their cloud offerings.

For example:

  • In the strategic CPM quadrant, IBM is relegated to the Visionary quadrant (bottom right) and SAP does not even make the cut.
  • In the financial CPM quadrant, IBM is relegated to the Challenger quadrant (top left) and SAP again does not even make the cut.

Well, I suppose one might then ask, well if IBM and SAP do poorly in the cloud financial and strategic CPM magic quadrants, then how do they do in the “regular” ones?

To which the answer is, there aren’t any “regular” ones; they only made cloud ones.  That’s the point.

So I view this as the mainstreaming of cloud in EPM [1].  Gartner is effectively saying a few things:

  • Who cares how much maintenance fees a vendor derives from legacy products?
  • The size of a vendor’s legacy base is independent of its position for the future.
  • The cloud is now the norm in CPM product selection, so it’s uninteresting to even produce a non-cloud MQ for CPM. The only CPM MQs are the cloud ones.

While I have plenty of beefs with Oracle as a prospective business partner — and nearly as many with their cloud EPM offerings — to their credit, they have been making an effort at cloud EPM while IBM and SAP seem to have somehow been caught off-guard, at least from an EPM perspective.

(Some of Oracle’s overall cloud revenue success is likely cloudwashing though they settled a related lawsuit with the whistleblower so we’ll never know the details.)

Unlikely Bedfellows:  Only Two Vendors are Leaders in Both FCPM and SCPM Magic Quadrants

This creates the rather odd situation where there are only two vendors in the Leaders section of both the financial and strategic CPM magic quadrants:  Host Analytics and Oracle.  That means only two vendors can provide the depth and breadth of products in the cloud to qualify for the Leaders quadrant in both the FCPM and SCPM MQ.

I know who I’d rather buy from.

In my view, Host Analytics has a more complete, mature, and proven product line – we’ve been at this a lot longer than they have — and, well, oligopolists aren’t really famous for their customer success and solutions orientation.  More infamous, in fact.  See the section of the FCPM report where it says Oracle ranks in the “bottom 25% of vendors in this MQ on ‘overall satisfaction with vendor.’”

Or how an Oracle alumni once defined “solution selling” for me:

Your problem is you are out of compliance with the license agreement and we’re going to shut down the system.  The solution is to give us money.

Nice.

For more editorial, you can read John O’Rourke’s post on the Host Analytics corporate blog.

Download the 2017 FCPM and SCPM Magic Quadrants

Or you can download the new 2017 Gartner CPM MQs here.

# # #

Notes:

[1] Gartner refers to the category as corporate performance management (CPM).  I generally refer to it as enterprise performance management (EPM), reflecting the fact that EPM software is useful not only for corporations, but other forms of organization such as not-for-profit, partnerships, government, etc.  That difference aside, I generally view EPM and CPM as synonyms.

The New Split CPM Magic Quadrants from Gartner

This week Gartner research vice president John Van Decker and research director Chris Iervolino took the bold move of splitting the corporate performance management (CPM), also known as enterprise performance management (EPM), magic quadrant in two.

Instead of publishing a single magic quadrant (MQ) for all of CPM, they published two MQs, one for strategic CPM and one for financial CPM, which they define as follows:

  • Strategic Corporate Performance Management (SCPM) Solutions – this includes Corporate Planning and Modeling, Integrated Financial Planning, Strategy Management, Profitability Management, and Performance Reporting.
  • Financial Corporate Performance Management (FCPM) Solutions – this includes Financial Consolidation, Financial Reporting, Management Reporting/Costing/Forecasting, Reconciliations/Close Management, Intercompany Transactions, and Disclosure Management (including XBRL tagging)

You can download these new CPM magic quadrants here.

What do I think about this?

  • It’s bold.  It’s the first time to my recollection that an MQ has included product from different categories.  Put differently, normally MQs are full of substitute products — e.g., 15 different types of butter.  Here, we have butter next to olive oil on the same MQ.
  • It’s smart.  Their uber point is that while CPM solutions are now pretty varied, that you can pretty easily classify them into more tactical/financial uses and more strategic uses.  Highlighting this by splitting the MQs does customers a service because it reminds them to think both tactically and strategically.  That’s important — and often needed in many finance departments who are struggling simply to keep up with the ongoing tactical workload.
  • It’s potentially confusing.  You can find not just substitutes but complements on the same MQ.  For example, Host Analytics and our partner Blackline are both on the FCPM MQ.  That’s cool because we both serve core finance needs.  It’s potentially confusing because we do one thing and they do another.
  • We are stoked.  Among cloud pure-play EPM vendors, Host Analytics is the only supplier listed on both MQs.   We believe this supports our contention that we have the broadest pure-play cloud EPM product line in the business.  Only Host has both!
  • In a hype-filled world, I think Gartner does a great job of seeing through the hype-haze and focusing on customers and solutions.  They do a better job than most at not being over-influenced by Halo Effects, and I suspect that’s because they spend a lot of time talking to real customers about solving real problems.

For more, see the Future of Finance blog post on the new MQs or just go ahead and download them here.

Free Download of the Gartner 2015 Magic Quadrant for Corporate Performance Management Suites

Just a quick post to let you know that my company, Host Analytics, is offering a free copy / free download of the Gartner 2015 Magic Quadrant (MQ) for Corporate Performance Management (CPM) Suites.

You need to give about six fields of basic contact information to get the report, which can be downloaded here.  CPM is also known as enterprise performance management (EPM) and financial performance management (FPM) and includes corporate financial planning, scenario planning, budgeting, consolidations, financial reporting, profit modeling and optimization, and analytics.

Thoughts on the Gartner Magic Quadrant for Corporate Performance Management Suites 2015

“Mirror mirror on the wall, who’s the fairest EPM vendor of them all?”

It’s that time of the year again.  Gartner just released their Magic Quadrant for Corporate Performance Management (CPM) Suites 2015.  At Host Analytics, we call the category Enterprise Performance Management (EPM), so I’ll refer to the market henceforth as EPM.

In this post, I’ll share some of my commentary on the most recent Gartner Magic Quadrant (MQ) for EPM.

  • Get the big picture — and that picture is cloud.  All four cloud EPM vendors are in the Visionaries quadrant.  The first two trends in the market overview are about cloud.  The word “cloud” appears 113 times in the document, more than twice as much as the word “performance” at 46.   Cloud EPM is a huge trend in the market.
  • Use their critical capabilities research for short lists.   The MQ is a lot of fun to look at, but remember that Gartner analysts generally don’t recommend using the MQ for making vendor short lists.  Because markets are broad (and suite markets particularly so), they generally recommend using their critical capabilities documents to create a short list of which vendors are most likely to meet your specific needs.
  • Avoid dot-vector analysis.  Remember that Gartner analysts view the quadrant as re-created from scratch every year.  Vendors tend to obsess with dot-movement vectors, but when I ask the Gartner team about this issue, they tell me that the CPM MQ is effectively recreated from scratch each year and thus all about positions, not movements.  I know it’s hard to resist the temptation (heck one year at Business Objects I made an animated GIF of the MQ where the Brio dot even exploded in the end), but the folks there tell me it’s not meaningful, so I won’t do it.
  • Analyze via the football.  If you look at one-hundred Gartner MQs, you’ll see the pattern that the MQ placement algorithm generally lays out the vendors in a football shaped way.  I believe that inside the football you find the standard leaders, visionaries, challengers, etc.  Since most vendors are in the football, it’s more interesting to me to look who’s outside it and understand why.

football

  • Beware the impacts of the customer satisfaction survey.  I’ve seen and conducted customer satisfaction research on EPM vendors and what I see generally doesn’t jibe with the results that Gartner uses as an important data source in making the MQ.  By analogy, if you were trying to decide where to have dinner and I gave you the choice of two surveys, which would you pick?  Survey 1 — a random sample of diners leaving the restaurant.  Survey 2 — ask each restaurant to provide 10 customers.  You can argue that Survey 2 is “fair” because every restaurant had the same opportunity to prime, coach, offer free meals, etc., to the respondents.  But for my dinner investment, I’d pick survey 1.  The Gartner MQ survey is more like Survey 2.
  • Factor in degree of difficultly.  Another factor that influences the Gartner customer satisfaction survey is more subtle:  low-end vendors trying to solve easier problems should generally get better marks than high-end vendors trying to solve harder ones:  imagine a diving contest where we scored dives on execution but forgot to multiply by degree of difficulty.  Note that I do believe that the analysts understand this issue and try to mentally correct for it.  But the point is should I care how much QuickBooks users like QuickBooks if I’m in the league where only Oracle or SAP can solve my problem?
  • Read the text.   I sometimes think Gartner MQs are like old Playboy magazines, full of good journalism but where nobody actually reads the articles.  The MQ document is 1/35th pictures and 34/35th text.  Read the text. There’s a lot of great insight in there — where they even touch on some of the issues I’m raising above.
  • My favorite quote:  “Although the base of cloud CPM applications is still larger for small and mid-market organizations, the number of larger organizations (more than $1 billion in annual revenue) turning to the cloud for CPM continues to increase.”  This is why I joined Host Analytics 2.5 years ago.  EPM is an amazing market.  The cloud is still only scratching the surface of the market.  We are the leader in taking up-market EPM applications to the cloud.  And cloud EPM is increasingly coming up-market.

You can download a free copy of the Gartner Magic Quadrant for Corporate Performance Management Suites 2015 via Host Analytics, here.

ZL vs. Gartner: If At First You Don’t Succeed, Try, Try Again

As previously covered, email archiving vendor ZL Technologies of San Jose, California sued leading IT market researcher Gartner for $132M earlier this year for its treatment in one of Gartner’s vaunted magic quadrants.

On November 4th, the case was dismissed, but have no worries. Armed with a seemingly endless legal budget and apparent certainty in its position, ZL is back at it. One month later, on 12/4/09, ZL filed an amended complaint (PDF) in US District Court of Northern California, naming both Gartner as well as the lead analyst for email archiving, Carolyn DiCenzo, as defendants.

After a quick read, it appears primarily to be more of the same story, just better and more clearly argued.

They assert that:

  • ZL has superior products and services
  • Gartner dominates IT research with make-or-break power over vendors
  • Placement in the niche vendor quadrant is basically a fate worse than death
  • Gartner’s marketing creates the impression that analyst statements and reports are facts rather than opinion. Recall that freedom of opinion was Gartner’s defense, so this claim strikes me as pivotal.
  • Gartner’s business model contains an inherent conflict of interest and therefore its reviews are not unbiased third-party opinions
  • Gartner labeled ZL a “niche player” every year from 2005-2009. (Ouch. Now we see where the anger comes from!)
  • Magic quadrants are not based on objective, verifiable fact but on subjective opinion. (Here ZL seems to be arguing against itself. As I understand the law, we are all entitled to our opinions and we have the right to be wrong. What we can’t do is assert known falsity as truth — that’s defamation. But if the magic quadrants are opinion, then Gartner’s allowed to be wrong.)

English majors will enjoy some of the language used later in the document — I’m mixing and matching some words and phrases just to show the language:

… statements were false, malicious, fraudulent, oppressive, vile, base, contemptible … made with malice, hatred, ill-will, improper and malevolent purpose, reckless disregard, and knowledge of falsity … exposed ZL to hatred, contempt, ridicule, and obloquy

OK. What the heck is obloquy?

Part of me likes this offbeat little company trying to buck the system. Gartner is very influential, no doubt. As someone who’s worked with analysts for 20 years, I can say they’re not always the easiest people with whom to work. Discussions about magic quadrants (and imitations thereof) and the factors that drive them can be frustrating. But every company, including Gartner, has its flaws and every industry has its movie critics.

So a bigger part of me thinks that ZL’s just plain nuts. They are breaking glass all around them and spending real money to do it. If their software really is as good as they claim, then if they’d simply done a better job at marketing then they probably could have avoided all of this.

Because — and I agree with Gartner here — the best product / technology doesn’t always win. It takes a great business, too. And suing Gartner is, frankly, not something that I’ve seen many great businesses do.

Put differently, you can catch more flies with honey than vinegar. And ZL is dropping vinegar on Gartner right now like a helicopter dropping water on a forest fire.

For those who enjoy reading source documents, I’ve embedded the complaint below.

ZLFirst Amended Complaint

Gartner Sued Over Magic Quadrant for Alleged Damages of $132M plus Punitives of $1.3B

I found this story today, entitled Gartner’s Magic Quadrant Goes to Court, about ZL Technologies who, citing damages of $132M, has decided to sue Gartner over its Magic Quadrants. From ZL’s web page on the suit:

ZL Technologies, a San Jose-based IT company specializing in … enterprise software solutions for e-mail and file archiving, is challenging Gartner Group and the legitimacy of Gartner’s “Magic Quadrant.” In a complaint … ZL claims that Gartner’s use of their proprietary “Magic Quadrant” is misleading and favors large vendors with large sales and marketing budgets over smaller innovators such as ZL that have developed higher performing products.

The complaint alleges: defamation; trade libel; false advertising; unfair competition; and negligent interference with prospective economic advantage.

“Sour grapes” spring to mind as an immediate reaction. In fact, ZL concedes that they’ve been ranked in the “niche” segment of every email archiving quadrant since 2005. (Ouch!) But they nevertheless argue that bigger stakes are in play and that this is not only about ZL, but Gartner itself, technological innovation, and very nearly preservation of the American way of life. Excerpt, edited for brevity:

Regardless of how the court may decide the First Amendment arguments, ZL hopes to achieve the following …

  • Fair Disclosure on Conflicts of Interest. Gartner generates its revenues from payments made by the same vendors whose products it evaluates. …
  • Fair Disclosure on Evaluation Scores. The tech industry would benefit if Gartner were required to disclose more data in its evaluation process and disclose component scores so vendors know exactly where they are lacking and by how much and take corrective action. …
  • Better Oversight. Gartner currently has an employee act as ombudsman to handle disagreements. The conflict of interest is self-evident in the way ZL’s concerns were summarily dismissed with little supporting evidence. ZL believes that Gartner’s immense heft and power in the marketplace necessitate careful checks and balances against abuse of power. ZL believes that if IT innovation is to remain a driver for the US economy, there must be assurances that ratings agencies such as Gartner do not subvert the competitive forces which drive innovation.

I remember a long time ago CA boycotted all Gartner research after some research-related dispute. It certain did nothing to help them: picking a fight with the movie critics always seems a risky strategy for a producer.

But it is hard to argue that Magic Quadrants are good for competition. They are inherently subjective in their assessments, they two-dimensionalize an N-dimensional problem, they encourage mental laziness on the part of customers, and –- heck –- some of us work in sectors that don’t even have a magic quadrant. What’s worse, ZL? Getting a poor ranking on an existing quadrant, or selling in a software category that Gartner doesn’t even recognize?

Since I think it’s fun to read court filings (when I have the time), let’s dig down a little deeper. The court documents are here, and I’ll embed them along the way as well. Here’s the initial complaint.

One of the many arguments made in the complaint is that Gartner doesn’t do “a single minute of independent testing of the products it purports to evaluate.” When I was younger in my career, I used to buy that argument. As I gotten older, I now realize (think: Wisdom of Crowds) that it is indeed possible to get a pretty good picture of
a product’s strengths and weaknesses simply by talking to lots of people who use it.

And that’s what Gartner does. Yes, there are no guys in lab coats doing Consumer Reports style testing. But, sometimes the guys in the lab coats measure the wrong things anyway. So while Gartner does not, to my knowledge, do hands-on testing, they neither claim to do so nor, in my estimation, is such testing strictly necessary to develop an informed opinion.

That said, a pathological case of that research model is when a vendor has very small market share. If research is done primarily through talking and there’s no one to talk to, then you’re not going to get on the map very easily.

On the other hand, I love their brass tacks description of the reality behind being labeled a “niche player”:

These MQ placements were, and are, derogatory because they are understood by technology purchasers as a warning, by Gartner, that ZL and ZL products are not good choices for enterprise email archive applications.

Yep.

Also of interest was this statement by Gartner’s ombudsman:

My sense is that there has been a relationship issue for many years with [archiving analyst] Carolyn DiCenzio and at this point it’s come down to level of trust and respect.”

I suppose there’s some logical consistency at least — if you’re going to declare war on the #1 analyst firm, well, why not make it personal as well. :-)

Let’s move on. Here’s Gartner’s response, a motion to dismiss, which is much tougher reading and more techno-legal:

Clearly, Gartner’s response is based on opinion and freedom of speech. Excerpt:

Whether plaintiff’s opinions about its product are correct, comprehensible or sincere has no legal significance; what matters is that the Complaint fails to state a claim because it attacks opinions expressed by Gartner, Inc. These opinions are constitutionally protected, in part to discourage lawsuits like this one, which are aimed at chilling the free expression of ideas and opinions.

While Gartner marketing may not love that response (imagine: “could we please defend the research as well as our right to have opinions?”) it’s not a terribly surprising one.

If nothing else, you can being to see why lawsuits cost so much money. Bear in mind the legal meters are probably running at $600/hour and they’re still debating whether the case should be immediately thrown out: it’s like dropping $20K standing on the starting line fighting about the start time for the race.

Here is ZL’s opposition to Gartner’s motion for dismissal, another 32-pages:

If you didn’t jump into the document above, let me pull out the first zinging paragraph (bolding mine):

This is a commercial case about a dominant industry player’s baseless defamation of an independent startup whose growth prospects have been crushed by the defendant’s unfair business practices. Defendant Gartner, Inc. (“Gartner”), which advises businesses on information technology decisions, exercises hegemonic control over the purchases made by a wide swath of the international corporate and governmental market. The technology Gartner says to buy is bought; what Gartner says not to buy languishes unsold, leaving its developers scrambling for the leftover market share Gartner does not dictate. The problem arises when Gartner exercises its market power recklessly, maliciously or—because of its tremendous influence—negligently. When that occurs, as it has here, innovation and competition are stifled, to the detriment of small companies who lack the resources to challenge Gartner, and to the consuming public at large.

Wow, someone turned up the rhetoric meter! At this point things are quickly getting over my legal head. There arguments seem to be largely about what is fact vs. opinion. Since I’m unable to comment on the legal issue, I’ll move on.

Finally, for the strong of legal stomach, here is Gartner’s reply to the opposition t
o the motion to dismiss. (Say that ten times fast.)

Here’s a nice summary of the counter-argument:

Try as it might, ZL cannot create a dispute where there is none. ZL alleges at great length in its Complaint (and recapitulates in its Opposition) that it has a strong product and satisfied customers. The Magic Quadrant reports do not say otherwise; the real point of contention here is not the quality of ZL’s product, but instead the subjective analytical model Gartner used to assess ZL’s market position and prospects. ZL does not contest Gartner’s basic assessments of ZL—that it has a good product but needs to expand its sales and marketing—but ZL challenges its placement on the Magic Quadrant Report because Gartner uses a “misguided analytical model” that gives “undue weight to sales and marketing.”

I have no idea how this will end. Will it be quickly thrown out of court? Will it a long drawn-out case? I don’t know. I would say that Gartner’s quadrants wield enormous power and that vendors go to great lengths to maxmize their position on them. And I’d say that you can’t judge a vendor by the quality of its technology alone. While Ingres arguably had the best database technology in the 1980s, Oracle’s sales and marketing prowess caused it to win the market and any analyst who — focused solely on the technology — would have recommended Ingres at that time would have done his customers a disservice.

I don’t know how the movie here ends, but I at least expect it to be interesting.