Thoughts on the Gartner Magic Quadrant for Corporate Performance Management Suites 2015

“Mirror mirror on the wall, who’s the fairest EPM vendor of them all?”

It’s that time of the year again.  Gartner just released their Magic Quadrant for Corporate Performance Management (CPM) Suites 2015.  At Host Analytics, we call the category Enterprise Performance Management (EPM), so I’ll refer to the market henceforth as EPM.

In this post, I’ll share some of my commentary on the most recent Gartner Magic Quadrant (MQ) for EPM.

  • Get the big picture — and that picture is cloud.  All four cloud EPM vendors are in the Visionaries quadrant.  The first two trends in the market overview are about cloud.  The word “cloud” appears 113 times in the document, more than twice as much as the word “performance” at 46.   Cloud EPM is a huge trend in the market.
  • Use their critical capabilities research for short lists.   The MQ is a lot of fun to look at, but remember that Gartner analysts generally don’t recommend using the MQ for making vendor short lists.  Because markets are broad (and suite markets particularly so), they generally recommend using their critical capabilities documents to create a short list of which vendors are most likely to meet your specific needs.
  • Avoid dot-vector analysis.  Remember that Gartner analysts view the quadrant as re-created from scratch every year.  Vendors tend to obsess with dot-movement vectors, but when I ask the Gartner team about this issue, they tell me that the CPM MQ is effectively recreated from scratch each year and thus all about positions, not movements.  I know it’s hard to resist the temptation (heck one year at Business Objects I made an animated GIF of the MQ where the Brio dot even exploded in the end), but the folks there tell me it’s not meaningful, so I won’t do it.
  • Analyze via the football.  If you look at one-hundred Gartner MQs, you’ll see the pattern that the MQ placement algorithm generally lays out the vendors in a football shaped way.  I believe that inside the football you find the standard leaders, visionaries, challengers, etc.  Since most vendors are in the football, it’s more interesting to me to look who’s outside it and understand why.

football

  • Beware the impacts of the customer satisfaction survey.  I’ve seen and conducted customer satisfaction research on EPM vendors and what I see generally doesn’t jibe with the results that Gartner uses as an important data source in making the MQ.  By analogy, if you were trying to decide where to have dinner and I gave you the choice of two surveys, which would you pick?  Survey 1 — a random sample of diners leaving the restaurant.  Survey 2 — ask each restaurant to provide 10 customers.  You can argue that Survey 2 is “fair” because every restaurant had the same opportunity to prime, coach, offer free meals, etc., to the respondents.  But for my dinner investment, I’d pick survey 1.  The Gartner MQ survey is more like Survey 2.
  • Factor in degree of difficultly.  Another factor that influences the Gartner customer satisfaction survey is more subtle:  low-end vendors trying to solve easier problems should generally get better marks than high-end vendors trying to solve harder ones:  imagine a diving contest where we scored dives on execution but forgot to multiply by degree of difficulty.  Note that I do believe that the analysts understand this issue and try to mentally correct for it.  But the point is should I care how much QuickBooks users like QuickBooks if I’m in the league where only Oracle or SAP can solve my problem?
  • Read the text.   I sometimes think Gartner MQs are like old Playboy magazines, full of good journalism but where nobody actually reads the articles.  The MQ document is 1/35th pictures and 34/35th text.  Read the text. There’s a lot of great insight in there — where they even touch on some of the issues I’m raising above.
  • My favorite quote:  “Although the base of cloud CPM applications is still larger for small and mid-market organizations, the number of larger organizations (more than $1 billion in annual revenue) turning to the cloud for CPM continues to increase.”  This is why I joined Host Analytics 2.5 years ago.  EPM is an amazing market.  The cloud is still only scratching the surface of the market.  We are the leader in taking up-market EPM applications to the cloud.  And cloud EPM is increasingly coming up-market.

You can download a free copy of the Gartner Magic Quadrant for Corporate Performance Management Suites 2015 via Host Analytics, here.

6 responses to “Thoughts on the Gartner Magic Quadrant for Corporate Performance Management Suites 2015

  1. Dave, this is a great discussion of the MQ and captures many of the best practices we urge client to follow in getting the most value from them. I’ll add one caveat: many analysts (myself included) supplement the supplied references with our own, drawn from lists of clients who do inquiries with us (itself an admittedly skewed sample.) We don’t typically rely exclusively on supplied references. Even more interesting, many vendors do a TERRIBLE job vetting their references. I’ve had some tell me they were not in production, just test, or even “we tried and and selected XX instead.”

  2. Thanks Merv. Amazing to hear.

  3. Thanks for this analysis. It tends to reinforce my view. Surely Gartner model is ripe for disruption.

  4. Thanks for this analysis. It tends to reinforce my view. Surely the Gartner model is ripe for disruption.

  5. And some Gartner analysts do a TERRIBLE job on things like primary research, following-up on references and not offering true transparency or flexibility in applying weighting. MQs are a broad, monolithic model. MQs are more famous, but Waves offer a more transparent and flexible model. Waves, actually, were even better in their earlier days when vendors actually had to implement and demonstrate features/functions.

    Never use MQs for short list or long list. In many cases the vendor(s) who might do the best job for you didn’t make the MQ cut for one reason or another, perhaps too low in revenues in the market sector (which is a misguided filter because of the brittle manner in which Gartner counts revenue) or some other obtuse factor. If you want serious enterprise software feature/function comparison analysis in detail, TEC has them and lets you choose your weightings and categories. Gartner is just beginning to travel down this path with “critical capabilities.”

    The problem with the MQ reference model is the N is never close to high enough and the sample is never random. Gartner does not perform statistically significant customer surveys for any MQ. Think of it: the references are (a) supplied by vendors or (b) known by the analysts – both of those break the rules of creating a legitimate sample.

    Finally, agreed, the text of some MQs is often far more valuable and insightful than the simple graphic, which has become simply a marketing tool. However, some MQs are way better than others because the knowledge and skills of the analysts varying considerably from market to market. Apply the normal curve.

  6. Angel S. Viloria III

    Great MQ and GCC education – Thanks.
    And my favorite quote comes from EQ (it sums up they way in which I too often annoy people): “…the N is never close to high enough and the sample is never random.”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s