Category Archives: SaaS

Win Rates, Close Rates and Milestone vs. Cohort Analysis

Posted on November 9, 2017 | 6 comments

Hey, what’s your win rate?

It’s another seemingly simple question.  But, like most SaaS metrics, when you dig deeper you find it’s not.  In this post we’ll take a look at how to calculate win rates and use win rates to introduce the broader concept of milestone vs. flow analysis that applies to conversion rates across the entire sales funnel.

Let’s start with some assumptions.  Once an opportunity is accepted by sales (known as a sales-accepted opportunity, or SAL), it eventually will end up in one of three terminal states:

  • Won
  • Lost
  • Other (derailed, no decision)

Some people don’t like “other” and insist that opportunities should be exclusively either won or lost and that other is an unnecessary form of lost which should be tracked with a lost reason code as opposed to its own state.  I prefer to keep other, and call it derailed, because a competitive loss is conceptually different from a project cancellation, major delay, loss of sponsor, or a company acquisition that halts the project.  Whether you want to call it other, no decision, or derailed, I think having a third terminal state is warranted from first principles.  However, it can make things complicated.

For example, you’ll need to calculate win rates two ways:

  • Win rate, narrow = wins / (wins + losses)
  • Win rate, broad = wins / (wins + losses + derails)

Your narrow win rate tells you how good you are at beating the competition.  Your broad rates tells you how good you are at closing deals (that come to a terminal state).

Narrow win rate alone can be misleading.  If I told you a company had a 66% win rate, you might be tempted to say “time to add more salespeople and scale this thing up.”  If I told you they got the 66% win rate by derailing 94 out of every 100 opportunities it generated, won 4, and lost the other 2, then you’d say “not so fast.”  This, of course, would show up in the broad win rate of 4%.

This brings up the important question of timing.  Both these win rate calculations ignore deals that push out of a quarter.  So another degenerate case is a situation where you win 4, lose 2, derail 4, and push 90 opportunities.  In this case, narrow win rate = 66% and broad win rate = 40%.  Neither is shining a light on the problem (which, if it happens continuously, I call a rolling hairball problem.)

The issue here is thus far we’ve been performing what I call a milestone analysis.  In effect, we put observers by the side of the road at various milestones (created, won, lost, derailed) and ask them to count the number opportunities that pass by each quarter.  The issue, especially with companies that have long sales cycles, is that you have no idea of progression.  You don’t know if the opportunities that passed “win” this quarter came from the opportunities that passed “created” this quarter, or if they came from last quarter, the quarter before that, or even earlier.

Milestone analysis has two key advantages

  • It’s easy — you just need to count opportunities passing milestones
  • It’s instant — you don’t have to wait to see how things play out to generate answers

The big disadvantage is it can be misleading, because the opportunities hitting a terminal state this quarter were generated in many different time periods.  For a company with an average 9 month sales cycle, the opportunities hitting a terminal state in quarter N, were generated primarily in quarter N-3, but with some coming in quarters N-2 and N-1 and some coming in quarters N-4 and N-5.  Across that period very little was constant, for example, marketing programs and messages changed.  So a marketing effectiveness analysis would be very difficult when approached this way.

For those sorts of questions, I think it’s far better to do a cohort-based analysis, which I call a flow analysis.  Instead of looking at all the opportunities that hit a terminal state in a given time period, you go back in time, grab a cohort of opportunities (e.g., all those generated in 4Q16) and then see how they play out over time.  You go with the flow.

For marketing programs effectiveness, this is the only way to do it.  Instead of a time-based cohort, you’d take a programs-based cohort (e.g., all the opportunities generated by marketing program X), see how they play out, and then compare various programs in terms of effectiveness.

The big downside of flow analysis is you end up analyzing ancient history.  For example, if you have a 9 month average sales cycle with a wide distribution around the mean, you may need to wait 15-18 months before the vast majority of the opportunities hit a terminal state.  If you analyze too early, too many opportunities are still open.  But if you put off analysis then you may get important information, but too late.

You can compress the time window by analyzing programs effectiveness not to sales outcomes but to important steps along the funnel.  That way you could compare two programs on the basis of their ability to generate MQLs or SALs, but you still wouldn’t know whether and at what relative rate they generate actual customers.  So you could end up doubling down on a program that generates a lot of interest, but not a lot of deals.

Back to our original topic, the same concept comes up in analyzing win rates.  Regardless of which win rate you’re calculating, at most companies you’re calculating it on a milestone basis.  I find milestone-based win rates more volatile and less accurate that a flow-based SAL-to-close rate.  For example, if I were building a marketing funnel to determine how many deals I need to hit next year’s number, I’d want to use a SAL-to-close rate, not a win rate, to do so.  Why?  SAL-to-close rates:

  • Are less volatile because they’re damped by using long periods of time.
  • Are more accurate because they actually tracking what you care about — if I get 100 opportunities, how many close within a given time period.
  • Automatically factor in derails and slips (the former are ignored in the narrow win rate and the latter ignored in both the narrow and broad win rates).

Let’s look at an example.  Here’s a chart that tracks 20 opportunities, 10 generated in 1Q17 and 10 generated in 2Q17, through their entire lifetime to a terminal stage.

oppty tracking

In reality things are a lot more complicated than this picture because you have opportunities still being generated in 3Q17 through 4Q18 and you’ll have opportunities that are still in play generated in numerous quarters before 1Q17.  But to keep things simple, let’s just analyze this little slice of the world.  Let’s do a milestone-based win/loss analysis.

win-loss

First, you can see the milestone-based win/loss rates bounce around a lot.  Here it’s due in part due to law of small numbers, but I do see similar volatility in real life — in my experience win rates bounce within a fairly broad zone — so I think it’s a real issue.  Regardless of that, what’s indisputable is that in this example, this is how things will look to the milestone-based win/loss analyzer.  Not a very clear picture — and a lot to panic about in 4Q17.

Let’s look at what a flow-based cohort analysis produces.

cohort1

In this case, we analyze the cohort of opportunities generated in the year-ago quarter.  Since we only generate opportunities in two quarters, 1Q17 and 2Q17, we only have two cohorts to analyze, and we get only two sets of numbers.  The thin blue box shows in opportunity tracking chart shows the data summarized in the 1Q18 column and the thin orange box shows the data for the 2Q18 column.  Both boxes depict how 3 opportunities in each cohort are still open at the end of the analysis period (imagine you did the 1Q18 analysis in 1Q18) and haven’t come to final resolution.  The cohorts both produce a 50% narrow win rate, a 43% vs. 29% broad win rate, and a 30% vs. 20% close rate.  How good are these numbers?

Well, in our example, we have the luxury of finding the true rates by letting the six open opportunities close out over time.  By doing a flow-based analysis in 4Q18 of the 1H17 cohort, we can see that our true narrow win rate is 57%, our true broad win rate is 40%, and our close rate is also 40% (which, once everything has arrived at a terminal state, is definitionally identical to the broad win rate).

cohort7

Hopefully this post has helped you think about your funnel differently by introducing the concept of milestone- vs. flow-based analysis and by demonstrating how the same business situation results in a very different rates depending on both the choice of win rate and analysis type.

Please note that the math in this example backed me into a 40% close rate which is about double what I believe is the benchmark in enterprise software — I think 20 to 25% is a more normal range. 

 

6 Comments

Posted in Marketing, Metrics, SaaS, Sales, Startups, Uncategorized

Kellblog (Dave Kellogg) Featured on the Official SaaStr Podcast

Posted on September 18, 2017 | Leave a comment

Just a quick post to highlight the fact that last week I was the featured guest on Episode 142 of the Official SaaStr  podcast produced by the SaaStr organization run by Jason Lemkin and interviewed by a delightful young Englishman named Harry Stebbings (who also runs his own podcast entitled The Twenty Minute VC).

In the 31-minute episode — which Harry very nicely says was “probably one of his favorite interviews to record” — we cover a wide range of my favorite topics, including:

    • How I got introduced to SaaS, including my experience as an early customer of Salesforce in about 2003.
    • Challenges in scaling a software business, learned at BusinessObjects as we scaled from $30M to $1B in revenues, as well as at MarkLogic and Host Analytics.
    • My favorite SaaS metric.  If you had to pick one, I’d pick LTV/CAC.
    • Why simple churn is the best way to value the annuity of a SaaS business.
    • The loose coupling of customer satisfaction and renewal rates.
    • Why SaaS companies need to “chew gum and walk at the same time” when it comes to driving the mix of new and renewal business.
    • User-based vs. usage-based pricing in SaaS and how the latter can backfire in disincenting usage of the application.
    • My thoughts on bookings vs. ARR as a SaaS metric.  (Bookings is generally seen as a four-letter word!)
    • Why SaaS companies should make “the leaky bucket” the first four lines of their financial presentation.
    • Why I think it’s a win/win when a SaaS company gives a multi-year prepaid discount that’s less than its churn rate.
    • Why I view non-prepaid, multi-year deals as basically equivalent to renewals (just collected by finance/legal instead of customer success.)
    • Why it’s OK to “double compensate” sales and customer success on renewals and incidental upsells, and why it’s OK to pay sales on non-incidental upsells to existing customers (don’t put your farmer against someone else’s hunter).
    • Why you can’t analyze churn by analyzing churn and why you should have a rigorous taxonomy of churn.
    • My responses to Harry’s “quick fire” round questions.

You can listen to the podcast via iTunes, here.  Enjoy!

 

Leave a comment

Posted in Management, Metrics, SaaS, Startups, VC

Tagged

Detecting and Eliminating the Rolling Hairballs in your Sales Pipeline

Posted on July 20, 2017 | 5 comments

Quick:  what’s the biggest deal in this quarter’s sales pipeline?  Was that the biggest deal in last quarter’s pipeline?  How about the quarter before?  Do you have deals in your pipeline older than your children?

If you’re answering yes to these questions, then you’re probably dealing with “rolling hairballs” in your pipeline.  Rolling hairballs are bad:

  • They exaggerate the size of the pipeline.
  • They distort coverage and conversion ratios.
  • They mess up expected-value forecasts, like a forecast-category or stage-weighted sales forecast.

Maybe they’re real deals; maybe they’re figments of a rep’s imagination.  But, if you’re not careful, they pollute your pipeline and your metrics.

Let’s define a rolling hairball

A rolling hairball is a typically large opportunity that sits in your current-quarter pipeline every quarter, with a close date that slips every quarter.  At 2 quarters it’s a suspected rolling hairball; at 3 or more quarters it’s a confirmed one.

Rolling Hairball Detection

The first thing you need to do is find rolling hairballs.  They’re tricky because salesreps always swear they’re real deals that are supposed to finally close this quarter.  What makes rolling hairballs obvious is their ever-sliding close dates.  What makes them dangerous is their size (including an accumulation of them that aggregate to a material fraction of the pipeline).

If you want to find rolling hairballs, look for opportunities in the current-quarter pipeline that were also in last-quarter’s pipeline.  That will find numerous bona fide slipped deals, but it will also light-up potential rolling hairballs.  To determine if an opportunity is  a rolling hairball, for sure, you can do one of two things:

  • See if it also appeared in the current-quarter pipeline in any quarters prior to the previous one.
  • Look at its stage or forecast category.  If either of those suggest it won’t be closing this quarter, it’s another big hairball indicator.

The more sophisticated way to find them is to examine “stuck opportunity” reports that light-up deals that are moving through pipeline stages too slowly compared to your norms.

But typically, the hairball is a big opportunity hiding in plain sight.  You know it was in last quarter’s pipeline and the quarter before that.  You’ve just been deluded into believing it’s not a hairball.

Fixing Rolling Hairballs

There are two ways to fix rolling hairballs:

  • Fix the close date.  Reps are subtly incented to put deals in the current quarter (e.g., to show they’re working on something, to show they might bring in some big sales this quarter). The manager needs to get on the phone with the customer and, after having verified it’s a real opportunity, get the real timeframe in which it might close.  Assigning a realistic close date to the opportunity makes your pipeline more real and reminds the rep that they need to be working on other shorter-term opportunities as well.  (There is no mid-term if you fail enough in the short term.)  The deal will still remain in the all-quarters pipeline, but it won’t always be in the current-quarter pipeline, ever-sliding, and distorting metrics and ratios.

 

  • Fix the size. While a realistic close date is the best solution, what makes rolling hairballs dangerous is their size.  So, if the salesrep really believes it’s a current-quarter opportunity, you can either reduce its size or split it into two opportunities (particularly if that’s a possible outcome), a small one in the current quarter along with an upsell in the future.  Note that this approach can be dangerous, with lots of little hairball-lets flying below radar, so you should only try if it you’re sure your salesops team can produce the reports to find them and if you believe it reflects real customer buying patterns.

Don’t let rolling hairballs pollute your pipeline metrics and ratios.  Admit they exist, find them, and fix them.  Your sales and sales forecasting will be more consistent as a result.

5 Comments

Posted in Management, SaaS, Sales, Silicon Valley, Startups

A Look at the Tintri S-1

Posted on July 6, 2017 | 2 comments

Every now and then I take a dive into an S-1 to see what clears the current, ever-changing bar for going public.  After a somewhat rocky IPO process, Tintri went public June 30 after cutting the IPO offering price and has traded flat thus far since then.

Let’s read an excerpt from this Business Insider story before taking a look at the numbers.

Before going public, Tintri had raised $260 million from venture investors and was valued at $800 million.

With the performance of this IPO, the company is now valued at about about $231 million, based on $7.50 a share and its roughly 31 million outstanding shares, (if the IPO’s bankers don’t buy their optional, additional roughly 1.3 million shares.)

In other words, this IPO killed a good $570 million of the company’s value.

In other words, Tintri looks like a “down-round IPO” (or an “IPO of last resort“) — something that frankly almost never happened before the recent mid/late stage private valuation bubble of the past 4 years.

Let’s look at some numbers.

tintri p+l

Of note:

  • $125M in FY2017 revenue.  (They have scale, but this is not a SaaS company so the revenue is mostly non-recurring, making it easier to get to grow quickly and making the revenue is worth less because only the support/maintenance component of it renews each year.)
  • 45% YoY total revenue growth.  (On the low side, especially given that they have a traditional license/maintenance model and recognize revenue on shipment.)
  • 65% gross margins  (Low, but they do seem to sell flash memory hardware as part of their storage solutions.)
  • 87% of revenue spent on S&M (High, again particularly for a non-SaaS company.)
  • 43% of revenue spent on R&D  (High, but usually seen as a good thing if you view the R&D money as well spent.)
  • -81% operating margins (Low, particularly for a non-SaaS company.)
  • -$70.4M in cashflow from operating activities in 2017 ($17M average quarterly cash burn from operations)
  • Incremental S&M / incremental product revenue = 73%, so they’re buying $1 worth of incremental (YoY) revenue for an incremental 73 cents in S&M.  Expensive but better than some.

Overall, my impression is of an on-premises (and to a lesser extent, hardware) company in SaaS clothing — i.e., Tintri’s metrics look like a SaaS company, but they aren’t so they should look better.  SaaS company metrics typically look worse than traditional software companies for two reasons:  (1) revenue growth is depressed by the need to amortize revenue over the course of the subscription and (2) subscriptions companies are willing to spend more on S&M to acquire a customer because of the recurring nature of a subscription.

Concretely, if you compare two 100-unit customers, the SaaS customer is worth twice the license/maintenance customer over 5 years.

saas compare

Moreover, even if Tintri were a SaaS company, it is quite out of compliance with the Rule of 40, that says growth rate + operating margin >= 40%.  In Tintri’s case, we get -35%, 45% growth plus -81% operating margin, so they’re 75 points off the rule.

Other Notes

  • 1250+ customers
  • 21 of the Fortune 100
  • 527 employees as of 1/31/17
  • CEO 2017 cash compensation $525K
  • CFO 2017 cash compensation $330K
  • Issued special retention stock grants in May 2017 that vest in the two years following an IPO
  • Did option repricing in May 2017 to $2.28/share down from weighted average exercise price of $4.05.
  • $260M in capital raised prior to IPO
  • Loans to CFO and CEO to exercise stock options at 1.6% to 1.9% interest in 2013
  • NEA 22.7% ownership prior to opening
  • Lightspeed 14.5% ownership
  • Insight Venture Partners 20.2% ownership
  • Silver Lake 20.4% ownership
  • CEO 3.8% ownership
  • CFO 0.7% ownership
  • $48.9M in long-term debt
  • $13.8M in 2017 stock-based compensation expense

Overall, and see my disclaimers, but this is one that I’ll be passing on.

 

2 Comments

Posted in Entrepreneurship, IPO, Private Equity, SaaS, Startups, Uncategorized, VC

Tagged

The Role of Professional Services in a SaaS Business

Posted on March 19, 2017 | 16 comments

I love to create reductionist mission statements for various departments in a company.  These are designed to be ultra-compact and potentially provocative.  My two favorite examples thus far:

I like to make them based on real-life situations, e.g., when someone running a department seems confused about the real purpose of their team.

For example, some police-oriented HR departments seem to think their mission is protect employees from management.  Think: “Freeze, you can’t send an email like that; put your hands in the air and step away from the keyboard!”

I think otherwise. If the HR team conceptualizes itself as “helping managers manage,” it will be more positively focused, help deliver better results, and be a better business partner — all while protecting employees from bad managers (after all, mistreating employees is bad management).

Over the past year, I’ve developed one of these pithy mission statements for professional services, also known as consulting, the (typically billable) experts employed by a software company who work with customers on implementations after the sale:

Professional services exists to maximize ARR while not losing money.

Maximizing ARR surprises some people.  Why say that in the context of professional services?  Sales brings in new ARR.  Customer Success (or Customers for Life) is reponsible for the maintenance and expansion of existing ARR.  Where does professional services fit in?  Shouldn’t they exist to drive successful implementations or to achieve services revenue targets?  Yes, but that’s actually secondary to the primary mission.

The point of a SaaS business is to maxmize enterprise value and that value is a function of ARR.  If you could maximize ARR without a professional services team then you wouldn’t have one at all (and some SaaS firms don’t).  But if you’re going to have a professional services team, then they — like everybody else — should be there to maximize ARR.  How does professional services help maximize ARR?  They:

  • Help drive new ARR by supporting sales — for example, working with sales to draft a statement of work and by building confidence that the company can solve the customer’s problem.  If you remember that customers buy “holes, not bits” you’ll know that a SaaS subscription, by itself, doesn’t solve any business problem.  The importance of the consultants who do the solution mapping is paramount.
  • Help preserve/expand existing ARR by supporting the Customer Success (aka, the Customers for Life) team, either by repairing blown implementations or by doing new or expanded implementations at existing customers.  This could entail anything from a “save” to a simple expansion, but either way, professional services is there maximizing ARR.
  • Help do both by enabling the partner ecosystem.  Professional services is key to enabling partners who can both provide quality implementation services for customers and who can extend the vendor’s reach through go-to-market partnering.

Or, as our SVP of Services says, “our role is to make happy customers.”

I prefer to say “maximize ARR without losing money” but we’re very much on the same page.  Let’s finish with the “not losing money” part.  In my opinion,

  • A typical on-premises software vendor drove 25% to 30% gross margins on professional services.  Those were the days of one big one-shot license fees and huge multi-million dollar implementations.  In those days, customers weren’t necessarily too happy but the services team had a strong “make money” aspect to its mission.
  • A typical SaaS vendor has negative 10% to 20% gross margins on services (and sometimes a lot more negative than that).  That’s because some vendors subsidize their ARR with free or heavily discounted services because ARR recurs whereas services do not.

I believe that professional services has real value (e.g., I’ve worked with several amazing services teams) and that if you’re driving 0% to 5% gross margins with such a team that you are already supporting the ARR pool with discounted services (you could be running 25% to 30% margins).  Whether you make 0% or 10% doesn’t much matter — because it won’t to someone valuing your company — but I think it’s a mistake to shoot for the 30% margins of yore as well as a mistake to tolerate -50% margins and completely de-value your services.

16 Comments

Posted in HR, Management, Marketing, SaaS, Startups

Tagged