The ways VCs can use metrics in discussions with founders and CEOs.
A deep dive into CAC payback period (CPP) itself, how it’s defined, what it measures, and how its often “corrected.”
How investors like compound metrics (e.g., CPP, Rule of 40) whereas operators are best focused on atomic metrics — e.g., you should set accountability and OKRs around atomic metrics.
How some metrics are stealthly more compound that you might think — e.g., CAC based on net-new ARR or gross profit (or both).
Why I like to say, “you can’t fix a CAC payback period.” It’s a compound metric which can be driven by at least 5 different factors.
How to apply my observations to everyday SaaS life.
The slides are below. Thanks to Ray Rike for inviting me to the palooza!
The other day I was looking at an ARR bridge [1] with a young finance ace. He made a few comments and concluded with, “and net revenue retention (NRR) is thus 112%, not bad.”
I thought, “Wait, stop! You can’t calculate NRR from an ARR bridge [2].” It’s a cohort-based measure. You need to look at the year-ago cohort of customers, get their year-ago ARR, get that same group’s current ARR, and then divide the current ARR by the year-ago.
“Yes, you can,” he said. “Just take starting ARR, add net expansion, and divide by starting ARR. Voila.”
Expecto patronum! Protect me from this dark magic. I don’t know what that is, I thought, but that’s not NRR.
Then I stewed on it for a bit. In some ways, we were both right.
Under the right circumstances, I think you can calculate NRR using an ARR bridge [3]. But the whole beauty of the metric is to float over that definitional swamp and just divide two numbers — so I inherently don’t want to.
My friend’s definition, one I suspect is common in finance whiz circles, was indeed one shortcut too short. But, under the right circumstances, you can improve it to work better in certain cases.
The Trouble with Churn Rates
For a long time, I’ve been skeptical of calculations related to churn rates. While my primary problems with churn rates were in the denominator [4], there are also potential problems with the numerator [5]. Worse yet, once churn rates get polluted, all downstream metrics get polluted along with them – e.g., customer lifetime (LT), lifetime value (LTV), and ergo LTV/CAC. Those are key metrics to measure the value of the installed base — but they rely on churn rates which are easily gamed and polluted.
What if there were a better way to measure the value of the installed base?
There is. That’s why my SaaStr 2019 session title was Churn is Dead, Long Live Net Dollar Retention [6]. The beauty of NRR is that it tells you want you want to know – once you acquire customers, what happens to them? – and you don’t have to care which of four churn rates were used. Or how churn ARR itself was defined. Or if mistakes were made in tracking flows.
You just need to know two things: ARR-now and ARR-then for “then” cohort of customers [7].
A Traditional ARR Bridge
To make our point, let’s review a traditional ARR bridge.
Nothing fancy here. Starting ARR plus new ARR of two types: new logo customers (aka, new logo ARR) and existing customers (aka, expansion ARR). We could have broken churn ARR into two types as well (shrinkage and lost), but we didn’t need that breakout for this exercise.
Now, let’s add my four favorite rows to put beneath an ARR bridge [8]:
Here’s a description:
Net new ARR = New ARR – churn ARR. How much the water level increased in the SaaS leaky bucket of ARR. Here in 1Q21, imagine we spent $2,250K in S&M in the prior quarter. Our CAC ratio would be a healthy 1.0 on a new ARR basis, but a far less healthy 2.1 on a net new ARR basis. That’s due to our quarterly churn of 8%, which when annualized to 32%, flies off the charts.
Expansion as a percent of new ARR = expansion ARR / new ARR. My sense is 30% is a magic number for an established growth-phase startup. If you’re only at 10%, you’re likely missing the chance to expand your customers (which will also show up in NRR). If you’re at 50%, I wonder why you can’t sell more new logo customers. Has something changed in the market or the salesforce?
Net expansion = expansion ARR – churn ARR. Shows the net expansion or contraction of the customer base during the quarter. How much of the bucket increase was due to existing (as opposed to new) customers?
Churn rate, quarterly. I included this primarily because it raises a point we’ll hit when discussing lazy NRR. Many people calculate this as = churn ARR / starting ARR (quarter). That’s what I call “simple quarterly,” and you’ll note that it’s always lower than just “quarterly,” which I define as = churn ARR / starting ARR (year) [9]. The trace-precedents arrows below show the difference.
Lazy NRR vs. Cohort-Based NRR
With that as a rather extensive warm-up, let’s discuss what I call lazy NRR.
Lazy NRR is calculated as described above = (starting ARR + net expansion) / starting ARR. Lazy NRR is a quarterly expansion metric.
Let’s look at a detailed example to see what’s really being measured.
This example shows the difference between cohort-based NRR and Lazy NRR:
Cohort-based NRR, a year-over-year metric that shows expansion of the two year-ago customers (customers 1 and 2). This is, in my book, “real NRR.”
Lazy NRR, simple quarterly, which compares net expansion within the current quarter to starting ARR for that quarter.
The point of the trace-precendents arrows shows you that while the result coincidentally might be similiar (and in this case it is not), that they are measuring two completely different things.
Let’s talk about the last row, lazy NRR, cohort-based approximation, which takes starting ARR from year-ago customers, then adds (all) net expansion over the year and divides by the year-ago starting ARR. The problem? Customer 3. They are not in the year-ago cohort, but contribute expansion to the numerator because, with only an ARR bridge, you can’t separate year-ago cohort net expansion from new-customer net expansion. To do that, you’d need to have ARR by customer [10].
Lazy NRR is not NRR. NRR is defined as snapshot- and cohort-based. Accept no substitutes or imitations. Always calculate NRR using snapshots and cohorts and you’ll never go wrong.
Layer Cakes Tell No Lies
While I’m usually quite comfortable with tables of numbers and generally prefer receiving them in board reports, this is one area where I love charts, such as this layer cake that stacks annual cohorts atop each other. I like these layer cakes for several reasons:
They’re visual and show you what’s happening with annual cohorts.
Like snapshot- and cohort-based NRR, they leave little to no room for gaming. (They’re even harder to survivor bias as you’d have to omit the prior-period ARR.)
Given my now-distant geophysics background, they sometimes remind me of sedimentary rock. (Hopefully yours don’t look like that, as unmetamorphized, sedimentary rock represents an NRR of only 100%!)
(The post was revised a few times after initial publication to fix mistakes and clarify points related to the cohort-based approximation. In the end, the resultant confusion only convinced me more to only and always calcuate NRR using cohorts and snapshots.)
# # #
Notes
Edited 10/8/22 to replace screenshots and fix spreadsheet bug in prior version.
[1] Starting ARR + new ARR (from new logo and expansion) – churn ARR (from shrinkage and lost) = ending ARR
[2] I probably should have said “shouldn’t.” Turns out, I think you can, but I know you shouldn’t. We’ll elaborate on both in this post.
[3] Those conditions include a world where customers expand or contract only on an annual basis (as you are unable to exclude expansion or contraction from customers signed during the year since they’re not sepearated in an ARR bridge) and, of course, a clear and consistent definition of churn, playing fairly with no gaming designed understate churn or overstate expansion, and avoidance of mistakes in calculations.
[4] Churn rates based off the whole ARR pool can halve (or more than halve) those based on the available to renew (ATR) pool, for example if a company’s mean contract duration is 2 or 3 years. ARR churn rates are probably better for financial calculations, but ATR churn rates are a better indicator of customer satisfaction
[5] Examples of potential problems, not all strictly related to calculation of churn ARR, but presented for convenience.
Expansion along the way. Consider a customer who buys 100-unit contract, expands to 140 the next quarter (without signing a new one-year agreement that extends the contract), and then at the annual renewal renews for 130. The VP of CS wants to penalize the account’s CSM for 10 units of churn whereas the CFO wants to tell investors its 30 units of expansion. Which is it? Best answer IMHO is 40 units of expansion in the second quarter and 10 units of churn at the renewal, but I’ve seen people/systems that don’t do it that way. NRR sees 130% rate regardless of how you count expansion and churn.
Potential offsets and the definition of customer – division 1 has 100 units and shrinks to 80 at renewal while a small 40-unit new project starts at division 2. Is that two customers, one with 20 units of churn and one new 40-unit customer or is it one customer with 20 units of expansion? NRR sees either 80% rate or 120% rate as function of customer definition, but I’d hope the NRR framing would make you challenge yourself to ask: was division 2 really a customer and ergo belong in the year-ago cohort?
Potential offsets and the definition of product – a customer has 100 units of product A, is unhappy, and shrinks to A to 60 units while buying your new product B for 40. Did any churn happen? In most systems, the answer is no because churn is calculated at the account level. Unless you’re also tracking product-level churn, you might have trouble seeing that your new product is simply being given away to placate customers unhappy with your first one. NRR is inherently account-level and doesn’t solve this problem – unless you decide to calculate product-level NRR, to see which products are expanding and which are shrinking.
Adjustments. International companies need to adjust ARR for fluctuations in exchange rates. Some companies adjust ARR for bad debt or non-standard contracts. Any and all of these adjustments complicate the calculation of churn ARR and churn rates.
Gaming. Counting trials as new customers and new ARR, but excluding customers <$5K from churn ARR calculations (things won’t foot but few people check). Renewing would-be churning customers at $1 for two years to delay count-based churn reporting (ARR churn rates and NRR will see through this). Survivor biasing calculations by excluding discontinuing customers. Deferring ARR churn by renewing would-be churning customers with net 360 payables and a handshake (e.g., side letter) to not collect unless thing XYZ can be addressed (NRR won’t see through this, but cash and revenue won’t align).
[6] Since I now work frequently with Europe as part of my EIR job with Balderton Capital, I increasingly say “NRR” instead of “NDR” (net dollar retention), because for many of the companies I work with it’s actually net Euro retention. The intent of “dollar” was never to indicate a currency, but instead to say: “ARR-based, not count-based.” NRR accomplishes that.
[8] If you download the spreadsheet and expand the data groups you can see some other interesting rows as well.
[9] The flaw in “simple quarterly” churn is that, in a world that assumes pure annual contracts, you’re including people who were not customers at the start of the year and ergo cannot possibly churn in the calculations. While you use the same numerator in each case, you’re using an increasing denominator and with no valid reason for doing so. See here for more.
[10] In which case you might as well calculate NRR as defined, using the current and year-ago snapshots.
Question: what do you call a 10-year old startup with $10M in ARR? Answer: a small business [1].
When you make a list of key SaaS metrics, you’ll rarely find age listed among them. That’s correct in the sense that age by itself tells you little, but when size is measured against age, you get a rough measure of velocity.
It’s a lot like people. Tell me you can play Mozart’s Piano Concerto No. 23 and I’ll be impressed [2]. Tell me you can play it at age 12, and I’ll think you’re an absolute prodigy. Tell me you have $10M in ARR after 10 years and I’ll be impressed [3]. Tell me you have it 3 and I’ll run for my checkbook.
All this begs the question of growth velocity: at what age is a given size impressive? Towards that end, and working with my friends at Balderton Capital, I’ve come up with what I’m calling the Rule of 56789.
5 years to break $10M
6 years to break $20M
7 years to break $50M
8 years to break $75M
9 years to break $100M
Concretely put, if you walk through the doors to Balderton’s London offices with $54M in ARR after 7 years, you’ll be in the top quartile of those who have walked before you.
Commentary
I’m effectively defining “impressive” as top quartile in the Balderton universe of companies [4].
Remembering 56789 is easy, but remembering the milestones is harder. Once you commit the series {10, 20, 50, 75, 100} to memory, it seems to stick [5].
Remember that these are milestones to pass, not ending ARR targets, so this is not equivalent to saying grow 100% from $10M to $20M, 150% from $20 to $50M, and so on. See note [6] before concluding {100%, 150%, 50%, 33%} is an odd growth trajectory.
For example, this is a 56789-compliant growth trajectory that has no whipsawing in growth rates.
Three Situtions That Break The Rule
Rules are made to be broken, so let’s talk about three common situations which confound the Rule of 56789.
Bootstraps, which are capital constrained and grow more slowly. Bootstraps should largely ignore the rule (unless they plan on changing their financing strategy) because they are definitionally not trying to impress venture capitalists [7].
Platforms, that require years of time and millions of dollars before they can go to market, effectively resetting the starting clock from company inception to beta product release [8].
Pivots, where a company pursues strategy A for a few years, abandons it, and takes some salvage value over to a new strategy B. This effectively resets the starting clock from inception to pivot [9].
Alternative Growth Velocity Rules
Let’s compare the trajectory we showed above to similar one generated using a slightly different rule, which I’ll call the 85% Growth Retention Rule, which says to be “impressive” (as defined above), you should:
Pass $1M in ARR at a high growth rate (e.g., above ~180%)
Subsequently retain 85% of that growth rate every year
I view these as roughly equivalent rules, or more precisely, alternate expressions of nearly the same underlying rule. I prefer 56789 because it’s more concrete (i.e., do X by Y), but I think 85% growth retention is somewhat more general because it says no matter where you are and how you got there, try to retain 85% (or more) of your growth rate every year. That said, I think it stops working at 8-10 years because the asymptote on great company growth is somewhere around 40% [10] and some would argue 60% [11]. It also fails in situations where you need to reaccelerate growth.
There’s one well-known growth velocity rule to which we should also compare. The triple/triple/double/double/double (T2D3) rule, which says that once you hit $2M in ARR, you should triple to $6M, triple again to $18M, then double three times to $36M, $72M, and $144M.
Let’s compare the 56789 and the 85% Growth Retention rules to the T2D3 rule:
Clearly T2D3 is more aggressive and sets a higher bar. My beef is that it fails to recognize the law of large numbers (by failing to back off on the growth rates as a function of size across considerable scale), so as an operator I’m more intuitively drawn to the 85% Growth Retention rule. That said, if you want to be top 5% to 10% (vs. top 25%), then go for T2D3 if you can do it [12]. You’ll clearly be creating a lot more value.
I like all of these rules because they help give you a sense for how quickly you should be getting to a certain size. Growth conversations (e.g., trying to get a CRO to sign up for a number) are never easy. Rules like these help by providing you with data not about what the average companies are doing, but what the great ones are. The ones you presumably aspire to be like.
The limitation, of course, is that none of these rules consider the cost of growth. There’s a big difference between a company that gets to $100M in 9 years on $100M in capital vs. one that does so on $400M in capital. But that’s why we have other metrics like cash conversion score. Different metrics measure different things and these ones are focused solely on size/growth vs. age.
A big tip of the hat to Michael Lavner at Balderton Capital for working with me on this post.
# # #
Notes
[1] See the definition of small business, which is somewhat broader than I’d have guessed.
[2] Even though it’s only classified as “less difficult” on this rather amazing scale from less difficult to difficult, very difficult, extremely difficult, ridiculously difficult, and extraordinarily difficult. (Perhaps CEO’s can use that scale to classify board members.)
[3] It’s not as if just anybody can do either. Founding a company and building it to $10M is impressive, regardless of the timeframe.
[4] Balderton universe = European SaaS startups who wanted to raise venture capital, who were sufficiently confident to speak with (what’s generally seen as) a top-tier European firm, and who got far enough into the process to submit performance data.
[5] I remember it by thinking that since it’s still pretty early days, jumping from $10M+ to $20M+ seems more reasonable than from $10M to $25M+.
[6] Don’t equate this rule with a growth vector of {100%, 150%, 50%, 33%} in years 5 through 9. For example, years in which companies break $10M often don’t conclude with $10.1M in ARR, but more like $15M, after having doubled from a prior year of $7 to $8M.
[7] The rule would probably be more useful in projecting the future of VC-backed competitor. (I think sometimes bootstrapped companies tend to underestimate the aggressiveness of their VC-backed competition.) This could help you say, “Well, in N years, BadCo is likely to be a $50M business, and is almost certainly trying to be. How should that affect our strategy?”
[8] That said, be sure you’re really building a mininum viable product and not overengineering either because it’s fun or it allows you to delay the scary of moment of truth when you try to sell it.
[9] Financings after a pivot sometimes require a recapitalization, in which case the company’s entire lifeclock, from strategy to product to cap table, are all effectively reset.
[10] Current median growth in Meritech Public Comps is 32% at median scale $657M in ARR.
[11] 0.85^10 = 0.2 meaning you’ll cut the starting growth rate by 80% after ten years. So if you start at 200% growth, you’ll be down to 40% after 10 years with 85% growth retention.
[12] I’ll need to take a homework assignment to figure out where in the distribution T2D3 puts you in my data set.
It was a combination of luck and foresight that I started talking with Allan Wille and Lauren Thibodeau about capital efficiency as a potential topic for their Metric Stack podcast many months ago. Because now, as the episode is coming out, capital efficiency is the hot topic of the day. Good luck (if not for a bad reason), but I’ll take it.
Here are some of the things we discussed on the podcast:
If you think of startups as organisms that convert venture capital (VC) into ARR, then we need some metric for how efficiently they do that.
I believe Bessemer defines CCS upside-down; I find it more intuitive to use capital consumed as the numerator and ARR (to show for it) in the denominator — as you would do with a CAC ratio.
< 1 is amazing (i.e., burning <$50M to get to $50M in ARR)
1-2 is good (i.e., burning $50M to $100M to get to $50M)
2-4 is questionable (i.e., burning $100M to $200M to get to $50M)
4+ is bad (i.e., burning $200M+ to get to $50M)
On IRR, Bessemer companies with a ratio of <1x had an IRR of 120%, 1-2 had an IRR of 80%, and 2-4 had an IRR of 40%.
At some point, I’d somewhat tongue-in-cheekily defined a metric called hype factor on the theory that startup organisms actually produced two things: ARR and hype.
When asked, “how is it going?” many companies will respond with something akin to, “things are looking strong, the pipeline is up to $50M.”
Not a bad statement, but certainly an imprecise one. “Over what timeframe?” you might ask. To which you’ll typically hear one of two answers
“Uh, that’s the whole thing.” I don’t love this answer as many companies –particularly the ones who answer with all-quarter pipeline — let junk opportunities get parked in the 5Q+ pipeline. (You can fix this by including a timeframe as part of the definition of opportunity and ensuring you review the entire pipeline whenever you do a pipeline scrub.)
“That’s the rolling four-quarter (R4Q) pipeline.” I don’t love this answer either because, in my experience, companies who focus on R4Q pipeline as their top pipeline metric tend not to put enough emphasis on pipeline timing. It’s too easy to say in January, “this year’s number is $20M and we’ve got $50M in the pipeline already (2.5x pipeline coverage) so we are golden.” The problem, of course, is if 80% of that pipeline is backloaded into Q4, then while “the year may look great,” you’re going to need to survive three wasteland quarters to get there. Even if that $40M Q4 pipeline were real, which it usually isn’t, most sales VPs won’t be around in October to close it.
I never look at rolling-four-quarter pipeline for the simple reason that I’ve never had a rolling-four-quarter sales target. We have quarterly targets. Instead of looking at R4Q pipeline and hoping it’s well distributed (over time and across sellers), my philosophy is the opposite:
Once you accept this viewpoint, a few things happen:
Someone needs to start forecasting day-1 next-quarter pipeline coverage. What’s the point of focusing on next-quarter coverage if no one is tracking it and taking corrective actions as needed? As mentioned, I think that person should be the CMO.
We need to start tracking the progression of the pipeline over time. This quarter’s starting pipeline is largely composed of last-quarter’s next-quarter pipeline and so on. Since there are so many ebbs and flows in the pipeline the best way to track this is via periodic snapshots.
Towards that end, here’s a chart I find useful:
Let’s examine it.
Each row is a snapshot of the pipeline, broken down by quarter, taken on the first day of the quarter. (Some allow a week or two, for pipeline cleanup before snapshotting, which is fine.)
We’re tracking pipeline dollars, not opportunity count, which generally works better if you have a range of deal sizes and/or a multi-modal distribution of average sales prices. Doing so, however, can leave you overconfident if you create new opportunities with a high placeholder value. (See this post for what to do about that.)
We show pipeline coverage in the block on the right. Most people want this-quarter coverage of around 3.0. Targets for next-quarter and N+2 quarter are usually less well understood because many people don’t track them. Coverage needed in the out quarters is a function of your sales cycle length, but the easiest thing is to just start tracking it so you get a sense for what out-quarter coverage normally is. If you’re worried about that 1.6x next-quarter coverage shown on the 7/1 snapshot, read this post for ideas on how to generate pipeline in a hurry.
It’s good to carry at least one year’s prior snapshots so you can see historical progression. Even more is better.
I’m assuming bigger deals and longer sales cycles (e.g., 6 to 12 months) so you will actually have material pipeline in the out-quarters. For a velocity model with 25-day sales cycles, I’d take this template but just switch the whole things to months.
The most fun part of this chart is this you read it diagonally. The $7.5M in starting this-quarter pipeline at the 7/1/21 snapshot is largely composed of the $6.5M in next-quarter pipeline at the 4/1/21 snapshot and the $3M in pipeline at the 1/1/21 snapshot. You can kind of see the elephant go through the snake.
When you add this chart to your mix, you’re giving yourself an early warning system for pipeline shortages beyond simply forecasting starting next-quarter pipeline. You should do this, particularly with big deals and long sales cycles, because one quarter’s notice is usually not enough time to fix the problem. Yes, you can and should always try to mitigate problems (and never give-up saying, “looks like we’re going to hit the iceberg”), but if you give yourself more advance notice, you’ll give yourself more options and a better chance at reaching the goal: starting every quarter with 3.0x coverage.
I’m Dave Kellogg, advisor, director, consultant, angel investor, and blogger focused on enterprise software startups. I am an executive-in-residence (EIR) at Balderton Capital and principal of my own eponymous consulting business.
I bring an uncommon perspective to startup challenges having 10 years’ experience at each of the CEO, CMO, and independent director levels across 10+ companies ranging in size from zero to over $1B in revenues.
From 2012 to 2018, I was CEO of cloud EPM vendor Host Analytics, where we quintupled ARR while halving customer acquisition costs in a competitive market, ultimately selling the company in a private equity transaction.
Previously, I was SVP/GM of the $500M Service Cloud business at Salesforce; CEO of NoSQL database provider MarkLogic, which we grew from zero to $80M over 6 years; and CMO at Business Objects for nearly a decade as we grew from $30M to over $1B in revenues. I started my career in technical and product marketing positions at Ingres and Versant.
I love disruption, startups, and Silicon Valley and have had the pleasure of working in varied capacities with companies including Bluecore, FloQast, GainSight, Hex, MongoDB, Pigment, Recorded Future, and Tableau.
I currently serve on the boards of Cyber Guru (cybersecurity training), Jiminny (conversation intelligence), and Scoro (work management).
I previously served on the boards of Alation (data intelligence), Aster Data (big data), Granular (agtech), Nuxeo (content services), Profisee (MDM), and SMA Technologies (workload automation).
I periodically speak to strategy and entrepreneurship classes at the Haas School of Business (UC Berkeley) and Hautes Études Commerciales de Paris (HEC).
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here:
Cookie Policy
