Category Archives: Marketing

Using Time-Based Close Rates to Align Marketing Budgets with Sales Targets

This post builds on my prior post, Win Rates, Close Rates, and Milestone vs. Flow Analysis.  In it, I will take the ideas in that post, expand on them a bit, and then apply them to difficult problem of ensuring you have enough marketing demand generation budget to hit your sales targets.

Let’s pretend it’s 4Q17 and that we need to model 2018 sales based solely on marketing-generated SALs (sales accepted leads).  To do that, we need to decompose our close rate over time because knowing we eventually close 40% of SALs is less useful than knowing the typical timing in how they close over time.

decompose closed

In a perfect world, we’d have 6-8 cohorts, not two.  The goal is to produce the last line, the average of the in-quarter, first-quarter, second-quarter, and so on close rates for a SAL.

Using these time-based average close rates, we can build a waterfall that takes historical, forecast (for the current quarter), and planned 2018 SALs and converts them into deals.

waterfall

This analysis suggests that with the currently planned SALs you can support an ARR number of $16.35M.  If sales needs more than that, you either need to assume an improvement in close rates or an increase in SAL generation.

Once you’ve established the required number of SALs, you can then back into a total demand-generation budget by knowing your cost/SAL, and then building out a marketing mix of programs (each with their own cost/SAL) that generates the requisite SALs at the targeted overall cost.

Win Rates, Close Rates and Milestone vs. Flow Analysis

Hey, what’s your win rate?

It’s another seemingly simple question.  But, like most SaaS metrics, when you dig deeper you find it’s not.  In this post we’ll take a look at how to calculate win rates and use win rates to introduce the broader concept of milestone vs. flow analysis that applies to conversion rates across the entire sales funnel.

Let’s start with some assumptions.  Once an opportunity is accepted by sales (known as a sales-accepted opportunity, or SAL), it eventually will end up in one of three terminal states:

  • Won
  • Lost
  • Other (derailed, no decision)

Some people don’t like “other” and insist that opportunities should be exclusively either won or lost and that other is an unnecessary form of lost which should be tracked with a lost reason code as opposed to its own state.  I prefer to keep other, and call it derailed, because a competitive loss is conceptually different from a project cancellation, major delay, loss of sponsor, or a company acquisition that halts the project.  Whether you want to call it other, no decision, or derailed, I think having a third terminal state is warranted from first principles.  However, it can make things complicated.

For example, you’ll need to calculate win rates two ways:

  • Win rate, narrow = wins / (wins + losses)
  • Win rate, broad = wins / (wins + losses + derails)

Your narrow win rate tells you how good you are at beating the competition.  Your broad rates tells you how good you are at closing deals (that come to a terminal state).

Narrow win rate alone can be misleading.  If I told you a company had a 66% win rate, you might be tempted to say “time to add more salespeople and scale this thing up.”  If I told you they got the 66% win rate by derailing 94 out of every 100 opportunities it generated, won 4, and lost the other 2, then you’d say “not so fast.”  This, of course, would show up in the broad win rate of 4%.

This brings up the important question of timing.  Both these win rate calculations ignore deals that push out of a quarter.  So another degenerate case is a situation where you win 4, lose 2, derail 4, and push 90 opportunities.  In this case, narrow win rate = 66% and broad win rate = 40%.  Neither is shining a light on the problem (which, if it happens continuously, I call a rolling hairball problem.)

The issue here is thus far we’ve been performing what I call a milestone analysis.  In effect, we put observers by the side of the road at various milestones (created, won, lost, derailed) and ask them to count the number opportunities that pass by each quarter.  The issue, especially with companies that have long sales cycles, is that you have no idea of progression.  You don’t know if the opportunities that passed “win” this quarter came from the opportunities that passed “created” this quarter, or if they came from last quarter, the quarter before that, or even earlier.

Milestone analysis has two key advantages

  • It’s easy — you just need to count opportunities passing milestones
  • It’s instant — you don’t have to wait to see how things play out to generate answers

The big disadvantage is it can be misleading, because the opportunities hitting a terminal state this quarter were generated in many different time periods.  For a company with an average 9 month sales cycle, the opportunities hitting a terminal state in quarter N, were generated primarily in quarter N-3, but with some coming in quarters N-2 and N-1 and some coming in quarters N-4 and N-5.  Across that period very little was constant, for example, marketing programs and messages changed.  So a marketing effectiveness analysis would be very difficult when approached this way.

For those sorts of questions, I think it’s far better to do a cohort-based analysis, which I call a flow analysis.  Instead of looking at all the opportunities that hit a terminal state in a given time period, you go back in time, grab a cohort of opportunities (e.g., all those generated in 4Q16) and then see how they play out over time.  You go with the flow.

For marketing programs effectiveness, this is the only way to do it.  Instead of a time-based cohort, you’d take a programs-based cohort (e.g., all the opportunities generated by marketing program X), see how they play out, and then compare various programs in terms of effectiveness.

The big downside of flow analysis is you end up analyzing ancient history.  For example, if you have a 9 month average sales cycle with a wide distribution around the mean, you may need to wait 15-18 months before the vast majority of the opportunities hit a terminal state.  If you analyze too early, too many opportunities are still open.  But if you put off analysis then you may get important information, but too late.

You can compress the time window by analyzing programs effectiveness not to sales outcomes but to important steps along the funnel.  That way you could compare two programs on the basis of their ability to generate MQLs or SALs, but you still wouldn’t know whether and at what relative rate they generate actual customers.  So you could end up doubling down on a program that generates a lot of interest, but not a lot of deals.

Back to our original topic, the same concept comes up in analyzing win rates.  Regardless of which win rate you’re calculating, at most companies you’re calculating it on a milestone basis.  I find milestone-based win rates more volatile and less accurate that a flow-based SAL-to-close rate.  For example, if I were building a marketing funnel to determine how many deals I need to hit next year’s number, I’d want to use a SAL-to-close rate, not a win rate, to do so.  Why?  SAL-to-close rates:

  • Are less volatile because they’re damped by using long periods of time.
  • Are more accurate because they actually tracking what you care about — if I get 100 opportunities, how many close within a given time period.
  • Automatically factor in derails and slips (the former are ignored in the narrow win rate and the latter ignored in both the narrow and broad win rates).

Let’s look at an example.  Here’s a chart that tracks 20 opportunities, 10 generated in 1Q17 and 10 generated in 2Q17, through their entire lifetime to a terminal stage.

oppty tracking

In reality things are a lot more complicated than this picture because you have opportunities still being generated in 3Q17 through 4Q18 and you’ll have opportunities that are still in play generated in numerous quarters before 1Q17.  But to keep things simple, let’s just analyze this little slice of the world.  Let’s do a milestone-based win/loss analysis.

win-loss

First, you can see the milestone-based win/loss rates bounce around a lot.  Here it’s due in part due to law of small numbers, but I do see similar volatility in real life — in my experience win rates bounce within a fairly broad zone — so I think it’s a real issue.  Regardless of that, what’s indisputable is that in this example, this is how things will look to the milestone-based win/loss analyzer.  Not a very clear picture — and a lot to panic about in 4Q17.

Let’s look at what a flow-based cohort analysis produces.

cohort1

In this case, we analyze the cohort of opportunities generated in the year-ago quarter.  Since we only generate opportunities in two quarters, 1Q17 and 2Q17, we only have two cohorts to analyze, and we get only two sets of numbers.  The thin blue box shows in opportunity tracking chart shows the data summarized in the 1Q18 column and the thin orange box shows the data for the 2Q18 column.  Both boxes depict how 3 opportunities in each cohort are still open at the end of the analysis period (imagine you did the 1Q18 analysis in 1Q18) and haven’t come to final resolution.  The cohorts both produce a 50% narrow win rate, a 43% vs. 29% broad win rate, and a 30% vs. 20% close rate.  How good are these numbers?

Well, in our example, we have the luxury of finding the true rates by letting the six open opportunities close out over time.  By doing a flow-based analysis in 4Q18 of the 1H17 cohort, we can see that our true narrow win rate is 57%, our true broad win rate is 40%, and our close rate is also 40% (which, once everything has arrived at a terminal state, is definitionally identical to the broad win rate).

cohort7

Hopefully this post has helped you think about your funnel differently by introducing the concept of milestone- vs. flow-based analysis and by demonstrating how the same business situation results in a very different rates depending on both the choice of win rate and analysis type.

Please note that the math in this example backed me into a 40% close rate which is about double what I believe is the benchmark in enterprise software — I think 20 to 25% is a more normal range. 

 

Just Effing Demo

I remember one time reading a win/loss report that went something like this.

“We were interested in buying Host and it made our short list.  When we invited you in for a demo with our team and the CFO, things went wrong.  After 20 minutes, your sales team was still talking about the product so the CFO left the meeting and didn’t want to evaluate your solution anymore.”

Huh?  What!  We spend a few hundred dollars to get a lead, maybe a few thousand to get it converted to a sales opportunity, we give it to our sales team and then they ‘show up and throw up’ on a prospect, talking for so long that the key decision maker leaves?

Yes, salespeople love to talk, but this can’t happen.  I remember another time a prospect called me.

“Look, I’ve been using EPM systems for 25 years.  I’ve used Hyperion, Essbase, TM1, and BPC.  I’ve been in FP&A my entire career.  I have an MBA from Columbia.  I am fully capable of determining my own needs and don’t want to play Twenty Questions with some 20-something SDR and then play it again with some sales consultant before I can get a live demo of your software.  Can we make that happen or not?”

Ouch.  In this case, our well defined and valued sales process (which required “qualification” and then “discovery”) was getting in the way of what the eminently qualified prospect wanted.

In today’s world, prospects both have and want more control over the sales process than ever before.  Yes, we might want to understand your requirements so we can put proper emphasis on different parts of the demonstration, but when a prospect — who clearly knows both what they’re doing and what they want — asks us for a demo, what should we do?  One thing:

Just effing demo  —  and then ask about requirements along the way

Look, I’m not trying to undo all the wisdom of learning how to do deep discovery and give customized demos, espoused by world-class sales trainers like Barry Rhein or in books like Just F*ing Demo (from whose title I derived the title of this post [1]).  These are all great ideas.  They should be your standard procedure.

But you need to remember to be flexible.  I always say don’t be a slave to metrics.  Don’t be a slave to process, either.

Here’s what I’ve learned from these situations:

Avoid triple-qualifying prospects with an SDR, then a rep, then an SC. Make SDR qualification quick and light.  Combine rep and SC qualification/discovery whenever possible. Don’t make the prospect jump through hoops just to get things started.

Intelligently adapt your process. If the prospect says they’re an expert, wants to judge for themselves, and just wants a quick look at your standard demo, don’t try to force a deep discovery call so you can customize – even if that’s your standard process.  Recognize that you’re in a non-standard situation, and just show up and do what they want.

Set expectations appropriately. There is a difference between a “Product Overview” and “Demonstration.”  If you think the right meeting is 30 minutes of slides to frame things and then a 30-minute demo, tell the prospect that, get their feedback, and if everyone agrees, then write “Product Overview” (not “Demonstration”) on the agenda.

Don’t make them wait. If you say the presentation is a one-hour demo, you should be demoing software within the first 5-7 minutes.  While brief personnel introductions are fine, anything else you do up-front should tee-up the demo.  This is not the time to talk about your corporate values, venture investors, or where the founder went to school.  Do that later, if indeed at all.

# # #

[1] A great book, by the way.  My favorite quote:  “in short, I stopped trying to deliver the perfect demo for my product and starting trying to deliver the perfect demo for my audience.”

Don’t Let Product Management Turn Into “The Roadmap Guys”

At many enterprise software companies product management (PM) ends up defaulting into a role that I can’t stand:  The Roadmap Guys*.

Like a restaurant with one item on the menu, the company defaults into ordering one thing from product management:  a roadmap pitch.

  • “The VP of PM is in Boston and Providence this week, can she visit some customers and do a few roadmap presentations?”
  • “Hey, there’s a local user group in NY this week; can PM do a roadmap pitch?”
  • “There’s a big customer in the executive briefing center today; can the PM do a roadmap?”
  • “As part of our sales cycle with prospect X, we’d love to get PM in to discuss the roadmap.”
  • “We’ve got a SAS day with Gartner next week, can PM come in a present the roadmap?”

You hear it all the time.  And I hate it.  Why?

From a sales perspective, roadmap presentations are the anti-sales pitch:  a well organized presentation of all the things your products don’t do.  Great, let’s spend lots of time talking about that.

From a competitive perspective, you’re broadcasting your plans.  If you’re presenting roadmap to every prospect who comes through the briefing center and at every local user group meeting, your competition is going to learn your roadmap, and fast.  Then they can copy it and/or blunt it.

But what irks me the most is what happens from a product management perspective:  you turn PM into “the talking guys” instead of “the listening guys.”  Given enough time, PM starts to view itself as the folks who show up and pitch roadmaps.

But that’s not their job.

PM should be the listening folks, not the talking folks.  Just like sales, PM should remember the adage:  we have two ears and one mouth; use them in proportion.

Wouldn’t the world be a better place if we changed the five previous bullets as follows?

  • “The VP of PM is in Boston and Providence this week, can she visit some customers and observe how people actually use the product?”
  • “Hey, there’s a local user group in NY this week; can PM break off a small focus group to ask customers about how they use the product?”
  • “There’s a big customer in the executive briefing center today; can PM come in and interview them about their impressions on evaluating the product?”
  • “As part of our sales cycle with prospect X, we’d love to get PM in to discuss what specifically they are trying to accomplish and how the product can do that?”
  • “We’ve got a SAS day with Gartner next week, can PM come in and hear from Gartner about what they’re seeing in the market and in their interactions with customers?”

So every time you hear the word “roadmap” in the same sentence as “product management,” stop, pause, and think of a better way to use the PM team.  Sure, there are certainly times when a roadmap presentation is in order.  But don’t default to it.  Keep your PM team listening instead of talking.

# # #

* I’m using “guys” here in a gender-neutral sense like “folks.”

Dear Marketing: Stop Putting the Template Ahead of the Story

I’ve always thought that if marketers wrote newspapers, the famous New York Times headline of August 8, 1974 would have looked like this:

nixon1

Instead of how it actually looked, which was:

pinsdaddy-richard-nixon-resigned-as-us-president-40-years-ago-this-week

What’s the difference?  While both of the above presentations are structured, the newspaper doesn’t let the template get in the way of story.  The newspaper works within the template to tell the story.

I think because marketing departments are so often split between “design people” and “content people,” that (1) templates get over-weighted relative to content and (2) content people get so busy adhering to the template that they forget to tell the story.

Here’s a real, anonymized example:

agf1What’s wrong here?

  • There is a lot of wasted vertical space at the top:  all large font, bolded template items with generous line spacing.
  • The topic section gets lost among the other template items.  Visually, author is as important as topic.
  • There is no storytelling.  There is effectively no headline — “Latest Release of Badguy Product” takes no point-of-view and doesn’t create an angle for a story.
  • The metadata is not reader-first, preferring to remind Charles of his title over providing information on how to contact him.

But there is one, much more serious problem with this:  the claim / rebuttal structure of the document lets the competitor, not the company, control the narrative.

For example, political affiliations aside, consider current events between Trump and Comey.  Like him or not, Trump knows how to control a narrative.  With the claim / rebuttal format, our competitive bulletin would read something like this if adapted to the Trump vs. Comey situation.

Competitive Update:  Team Comey
Trump says:

  • Comey is a coward
  • Comey is a leaker
  • Comey is a liar

But, don’t worry, our competitive team says: 

  • Comey isn’t really a coward, but it is interesting that he released the information through a colleague at Columbia Law School
  • Comey isn’t really a leaker because not all White House conversations can be presumed confidential and logically speaking you can either leak or lie, but you can’t both at the same time.

Great.  What are we talking about?  Whether Comey is a leaker, liar, or coward.  Who’s controlling the narrative?  Not us.

Here’s a better way to approach this document where you rework the header and metadata, add a story to the title, recharacterize each piece of the announcement on first reference (rather than saying it once “their way” and then challenging it), and then providing some broader perspective about what’s happening at the company and how it relates to the Fall17 release.

agf2

This is a very common problem in marketing.  It comes from a lack of storytelling and fill-in-the-template approach to the creation of marketing deliverables.  Avoid it by always remembering to put the story ahead of the template.

Just likes blogs and newspapers do.

The Role of Professional Services in a SaaS Business

I love to create reductionist mission statements for various departments in a company.  These are designed to be ultra-compact and potentially provocative.  My two favorite examples thus far:

I like to make them based on real-life situations, e.g., when someone running a department seems confused about the real purpose of their team.

For example, some police-oriented HR departments seem to think their mission is protect employees from management.  Think: “Freeze, you can’t send an email like that; put your hands in the air and step away from the keyboard!”

I think otherwise. If the HR team conceptualizes itself as “helping managers manage,” it will be more positively focused, help deliver better results, and be a better business partner — all while protecting employees from bad managers (after all, mistreating employees is bad management).

Over the past year, I’ve developed one of these pithy mission statements for professional services, also known as consulting, the (typically billable) experts employed by a software company who work with customers on implementations after the sale:

Professsional services exists to maximize ARR while not losing money.

Maximizing ARR surprises some people.  Why say that in the context of professional services?  Sales brings in new ARR.  Customer Success (or Customers for Life) is reponsible for the maintenance and expansion of existing ARR.  Where does professional services fit in?  Shouldn’t they exist to drive successful implementations or to achieve services revenue targets?  Yes, but that’s actually secondary to the primary mission.

The point of a SaaS business is to maxmize enterprise value and that value is a function of ARR.  If you could maximize ARR without a professional services team then you wouldn’t have one at all (and some SaaS firms don’t).  But if you’re going to have a professional services team, then they — like everybody else — should be there to maximize ARR.  How does professional services help maximize ARR?  They:

  • Help drive new ARR by supporting sales — for example, working with sales to draft a statement of work and by building confidence that the company can solve the customer’s problem.  If you remember that customers buy “holes, not bits” you’ll know that a SaaS subscription, by itself, doesn’t solve any business problem.  The importance of the consultants who do the solution mapping is paramount.
  • Help preserve/expand existing ARR by supporting the Customer Success (aka, the Customers for Life) team, either by repairing blown implementations or by doing new or expanded implementations at existing customers.  This could entail anything from a “save” to a simple expansion, but either way, professional services is there maximizing ARR.
  • Help do both by enabling the partner ecosystem.  Professional services is key to enabling partners who can both provide quality implementation services for customers and who can extend the vendor’s reach through go-to-market partnering.

Or, as our SVP of Services at Host Analytics says, “our role is to make happy customers.”

I prefer to say “maximize ARR without losing money” but we’re very much on the same page.  Let’s finish with the “not losing money” part.  In my opinion,

  • A typical on-premises software vendor drove 25% to 30% gross margins on professional services.  Those were the days one big one-shot license fees and huge multi-million dollar implementations.  In those days, customers weren’t necessarily too happy but the services team had a strong “make money” aspect to its mission.
  • A typical SaaS vendors have negative 20% to negative 10% gross margins on services (and sometimes a lot more negative than that).  That’s because some vendors subsidize their ARR with free or heavily discounted services because ARR recurs whereas services revenue does not.

I believe that professional services has real value (e.g., our team at Host Analytics is amazing) and that if you’re driving 0% to 5% gross margins with such a team that you are already supporting the ARR pool with discounted services (you could be running 25% to 30% margins).  Whether you make 0% or 10% doesn’t much matter — because it won’t to someone valuing your company — but I think it’s a mistake to shoot for the 30% margins of yore as well as a mistake to tolerate -50% margins and completely de-value your services.

The Dogshit Bar: A Memorable Market Research Concept

I can’t tell you the number of times I’ve seen market research that suffers from one key problem.  It goes something like this:

  • What do you think of PRODUCT’s user interface?
  • Do you think PRODUCT should be part of suite or a standalone module?
  • Is the value of PRODUCT best measured per-user or per-bite?
  • Is the PRODUCT’s functionality best delivered as a native application or via a browser?
  • Would you like PRODUCT priced per-user or per-consumption?
  • Rank the importance of features 1-4 in PRODUCT?

The problem is, of course, that you’ve never asked the one question that actually matters — would you buy this product — and are pre-supposing the need for the product and that someone would pay something to fulfill that need.

So try this:  substitute “Dogshit Bar” (i.e., a candy bar made of dog shit) for every instance of PRODUCT in one of your market research surveys and see what happens.  Very quickly, you’ll realize that you’re asking questions equivalent to:

  • Should the Dogshit Bar be delivered in a paper or plastic wrapper?
  • Would you prefer to buy the Dogshit Bar in a 3, 6, or 9 oz size?
  • Should the Dogshit Bar be priced by ounce or some other metric?

So before drilling into all the details that product management can obsess over, step back, and ask some fundamental questions first.

  • Does the product solve a problem faced by your organization?
  • How high a priority is that problem?  (Perhaps ranked against a list of high-level priorities for the buyer.  It’s not enough that it solves a problem, it needs to solve an important problem.)
  • What would be the economic value of solving that problem?  (That is, how much value can this product provide.)
  • Would you be willing to pay for it and, if so, how much?  (Which starts to factor in not just  value but the relative cost of alternative solutions.)

So why do people make this mistake?

I believe there’s some feeling that it’s heretical to ask the basic questions about the startup’s core product or the big company’s new strategic initiatiave that the execs dreamed up at an offsite.  While the execs can dream up new product ideas all day long, there’s one thing they can’t do:  force people to buy them.

That’s why you need to ask the most basic, fundamental questions in market research first, before proceeding on to analyzing packaging, interface, feature trade-offs, platforms, etc.  You can generate lots of data to go analyze about whether people prefer paper or plastic packaging or the 3, 6, or 9 ounce size.  But none of it will matter.  Because no one’s going to buy a Dogshit Bar.

Now, before wrapping this up, we need to be careful of the Bradley Effect in market research, an important phenomenom in live research (as opposed to anonymous polls) and one of several reasons why pollsters generally called Trump vs. Clinton incorrectly in the 2016 Presidential election.

I’ll apply the Bradley Effect to product research as follows:  while there are certain exception categories where people will say they won’t buy something that they will (e.g., pornography), in general:

  • If someone says they won’t buy something, then they won’t
  • If someone says they will buy something, then they might

Why?  Perhaps they’re trying to be nice.  Perhaps they do see some value, but just not enough.  Perhaps there is a social stigma associated with saying no.

I first learned about this phenomenom reading Ogivly on Advertising, a classic marketing text by the father of advertising David Ogilvy.  Early in his career Ogilvy got lucky and learned an important lesson.  While working for George Gallup he was assigned to do polling about a movie entitled Abe Lincoln in Illinois.  While the research determined the movie was going to be a roaring success, the film ended up a flop.  Why?  The participants lied.  After all, who wants to sound unpatriotic and tell a pollster that you won’t go see a movie about Abe Lincoln?  Here’s a picture of Ogilvy doing that research.  Always remember it.

ogilvy