I can’t tell you the number of times I’ve seen market research that suffers from one key problem. It goes something like this:
- What do you think of PRODUCT’s user interface?
- Do you think PRODUCT should be part of suite or a standalone module?
- Is the value of PRODUCT best measured per-user or per-bite?
- Is the PRODUCT’s functionality best delivered as a native application or via a browser?
- Would you like PRODUCT priced per-user or per-consumption?
- Rank the importance of features 1-4 in PRODUCT?
The problem is, of course, that you’ve never asked the one question that actually matters — would you buy this product — and are pre-supposing the need for the product and that someone would pay something to fulfill that need.
So try this: substitute “Dogshit Bar” (i.e., a candy bar made of dog shit) for every instance of PRODUCT in one of your market research surveys and see what happens. Very quickly, you’ll realize that you’re asking questions equivalent to:
- Should the Dogshit Bar be delivered in a paper or plastic wrapper?
- Would you prefer to buy the Dogshit Bar in a 3, 6, or 9 oz size?
- Should the Dogshit Bar be priced by ounce or some other metric?
So before drilling into all the details that product management can obsess over, step back, and ask some fundamental questions first.
- Does the product solve a problem faced by your organization?
- How high a priority is that problem? (Perhaps ranked against a list of high-level priorities for the buyer. It’s not enough that it solves a problem, it needs to solve an important problem.)
- What would be the economic value of solving that problem? (That is, how much value can this product provide.)
- Would you be willing to pay for it and, if so, how much? (Which starts to factor in not just value but the relative cost of alternative solutions.)
So why do people make this mistake?
I believe there’s some feeling that it’s heretical to ask the basic questions about the startup’s core product or the big company’s new strategic initiatiave that the execs dreamed up at an offsite. While the execs can dream up new product ideas all day long, there’s one thing they can’t do: force people to buy them.
That’s why you need to ask the most basic, fundamental questions in market research first, before proceeding on to analyzing packaging, interface, feature trade-offs, platforms, etc. You can generate lots of data to go analyze about whether people prefer paper or plastic packaging or the 3, 6, or 9 ounce size. But none of it will matter. Because no one’s going to buy a Dogshit Bar.
Now, before wrapping this up, we need to be careful of the Bradley Effect in market research, an important phenomenom in live research (as opposed to anonymous polls) and one of several reasons why pollsters generally called Trump vs. Clinton incorrectly in the 2016 Presidential election.
I’ll apply the Bradley Effect to product research as follows: while there are certain exception categories where people will say they won’t buy something that they will (e.g., pornography), in general:
- If someone says they won’t buy something, then they won’t
- If someone says they will buy something, then they might
Why? Perhaps they’re trying to be nice. Perhaps they do see some value, but just not enough. Perhaps there is a social stigma associated with saying no.
I first learned about this phenomenom reading Ogivly on Advertising, a classic marketing text by the father of advertising David Ogilvy. Early in his career Ogilvy got lucky and learned an important lesson. While working for George Gallup he was assigned to do polling about a movie entitled Abe Lincoln in Illinois. While the research determined the movie was going to be a roaring success, the film ended up a flop. Why? The participants lied. After all, who wants to sound unpatriotic and tell a pollster that you won’t go see a movie about Abe Lincoln? Here’s a picture of Ogilvy doing that research. Always remember it.
