In an era when even Goliaths like Pepsi and IBM are designing and innovating like startups, many leadership teams are surprisingly ill-equipped when it comes to a related–and basic–business task: conducting customer experiments.
“While there has been a rapid growth in experiments, especially within tech companies, we’ve seen too many run incorrectly,” observe Harvard Business School’s Oliver Hauser and Michael Luca, who’ve studied the topic, in a recent Harvard Business Review article. “Even when they’re set up properly, avoidable mistakes often happen during implementation.”
Hauser and Luca go on to list the key steps to conducting a customer experiment. As it happens, an intriguing experiment conducted by Sweden’s central bank meets most of their criteria. Since July, the central bank–known as the Riksbank–has kept interest rates at -0.35 percent.
Why would the Riksbank conduct an experiment like this? Mainly to try to stimulate spending. If keeping money in the bank will cost you–in the form of negative interest–you’re less likely to keep it in the bank. And in a digital-payments culture like Sweden’s–so the thinking goes–you are more likely to spend that money than you are to keep it beneath your mattress. “Although retail banks have yet to pass on that negative rate to Swedish consumers, the longer it’s held there the more financial pressure there is for banks to pass the costs onto their customers,” notes Business Insider.
Here’s where the experiment gets fascinating: Based on early responses, it’s not clear that negative interest rates will, indeed, boost spending. That’s because, even in a country where four of five transactions are cashless, there’s still resistance to the notion that a bank should charge you for holding your money. Which is why some people have begun to keep cash in their microwaves, as Björn Eriksson, former head of Sweden’s national police and now head of Säkerhetsbranschen, a lobbying group for the security industry, told the local press.
In other words, the Riksbank has already learned from its experiment. It’s learned that negative interest rates for more three months will indeed change the behavior of banks and customers. But it won’t necessarily stimulate spending. Nor will it stimulate the banks themselves to charge negative interest rates. (Instead, as Quartz points out, banks are likely to pass along the cost in the form of higher fees.)
The Riksbank trial is a fantastic example of how to run a successful customer experiment. Specifically, it exemplifies three of Hauser and Luca’s crucial steps:
1. Identify a narrow question.
For the Riksbank, that question is: How negative do we have to make the interest rate–and for how long–to stimulate banks and customers to change their spending behavior?
2. ‘Use a big hammer.’
What Hauser and Luca mean is this: If you want to change customer behavior, your experiment has to make waves. Otherwise, you’ll learn nothing. Had Riksbank used only slightly negative rates, like -.05 or .1, they might not have learned what they’ve learned from a -.35 percent rate. “Your goal should be to see whether some version of your intervention–your new change–will make a difference to your customers,” they write. “This requires a large-enough intervention.”
3. Commit to your experiment over a period of time.
“Once your experiment is running, leave it alone,” they write. “The one thing that’s not OK: Running your experiment until your results look as though they fit your hypothesis, rather than until the study has run its planned course.” Indeed, the Riksbank has stuck to its plan, keeping the rate at -.35 since July.
The larger takeaway here is a broad reminder about decision-making. The next time you’re faced with a big decision that will affect your customers, you don’t have to make your decision in a vacuum. You can conduct a small experiment to see how your action might change customer behavior.
Nathan Furr, co-author with Jeffrey Dyer of The Innovator’s Method, suggests that leaders reframe their jobs, to think of themselves not as chief decision-makers, but as chief experimenters.
“If you try to play the role of decision maker, you will stumble,” he writes on Inc.com. “Instead, frame up a low-cost experiment that tests the most critical assumption. Successful innovators play the role of chief experimenter by helping their teams frame the right assumption to test and then pushing the experiment.”