Experiment To Improve The Chance Of Innovation Success

We know that business needs to innovate. Whether a startup, established SME or major industry player, new customer offers are critical for survival.

We also know that most innovations fail in the market; despite our enthusiasm, creativity and persistence.

We need to test our ideas as they develop. We need to experiment.

Experimentation is critical to reducing risk

In science, an experiment attempts to test a hypothesis. To see whether it holds or not. In business, it is the same. We try to discover whether we can make the product at the required price, which benefits are most interesting to the customer, what is the size of the target market and how fast is it growing, and how much the customer will pay to solve their problem?

Innovation is a risky business; there are technical, economic and market risks. Our goal is to use experiments to reduce the risk and uncertainty in the most cost-effective way possible. We need the maximum bang for our experimental buck. So we must choose our experiments carefully.

Experiments must be survivable

The first rule is that experiments should be survivable. We don’t want to bet so much on a successful outcome that we have no resources left if things don’t work out. Given the failure rate of innovation, we should expect that most of our trials are negative.

It sounds obvious, but it is a common mistake to become so mesmerised by our exciting idea that we go all in; unable to imagine that it might not work.

The common advice “test early and test often” makes sense, but how to do it?

Design experiments to fit the market

Different markets have different dynamics, and that should guide our experimentation. Remember, that our goal is to get the maximum useful information from our investment.

Markets vary in their risk profile, the cost and time required to carry out experiments. In some markets you can develop prototype products quickly and cheaply, and there is little risk in testing them directly on customers. A lot of apps and other software products fall into this category. Here innovation guides talk about “launch and learn”, “move fast and break things”, and recommend agile development methods.

At the other end of the scale, you have pharmaceuticals and big engineering projects. Where it is expensive to create a new product, there are big regulatory hurdles and existential risks in putting an early prototype in the hands of the customer. There is really no ‘Minimum Viable Product’ for a nuclear power station or agile development for a new drug.

Where a market sits along the fast/cheap/low-risk to slow/expensive/ high-risk continuum is not even related to the complexity of the new product or service you have created. I was involved in the development of a new lubricant for aerospace applications. The idea was simple, patentable, and well within our manufacturing capabilities. The product looked like it would offer improved fuel efficiency and longer life. This was potentially very exciting. Unfortunately, this is an industry where a single failure can be devastating, and it is therefore very conservative about new products. They like to see lots of test data and evidence of performance in use over long periods of time. When we looked at the scale of the testing programme we would have to carry out before anyone would risk putting our new lubricant in a plane, both time and cost, we realised that the experiment was potentially not survivable. We could do the experiments, but it would severely affect our cash-flow. If the tests were positive, and we could persuade users to switch quickly, all might be well. If the results were not what we hoped it could bankrupt the company. In the end, we had to abandon the innovation.

Choosing what experiments to run

We need to match the experiment to the dynamics of the target market, the nature of the innovation and the risk profile. If the market is fast moving, development cycles short and cheap, and there is little consequential risk of releasing the product into the wild, then launch and learn and test marketing are useful approaches. If the market doesn’t match any one of these characteristics, you need a different approach. You can quickly, and relatively cheaply, develop a software tool for the banking industry, but it would be extremely unwise to put it into use and see what happens.

As the hurdles to innovation become higher, other experimentation techniques come into play. You need to identify the biggest risks to your innovation and figure out how to test them. If you are not sure you really understand the customer need, but it isn’t possible to do full test marketing, can you isolate the essence of the concept and test that out with customers? Can you use mock-ups or early prototypes to see how customers might react?

If manufacturing costs are critical, can you prototype sub-systems, work by analogy with similar products, or test by computer modelling?

For really complex developments where full-scale testing becomes impractical or unethical, constructing a new type of building or an innovative bridge, modelling and simulation become the only possible route.

Complying with the rules is not enough

In regulated industries, the temptation is to simply comply with the required testing regime. Unfortunately, as the current scandal over medical devices shows, being regulator approved will not save you if your product then goes on to harm consumers. Those companies got regulatory approval, but they are still toast. They chose to try and get away with the minimum, rather than explore the risks. They tested directly on consumers hoping there would be no unforeseen problems. They created an experiment that is not survivable.

Whatever the innovation, experimentation and testing are critical for success. You need to work out how to make the experiments survivable and deliver the best reduction in risk and uncertainty for the investment.