About the Post

Author Information

Author and chief editor of the effizine, online magazine for busy professionals desperate for getting things done efficiently

Pitfalls of vicious loop of testing

Another common reason for startups to fail is the lack of persistant approach, otherwise known as the vicious loop of testing. Many startups do tend to try different things, measure how they work and abandon ideas in light of lack of response. It sometimes looks like going to a bar and then chatting up one girl every five seconds and dropping the chat half-way if it immediately doesn’t convert to something more interesting. I tend to believe that some things only work in the long run in persistent approach to educate the market and larger audience of users by advocating certain ideas. Sometimes it’s not about finding what market wants but instead persistant creation of what market wants.

Killing off features or closing down entire show prematurally sometimes makes you miss out on a big opportunity. The mentality of quick turnarounds sometimes leads to some ideas being concluded before the market really can take, digest and rething the proposal and react to it properly. Some other times its not the product that fails or otherwise dissapoints the owners but its the marketing that needs to be changed, not the whole business. Some other times it’s simply the method of measureing success that is to blame.

Let’s take Nokia’s approach to touchscreens as the example. Touchscreens have been around for a while now with market flooded with stylus driven Windows Mobile bulky smartphones. Ahead of the current hype Nokia has been trying out different phone models but never dared to come up with one with a touchscreen. The reason for that was that Nokia has been representing the ultimate lean approach to phone manufacturing. Since no one declared any interest in the next generation of operating systems or devices that would be equipped with a touchscreen they haven’t built it.

At its best in 90s Nokia has been conducting the biggest market research in the phone industry on a continual basis. Every single market research has been leading them to a dozen new phones, none of them equipped with a touchscreen. They’ve been measuring sales of phones of different kinds, sizes, colours and even shapes. None of them lead them to an idea of what a smartphone should work like, or even look like a decade later. They simply looked at the numbers and decided that the touchscreen is not what consumers want.

What consumers never told them is what touchscreen would they won’t and people would never do that. The reason behind this is simple as Jobs has put it: “You cannot just ask people what it needs and deliver to that effect because people might not know what they want until you show it to them”. That’s right — none of the iterations would led Nokia to the conclusion that there’s entirely different operating system to make and to be enclosed in a simple case entirely covered by a touch enabled glass.

The reason why market research would never reveal the right path to Nokia, or to any other company doing any other type of innovative products is that people usually want the old stuff improved rather than a radical change, unless you show the radical change to them in its entirety and it proves to be better. Incremental changes can only provide incremental fine tuning of a product idea that has been already formulated but they cannot drive the whole product innovation or its development. Working on several concurrent ideas can actually make you distracted as it has happened with three dozens of Nokia phones on sale at the same time.

If you embrace the market research and the A/B testing wholeheartedly you might end up with a few dozens of mediocre or even wrong answers to the question of what the market needs really are. Sometimes it simply prolongs the process of finding the answer. Let’s think of how many combinations of something as simple as a button on a page can have. You can A/B test several combinations at once, but simple placement, size and colour yields 100s of permutations and you still have to have audience and time to test that. I think that if Daimler wouldn’t come up with an idea of a car people would refine the old business and even these days we would only get faster horses.

Going back to the development of digital products – A/B testing of a website for example might lead you to a wrong conclusion that the product doesn’t work of work a bit only with a certain positioning of a button on a site. What the testing will never tell you is if the button is needed at all and with what should it be replaced. Some innovations can only be invented in minds of their founders and the market adoption (or lack of it) should not be left with statistics. It might be that you would be aiming for a market that would only adopt your ideas by a small percentage in the first year or two. We very often overestimate what can be done in a timeframe of first five years and underestimate what would happen in timeframe of next 10 years.

Comments are closed.