Build winning products through continuous testing and validation

In a nutshell: What you think will make a good product or feature is only an assumption, and that needs validation!

In order to create winning products, we must implement approaches where we continuously test and validate our assumptions. This will ensure that customers will and can adopt the features we spent scare resources developing.

The reasons for faulty product development are many. In this article I will touch on some of the pitfalls, and shed light on how to setup a practice for developing products that your users love.

Assumptions lead to waste

One huge driver for wasting company resources is the inherit culture of developing and implementing products based on what we think customers want. I have seen countless examples of project proposals and implementations of solutions that after months and even years of development resulted in very limited end-user adoption. Sometimes because the core idea of the feature was not what customers wanted, other times because customers could not use the solution – including being able to find it and understand what it is.

The inconvenient truth is that:

  • Most ideas fail
  • Successful ideas have required several iterations

Key is to be able to challenge, test and validate assumptions – or hypothesis – as part of the on-going product development. A number of constraints can however limit the organisational capabilities of actually doing iterative product development

  • Rigid and lengthy road map processes
  • Too much distance between internal units
  • Dis-connect between actual users and product development

A bit of explanation required:

Rigid and lengthy road map process

The product development is planned and controlled via a rigid process in which ideas have to be described months or even quarters ahead of implementation. This leaves little room for actual experimentation in the project – both in terms of tech feasibility (will it be overly expensive to build?) and in terms of iterating on the idea once it is submitted for approval.

In addition, the rigid road map process leads organisation to be driven by deadlines rather than value generation. Projects have to be finalised on deadline in order for next project to start. Hence, you will find projects being ‘closed’ after launch, and resources moving on to the next project in the pipeline, rather than when delivering a certain value from the current project – e.g., reaching a certain end-user adoption rate.

Too much distance between internal units

This leads to a second problem – that there is too much distance between the demand and supply organisation. Often, you will find organisations that are structured in ‘Business Development’ being the demand organisation and ‘Technology’ being the supply organisation. The role of ‘Business Development’ is to further monetize on the existing business as well as to develop new business. The role of ‘Technology’ is to figure out how to build what business wants and what the associated costs are.

In ideation and specifying the idea (role of Business Development), there is no ‘tech budget’ meaning that there is little or no involvement of Technology for actually testing out or building ‘stuff’ to prove that the idea is solid – including prototypes, A/B tests etc.

Dis-connect between actual users and product development

When ideating and specifying the project there is little or no involvement of actual end-user. Involvement of end-users is being done once starting the actual project, leaving little room for fundamental changes to the scope that initiated the project.

Think: ‘Product discovery’

The backbone in product development should instead be to think of the development as an ongoing ‘discovery’ of what to build. To figure out who the target audience(s) is, what value it brings them and how to test this as fast and with as little effort as possible to avoid wasting time and effort.

Build your future product around exploring and answering four key questions:

  1. Will they use it?
  2. Can they use it?
  3. Can we build it?
  4. Will stakeholders support it?

For each question a number of sub questions will unfold. These you will have to validate in order to ensure that your take on the product, feature and solution will actually be a winner.

1) Will users use/buy your product or feature?

What is the value that you will deliver to the user? What is the ‘job’ that your product will solve for the user? Do they care about getting this job solved and are the willing to pay for it. If so, how much?

Who do you assume is your target audience? How large is this audience – is there a market?

What is your value proposition?

2) Can they use it?

Are users able to take the product or feature into use? Can they find it? Do they understand the product and what it is intended to do for them? Can they figure out how to use it? Where should it be placed in your current product offering? How should it be distributed?

3) Can we build it?

Is the product or feature feasible? Everything can be built, but does the cost of implementation correspond to the envisioned benefit?

4) Will stakeholders support it?

Will the rest of the organisation support the product – is it ‘on strategy’? Who are your stakeholders? Does it cannibalise or jeopardise your existing features or revenue streams?

 Organise for Product discovery

As described in earlier, there are a number of pitfalls that will limit the ability to work in the mode of product discovery. Enabling this way of working includes working closer together across the organisation, including end-users in the process and testing and validating systematically.

Collaborate across organisation units

Set up teams that work cross-functionally, meaning that both business, product, UX, analytics and tech are included in the team in order to get everybody on-board as fast as possible. Get input from all perspectives to fertilize ideation and enable testing of all perspectives of product development as soon as possible. Test pricing, naming of a feature etc., already at the early stages of development to encounter false assumptions as early as possible.

Include UX resources in order to do mock-ups and prototyping to bring to users.

Include tech in order to build fake features and live A/B test to test adoption.

Include end-users in your product development

Get feedback from real, intended users of your product. Find ways to meet or invite your customers in to your product development process. Get feedback all the way from conceptual level, inquiring about their current needs and behaviours. Later, discover how to create the actual solution, test for usability, etc.

Test and validate systematically

Adopt the Build-Measure-Learn loop to systematically and constantly test your hypothesis in a structured way via Minimum-Viable-Products (MVPs). Figure out what are your assumptions – there will be tons! (Look at the questions to answer above in ‘Think: Product discovery’).

For each of the hypothesis, figure out how you can test the hypothesis. Think ‘Minimum’, meaning: what is the lowest possible effort that will give you insights. ‘Viable’, meaning that the test – or rather tests – are solid enough that they will provide you with valid result.

Fail fast, fail cheap in order to succeed sooner

The methods for testing and validating your hypothesis are endless and really a matter of creativity.

It ranges from qualitative methods such as inquiries, interviews, observations, workshops, usability testing and to quantitative methods of live A/B testing looking at conversion rates and statistically significance of tests.

For a few hints on the methods, read: Introduction to service design and customer experience


Your comment is highly appreciated!