Choosing Your Experiment Type: Discovery or Validation?
This article is the third in an exploration of the experiment-led design work I’ve been conducting for the last 18 months. You’ll find links to the whole series at the bottom of this article.
Once you’ve formed your hypothesis, your tentative answer to the question driving your work, the next decision sits right in front of you: what type of experiment do you need to run?
This choice determines whether you’re about to learn something new about the problem or test whether your proposed solution works. Get this wrong and you end up asking questions that won’t move you forward, or worse, conducting experiments that answer the wrong question entirely.
Understanding the Two Types
in my work to date, I have focused on two main experiment types: discovery and validation.
Discovery experiments uncover information you don’t yet have. They clarify the problem, reveal what’s happening in the workplace, and help you understand the context. These are the experiments you run when you need to learn more about the situation before you can propose solutions.
Validation experiments test whether a proposed solution is the right way forward or at least a viable way to move forward. These are the experiments you run when you think you know what might work, but you need to check before committing significant resources.
Both serve different purposes. Choosing between them requires understanding what’s causing the risk in your current situation.
Deciding Which Type You Need
The deciding factor is straightforward: what’s creating the uncertainty?
If you lack information about the situation or environment, you need discovery. If you’re uncertain whether your proposed solution fits the problem, you need validation. This maps directly to where you sit on the Risk-to-Certainty Graph. Wherever risk outweighs certainty, you need to experiment. The type of experiment depends on the source of that risk.
Methodologies Work for Both
Some methodologies naturally suit one type over the other. Interviews and observational studies tend towards discovery; A/B testing and pilots lean towards validation. However, many methodologies work for both, depending on how you structure them.
Surveys can discover what people need or validate whether a solution meets those needs. Prototypes can help clarify problems or test solutions. Focus groups can explore unknown contexts or test whether proposed approaches resonate. We’ll explore these methodologies in more detail in a future article, but understanding the distinction between discovery and validation helps you select the right approach for your current question.
Progress Means Moving from Discovery to Validation
Progress in experiment-led design means moving from discovery to validation. That movement shows you’re advancing towards a solution rather than spinning in place.
If you find yourself conducting seemingly endless discovery experiments, pause and run a retrospective on your work. Ask yourself:
What’s going well in this project?
What’s not working?
What open questions remain?
What’s blocking progress?
The answers reveal whether you’re asking significant questions that move you forward or just interesting ones that keep you busy. You don’t need to be perfect; you’re not looking for an absolute answer. You need just enough to be confident in taking the next step or asking the next question. You don’t need to be 100% certain; you just need your level of certainty to outweigh the level of risk.
A Note on Verification
There’s potentially a third experiment type: verification. This focuses on post-intervention experimentation, testing whether your deployed solution works in the real world. However, that’s not part of this current body of work.
Like your experiments should move from discovery to validation, my own thinking has developed gradually over time. Verification will come when we’re ready for it, but right now we’re focused on getting you from problem to solution with confidence.
The Question That Matters
The question to keep front and centre remains simple: do we have enough to proceed?
That question, combined with understanding whether you need discovery or validation, gives you everything you need to choose your next experiment wisely. This is how experiment-led design should work: powering forward through small, deliberate steps, never trapping you in endless circles, always moving you closer to solving the actual problem.