Breaking Free from the Hypothesis Trap: How to overcome biases in product & customer discovery
- Jade Maravillas
- Dec 16, 2024
- 9 min read
Updated: Dec 17, 2024
In this article, I've collaborated with two Product experts to get their take on effective product discovery. Hint: it's all about listening to customers!
In my last blog post, “Understanding the Why and the Who,” I waxed lyrical about the importance of true customer and market discovery in both product and go-to-market development.
In this article, I wanted to take advantage of having many experienced and intelligent people in my network and approached a couple of experts in the Product side to weigh in on this topic: Joel Bond, a commercial payments product leader with a track record of driving innovation and customer-focused strategies, and Damian Lewandowicz, an entrepreneur, philosopher, and independent consultant specialising in UX/UI design for e-commerce and startups.
Together, they shared invaluable insights into overcoming the hypothesis trap and building impactful, data-driven products.
The hypothesis trap: what it is and why it happens
Every product team has faced this challenge: a room full of ideas, brimming with confidence that you know exactly what your customers want. It’s easy to trust your instincts after years in the industry, but assumptions without validation can lead to costly mistakes. Relying on them without validation is what we call the "hypothesis trap," and it’s more common than you think. “You know you’re falling for this trap when all you hear is ‘in my opinion’ with no evidence,” warns Joel.
What causes the hypothesis trap?
Overconfidence in internal expertise
Joel summed it up perfectly: "I once worked for an organisation that spent tens of millions of euros on a platform based on what they thought they knew, only to find the world had changed around them." Needless to say, that product flopped. Even the smartest teams can fall into the trap of thinking they know it all.
This highlights a deeper cultural issue: many businesses expect their leaders to have all the answers and to deliver results quickly. Leaders are often hired for their expertise, but this expectation can stifle curiosity.
Time pressures
“Many product organisations lack a structured product lifecycle. The lifecycle should include discovery, business case development, and production phases. Typically, a leader identifies a need for a new product or feature. However, the tech environment often pushes toward rapid execution without proper discovery. People often say, "There’s no time for interviews or research. Just move quickly!" In fast-paced environments, it becomes difficult to justify these essential steps. Customer interviews are often neglected because they are viewed as unproductive. Yet, they are crucial.”
Product teams being too technical
When product teams are overly technical, they often focus solely on building features based on what they think customers need or what’s technically possible. They ask, "What capability do we need to build?" and assume that if they build it, customers will come. However, this approach often misses critical considerations:
Strategic: What is the market really demanding? Do we need to replicate everything the customer currently has, or just focus on what's essential to drive growth?
Commercial: What are customers willing to pay for? Just because they have certain features today doesn’t mean they’d pay extra for them in the future.
Joel cites an example: “I worked with a software vendor upgrading a 20-year-old on-prem payments platform to SaaS. The existing platform was mature, but the new SaaS platform only had 30% of the existing features. The team assumed they’d need to reach 100% parity before customers would migrate, which would take five years.
But a more strategic and commercial lens showed:
Strategically, the market only required 50% of the features. Some features were over-engineered, redundant or could be solved in other ways.
Commercially, customers wouldn’t pay for all the old features and could still adopt the platform with 40% functionality.
By focusing on these factors, we could launch in a year instead of five, migrate customers sooner, and deliver the additional benefits of SaaS, like reduced complexity and better scalability.”
The takeaway? A great product leader doesn’t just ask, “What can we build?” but “What’s strategically, commercially, and technically necessary to achieve our goals?”
Siloed thinking
Product and channel teams often operate in separate bubbles. Without collaboration, blind spots multiply. Assumptions go unchallenged, and teams lose touch with the market.
Why does it matter?
The fallout from the hypothesis trap is real. Products miss the mark, resources are wasted, and team morale takes a nosedive. It's expensive. But it doesn’t have to be this way.
Steps to Overcome the Hypothesis Trap
1. Dump your assumptions (seriously, all of them)
The first step to recovery? A good old-fashioned brain dump. Damian emphasised, "Every team member tends to have thoughts they believe are obvious or true. However, many of these perceptions are just that: assumptions. The first task is to document all these assumptions and then validate them.”
Create a squad / task force with representation from the following teams who will be working on the project: Product Management, Product Design, Engineering (Product teams) and Sales, Marketing and to some extent, Partnerships (Channels).
Depending on the circumstance, it may even be worth seeking out external consultants to represent a “we-haven’t drunk the Kool-Aid” view.
Get everyone in a room and document every assumption and burning question about your product, market, and customers. Use sticky notes, whiteboards, or brainstorming tools like Miro to get it all out.
Categorise assumptions and questions by themes, then score based on potential impact.
From these, identify your Ideal Customer Profile (or as Michael Margolis coins, your Bullseye Customer - the very specific subset of your target market who is most likely to adopt your new product or service).
Think of the attributes and trigger events that this customer will have. While it might seem counterintuitive, be as narrow in your ICP definition as possible. This will help you focus on what’s a non-negotiable feature vs. a nice-to-have and make it easier to surface your unique value proposition. Then, translate the assumptions into the questions you will want to ask in your interviews (see below).
2. Talk to real customers (no, really, talk to them)
This is where the magic happens. Based on the ICP, identify a maximum of 5 people you can interview with the aim of validating your assumptions.
Find the appropriate representation of a typical customer. While we said a narrow ICP definition is best, try to avoid outliers or specialists that might skew your data too much.
Speak to both existing customers and the broader market. Don't assume your customer base represents the entire market. Reach out to prospects and ex-prospects (people who have said no to you - super lucrative!). People in your network are also a possibility, but to get an unbiased perspective, take advantage of external market research recruitment platforms. Set aside some budget to compensate these people for their time.
Get the same people in your squad to participate. Delegate one person to ask questions but it’s important for everyone to listen.
Avoid leading questions like, "Would you use this feature?" Instead, ask situational, open-ended ones: "Tell me about the last time you tried to solve this problem. How did that make you feel? Why was that frustrating?" In customer discovery, you’ll be relying on Why and How a lot.
Maintain an open mindset. Let customers express their thoughts freely without preconceived notions.
Be mindful of the information source. Who are you speaking to, the decision-maker, a user or an influencer?
In his Bullseye Customer sprint, Michael Margolis recommends these interviews to be done in 1-2 days max, back-to-back as it makes it easier for listeners to find obvious commonalities. That’s probably not feasible in most real-life situations, so it helps if participants employ active listening and take notes, with an aim to glean insight back to their assumptions.
Qualitative interviews are the best way to gain deep customer insight and uncover painpoints, but as part of a broader approach you might want to include secondary and market research, quantitative surveys attend trade shows and see what the exhibitors are presenting, among others.
3. Prototyping: The bridge between insights and validation
Show, don’t (just) tell. Creating prototypes based on your initial customer insights will not only bring the product to life, it will help validate concepts, gather useful feedback and most importantly, save time and resources by identifying potential issues early, which can reduce the risk of costly redesigns and production delays.
Gather the squad and collate all the information and findings from the interviews, and map the insights against the assumptions and questions you’ve laid out previously.
This is when having Product and Commercial teams on the same discovery path come in handy: As a team, create three different prototypes (product, landing page or both) to present back to the respondents. Align on three versions of the value proposition, then break out into groups: Product teams focus on creating the product prototype, while Sales and Marketing can translate this into brand promise, messaging and pricing options.
Mock up and design the three options and present these back to the respondents for voting and feedback.
UX expert Damian shares the following tips: “Tailor prototypes to the type of product, idea or feature you are trying to validate. If you’re trying to build something innovative and fresh, you might need a hi-fidelity prototype for the users to get behind the idea. When you’re improving upon an existing product, a lo-fi prototype would be sufficient enough to gather valuable feedback. In this case, you can cross-test different ideas through design-thinking methodology.”
“You don’t always have to BUILD the prototype,” Damian continues. “There are other easier options you can use.” His team worked on a startup project in the beauty industry where a clickable prototype designed in Figma was used instead of building a full-scale app. This cost-effective approach provided valuable market insights.
Another option could involve launching a landing page with a presale campaign to gauge interest. "If the idea is good," Damian notes, "customers will ‘vote’ with their money before the product even exists." This type of validation is invaluable, as it directly measures market demand while conserving resources.
4. Make validation cycles a habit
Here’s a radical idea: research isn’t a one-and-done task. Schedule iterative validation cycles every two weeks. "Every two weeks, review what you’ve learned and decide what to take forward," Joel advised.
Test assumptions incrementally and adjust your approach as needed.
Use findings to inform go/no-go decisions for features or product pivots.
How long does this process take? “I’ve observed products validated within just a few weeks. However, it greatly varies depending on your target market,” according to Damian. “If you're reaching a mass audience, you'll need more data points for meaningful insights. Conversely, if you’re in the B2B space, you may have fewer respondents but more in-depth discussions. I suggest having a three-month research process that includes both qualitative interviews and quantitative analysis. This timeframe allows you to adapt as needed.”
Real-world story: Breaking free from the trap
Damian recounts an example of a client operating a B2B e-commerce store. “They had done their homework and figured out that about 60-70% of all orders were the same as their last orders. Based on that, we designed and implemented a feature in the store that let logged-in customers re-order their previous order within seconds. That was a big win.”
He continues, “About a year after, the client came up with an idea of creating a mobile app based on this feature, as he thought it made sense since his clients primarily communicated through mobile. We went forward with a short product discovery call and before we started the project, asked the client to validate if his customers would use a mobile app.
One week later, he came back to us saying that only 20-30% of his clients could use the app. And even that was questionable. Thanks to that validation, he canceled the project, luckily without spending any more resources.”
Conclusion
The hypothesis trap is sneaky, but it’s not unbeatable. With a combination of curiosity, validation, and collaboration, your team can escape it and build products that truly resonate. As Joel wisely put it, "The brave decision is to say no quickly and painlessly when something doesn’t work."
Don’t let assumptions guide your next product—let your customers lead the way.
Speaker Spotlight: Joel Bond
Joel Bond is a commercial product leader with over 15 years of experience in payments, fintech, and blockchain, specialising in strategy, market development, and customer-focused innovation. Joel has a proven track record of launching transformative payment solutions, crafting go-to-market strategies, and driving measurable outcomes by aligning products with customer needs. Passionate about advancing financial technology, he excels at identifying market opportunities, enhancing product positioning, and delivering impactful results in dynamic, fast-paced environments. Joel holds an MSc in Data Science and a BA (Hons) in Management Science, showcasing his analytical and strategic expertise.
Speaker Spotlight: Damian Lewandowicz
Damian Lewandowicz is an entrepreneur, marketer, and philosopher who runs Uxtivity and Dezignite, boutique design agencies specialising in data-driven UX/UI design. He collaborates with e-commerce businesses and startups to optimise digital products for maximum efficiency. Co-author of "LiGHT Book 2," Damian also serves on several supervisory boards and brings a unique perspective to product strategy through his blend of design expertise and philosophical insights.
Comments