All posts

Methodology

The evidence gap in product decisions

August 12, 2026 · 8 min read

Your CRM explains what happened. It rarely explains why — or what would change your next bet.

Owned data bias

~70%

Typical CRM cohorts over-represent active buyers and under-sample rejections and near-misses.

Decision linkage

3 questions

Every brief must answer: what decision, what risk, and what evidence would change the call.

Triangulation bar

2+ sources

Material recommendations require converging evidence — not a single dataset or interview wave.

The problem with owned data

Most product teams sit on rich behavioral data — funnels, cohorts, feature usage, support tickets. That data is invaluable for describing what happened. It is structurally weak at explaining why a segment churned, why a price point failed, or why a new category entrant is winning share you never measured.

Owned data

What

CRM · product · support

Primary research

Why

Field · experiments

Owned data describes behavior inside your product; primary research explains choice outside it.

Insight

Selection bias is not a footnote

CRM and product analytics inherit the population that already chose you. Churned users, category considerers, and non-buyers are systematically under-represented — yet they hold the evidence for repositioning, pricing, and category expansion.

Owned data is biased toward people who already chose you. It under-represents rejections, near-misses, and segments you have never served. Strategic decisions — packaging, positioning, pricing, category expansion — require evidence from outside that lens.

CRM can tell you who upgraded, when they churned, and which features they used. It cannot tell you why a considerer chose a competitor, what price ceiling they had in mind, or which unmet job caused the downgrade. Those answers require designed fieldwork with the right sample and instrument.

Decision framing before fieldwork

Before any interview or survey runs, the research question must be tied to a decision. What will leadership do differently if the finding is X versus Y? Without that framing, fieldwork produces interesting narratives that do not change bets.

Decision
If X
If Y

The decision tree drives sampling, instrument design, and the triangulation plan.

1

Primary decision

2–3

Disconfirming tests

3+

Evidence sources

We map each engagement to a decision tree: the primary choice, the material risks, and the disconfirming evidence that would change the recommendation. That tree drives sampling, instrument design, and the triangulation plan.

Interesting research is cheap. Decision-grade research is expensive because it is designed to survive scrutiny.

Bundl research brief template

When to commission primary research

Primary research is warranted when the cost of being wrong exceeds the cost of learning — re-pricing, repositioning, entering a new segment, or defending a board narrative under diligence.

Insight

The bar is validity, not volume

Re-analysis of CRM exports rarely clears the validity bar for strategic moves. The question is not "do we have data?" but "do we have valid evidence for this specific decision under this specific risk profile?"

Leadership is debating two materially different bets. Sales and product disagree on the root cause. Post-launch metrics contradict pre-launch assumptions. Board or investors are asking for external validation. Any of these is a trigger to commission designed fieldwork — not another dashboard.

The bar is not "we lack data." The bar is "we lack valid evidence for this specific decision." Re-analysis of CRM exports rarely clears that bar for strategic moves.

Closing the gap

Closing the evidence gap means pairing behavioral signal with designed fieldwork: adaptive interviews with churned users, conjoint with category considerers, ethnography where self-report fails.

Qual
Quant
Market

Qual, quant, and market signal converge before a material recommendation ships.

Insight

Provenance beats polish

The output is not more slides — it is a recommendation with graded confidence, explicit limits, and a traceable link between each claim and the evidence that supports it. Leadership should be able to challenge any finding and see exactly where it came from.

The output is not more slides — it is a recommendation with provenance, graded confidence, and an explicit link between evidence and the decision it supports.

Want the full research brief or a similar engagement for your team?

Contact us →