All Research is Biased. Here’s What to do About it

All research is biased. Phone and online surveys suffer from sampling bias. Analytics may not work on visitors who have ad blockers. Averages can blind us to tails. Translating the world around us into data is by definition a lossy process – bias is introduced to help with digestion. And that’s okay because trying to completely eliminate bias is an exercise in futility and will make you question the value of research in the first place. It’s counterproductive. Instead, customer-driven organizations should seek to avoid, mitigate, and reduce bias in decision making.

That’s not to say that research should introduce unnecessary bias or noise. If a question can be worded in a more universally understood manner, it’s important to do so. But someone who says “this form of research is biased” is missing the point. We cannot create a form of research that is unbiased. 

But we can reduce bias and noise within individual forms of research and, perhaps more effectively, by combining various forms of research. Think of it as using mirrors to increase visibility and reduce blind spots. But we can never have a perfect 360° view at a single point of time.

Feedback Loop’s agile research platform is biased in two key ways: it’s (i) online surveys, which means we can’t survey someone without access to an internet-connected device, and (ii) participants are compensated for their time, which means that the interaction isn’t fully authentic or spontaneous. But with Feedback Loop, you get rapid access to audiences you could not easily or quickly learn from otherwise. And our built-in guard rails ensure that surveys are scripted in a way that mitigates bias. There’s far more signal than noise.

The benefits of our approach dramatically outweigh the shortcomings, so long as – and this part is critical – you combine Feedback Loop with other research platforms and techniques.

Feedback Loop is not a silver bullet. You need a research toolkit to limit bias, make data-driven decisions, and pursue growth opportunities with confidence. In addition to third-party research reports, most of our companies conduct primary research with a toolkit with some combination of the following:

strategies for reducing research bias

The above is a generalization – it will look different everywhere. Don’t get caught up in the specifics. The main point is that there’s a multi-pronged research approach to try to understand the world from multiple perspectives to triangulate and inform decision-making. In particular, pair your analytics (hindsight) with your discovery (foresight) to identify correlations between what you think will happen and what actually happens, and then adjust accordingly over time. 

Silver bullets don’t exist. There’s no platform to rule them all. Rather, the best toolkit wins, even if there is some overlap amongst the tools. The sum is greater than the parts. Research is one of the highest ROI activities in business as it has the ability to eliminate wasteful spending pursuing the wrong direction while simultaneously helping you discover lucrative opportunities in the right direction. 

Subscribe to the Feedback Loop