RE-STOKED: When Not to Love Bias (11/14/14)

We frequently find clients and colleagues protesting the use of improvisational, collaborative, or small-sample approaches to design research and hypothesis testing. Aside from the usual complaints about qualitative approaches (takes too long, costs too much, people lie), we find that people are really, really worried about researcher and respondent bias. And too often we joke, as we did last week, about learning to love bias.

But sometimes bias, especially the kind that the researchers deny or are willfully unaware of, is downright destructive. In this piece, Zeynep Tufecki looks at the role of bias in the Hollaback video about street harassment, and reminds us of a very important truth about research design: “every element of research design requires hard thinking”.

With access to cameras, social media, Google, and the occasional syndicated research report, we see brand and product strategists losing some of the social science skills needed to design research (and do design research) well.

But even if this "research" was designed unimpeachably, there could still be multiple explanations for what we see in this video (about street harassment, location, and race). In fact, there almost always are multiple explanations for any set of data - but it's our job as strategists and researchers to make good sense of it. To do this, we need and underlying theory.

To find the right theory is, like everything, an iterative process - one that starts with how we collect data, and never stops, through the selection of data and the interpretation of it. We have to look at many possible designs, many possible selection sets, and many possible explanations to find the best ones.

Designing good and useful research, in other words, isn't all that different from designing good and useful products.