Market Research's Surveillance Problem

First of all, hi. It’s been a minute, and you might notice that we’re no longer meeting each other over on tinyletter. I’ve migrated to Substack. I have reasons. Hopefully it won’t make a bit of difference to your experience at all. Though probably that logo thing should go.

I’m aiming to make this more regular, and a bit more blog-like. There are, broadly, three or four things I’m most interested in these days, and that’s what I’ll write about here.

1. I’m obsessed with the uses of technology in market research to uncover “the truth”, and with understanding why marketers don’t seem to trust their own customers.
2. I worry that despite all the surveillance and technology and tracking of consumers, most brands don’t know how to do anything with market research after the debrief has been delivered. Maybe market research itself needs to be more human, more experiential, more intimate - so it can be more usable and accessible and meaningful.
3. My most naive notion is that marketing can shape culture as well as reflect it, and that it probably should. I think brands should have authentic opinions, and they should walk away from investors and consumers who don’t share their values. (This also has implications for how agencies hire, cultivate & promote talent, by the way.)
4. I’m not sure what planners/strategists do in agencies and marketing departments anymore - though I’d love to know. But there are some skills, techniques and tools I think planners & strategists should have.

So today, a story about Wonder Woman.

Three years ago, while traveling across country to adopt an Irish Setter, I read Jill Lepore’s book, The Secret History of Wonder Woman. You should read it if you haven’t already, but one chapter in the history of one of Wonder Woman’s creators, William Moulton Marston, stood out to me.

Marston had degrees in psychology and law and was one of the early inventors of the lie detector, creating the systolic blood pressure test, which he sometimes called a Love Meter (the man was obsessed with love and eroticism, had practiced free love, was in a polyamorous relationship, and believed in woman’s superiority to man).

At about the same time that Marston was obtaining a PhD in Psychology and developing his Love Meter, he had also studied the structure, mechanics and emotional impact of motion pictures (even writing a few scripts himself), a new and rapidly evolving technology. Marston was one of several psychologists hired by major movie studios (Universal hired Marston) to consult on their films. Psychological research had been commissioned by the studios (often to elude censors), and by Will Hays (to make the case for censorship); other studies had to do with recall, emotional response, testing scripts, and developing “rules” for different genres.

To conduct this research, Marston ran “contests” - really, surveys, asking open-ended questions about, for example, “why beautiful women love ugly men”; he also observed and interviewed audiences who had been shown scenes, full movies, or movies without endings or with unhappy endings; and he tested audience reactions to horror movies with systolic blood pressure tests.

In the summer of 1941, Marston conducted one last experiment. At the Astor Theatre in New York, he strapped his subjects up to his lie detector during a screening of a new MGM film, Dr. Jekyll and Mr. Hyde , starring Spencer Tracy, Ingrid Bergman, and Lana Turner. (MGM’s film was a remake of a very scary and incredibly racy pre-code Paramount film that had been produced in 1931, while Marston was still in Hollywood.)

“More than 100 girls from all walks of life took part in a series of psychological experiments to determine feminine emotional reactions to male aggressiveness,” read the story in The Hollywood Reporter, under the headline, “MGM Tries Out ‘Hyde Detector.’ ” Marston divided his subjects by hair color (blondes, brunettes, and redheads); marital status (single and married); and occupation (“secretaries, débutantes, and students”). He wanted to know which “love scenes” the women enjoyed more: Jekyll’s tender embraces of his fiancée or Hyde’s brutal rape of a prostitute. He also wanted to know what portion his subjects thought of every man was a brute, as if this were merely a matter of mathematics. He concluded, “Average percentage of Hyde in every man, 34 percent.”

Lepore, Jill. The Secret History of Wonder Woman (p. 436). Knopf Doubleday Publishing Group. Kindle Edition.

Later, of course, Marston’s research and Love Meter were incorporated into more “sophisticated” lie detectors; those lie detectors were used in criminal investigations and judicial proceedings; it became essential to “prove” the validity of these tests; they remain both ubiquitous in their use and inadmissible at trial.

Meanwhile, in 2019…

Today, clients still want to find new ways to “validate” what they can see and hear and feel with their innate human sensorial capacity, and describe with their human language skills, by instead using technology that measures physiological responses: micro-expressions, heart rate, pupil dilation and contraction, electrical conductivity, blood flow, firing synapses, hormone secretions, and so on. They especially love to then sort these measures by demographics, so they can draw conclusions by gender, ethnicity, age, income, and so on.

But it seems to me that these technologies rarely offer an explanation of observed phenomenon; rather, they offer a more scientific (or, as in the case of lie detectors, pseudo-scientific) description of observed phenomenon. Using the biochemical, electrical mechanisms that create or enable an emotional response to explain that same response is, well, a bit tautological. “The audience screamed in terror because they were terrified.” “The scary scene was scary, which we were able to observe through audible gasps and visible startle responses, but also measure with fMRI and EEG.” When I forget to turn off the circuit and risk shocking myself while rewiring an outlet, the most important data point to know is “on v. off”… not the amperage.

Also today, these techniques - which emanate from worthy enough academic inquiries and scientific explorations - are quickly applied in market research environments to sell deodorant or movie tickets… before becoming tools of surveillance deployed by large corporations and government entities.

This is a long history - 100 years or so - of technology being used in market research to suss out what people really think and really feel. More often than not, what they think and feel is observable, without the aid of coils and wires and sensors and scanners. Innovation in market research technologies is driven by the allure of scale, of being able to “test” hundreds or thousands of people’s responses to marketing stimuli economically; and the desire for verifiability, a scientific explanation that brooks no interpretation and therefore is 100% trustworthy.

But the use of these technologies in market research plays a role in normalizing their use in other spaces, too. It reinforces the belief that most people’s default setting is deception. It undermines observation and description as valid methods, dehumanizing the researchers as well as the subjects. It suggests that observation and description are noise obscuring pure signal - it discounts all testimony as not merely inaccurate but false, by definition.

The obsession with how humans are poor recorders of their own actions and responses; with the phenomenon of post-rationalization; with language being a construction misses something essential about the human condition: we are animals with written, spoken, translatable and transferrable language. Our experience of life is always inflected by the narratives we construct around them, and vice versa. There is a world of information in the stories people tell, the way they construct them, how they position themselves within them, how the stories fail to perfectly align with some behavior, underplay other behavior, or are perfectly in sync with observable phenomena. What essential truth is there in the heartbeat, the firing synapse, the pupil?

Today we live in a technological reality of being constantly tracked and measured - not always through biometrics, mainly through behavioral data. This kind of research/data collection and analysis also started in research labs before making the leap to website and advertising analytics. By convincing us that our data made the services better, contributed to a greater good, or were, simply, required to use a service or “passively” collected in the course of using it, we became inured to this kind of tracking. Today, it can be used against you - as evidence in a criminal proceeding, or for the purpose of disinformation. It is, in this way, a lot like the lie detector test - which despite being pseudo-science, is still assumed by most people to be evidence, still applied to people with sensitive jobs, and still used to manipulate people.

Most of all, these technologies - their ubiquity, the enthusiasm with which they are met, the deep belief that they engender in decision-makers - constantly provoke the same response in me. We have tools for finding out what people do and why, yet we keep searching for technologies that atomize behavior and motivation in search of some greater truth - so why do marketers believe people are lying?

I don’t doubt that people are inaccurate, that they confuse what they do with what they wish they did (or want to be seen to have done), that they provide performative responses, or that they just pound on the keyboard to get through a particularly long or boring survey. I have met one or two people who “just like taking surveys”. I’ve been lied to by recruiters who fudged the recruiting criteria, skewing my sample. But the idea that a large percentage of ordinary folks, on a 10 minute survey, with no reward except a few bucks, or points, or being entered in a sweepstakes, are taking the time to consciously mislead a disembodied pollster, frankly astonished me the first time I heard it, and 20 years later hasn’t stopped astonishing me.

So I really want to know why marketers and agencies whole-heartedly believe it.

A few links of interest:

(assuming you’re interested…)

• In addition to Jill Lepore’s book, linked and referenced above, I read The Lie Detectors, entirely about the early days of inventing the polygraph and pushing its adoption. Let’s just say the inventors of this technology were rather odd ducks.

• For a glimpse at a more modern version of the Love Meter (“High Definition Electroencephalography, Galvanic Skin Response and Eye Tracking”) in movie testing, check out this journal article, The perception of movie trailers through consumer neurosciences techniques, by Luke Zarb at the University of Malta.

• Google “why people lie on surveys” and you’ll see how much survey companies and other researchers obsess over the phenomenon. There’s even a survey to find out how much people lie on surveys.
  
• But also, here’s an example of how a “lie” is actually a product of bad survey design. If marketers design their research poorly, select sample in non-representative ways, or ask questions that trigger social desirability biases, should the dominant concern be about research participants lying, or about shoddy design and sample practices? How might these same design and sample flaws/biases make their way into more “scientific” or technological data collection methods? Perhaps more philosophically, why do marketers - whose job it is to manipulate and persuade you into consumer behavior - obsess over the “truth” of that behavior?

Would love to know what you think.