By Tom Taylor, Account Advocate. Studio/D
Whatever our political affiliation, we can probably agree that the projections based on polling for the last two US presidential elections were way off. This raises the question of whether similar polling errors carry over into all types of surveys – including market research. Luckily, the short answer is “no” – particularly if your survey is designed with some best practices in mind.
It’s important to understand that while political polling and marketing surveys may appear to be the same – i.e., asking respondents for their opinion – they are, in fact, quite different. It’s a bit like comparing an elephant to a donkey. Yes, they are both four-legged mammals, but many similarities end there.
Many political polls, especially in this era of polarization, ask questions that tap into deep-seated core beliefs and can trigger (or suppress) strong emotions that influence answers, or even influence participation in the survey. On the other hand, marketing surveys are less likely to tap into such strong and divisive emotions.
We must first understand what likely leads to political polling bias, and how to best craft marketing surveys so they don’t fall into the same traps. A Pew Research article published shortly after the recent election identified several reasons why polling projections failed to reflect reality, including:
- Partisan nonresponse (i.e. one party’s members weren’t part of the polling audience or simply didn’t respond);
- “Shy” voters (i.e. respondents didn’t express their true opinion);
- Underestimated enthusiasm for a particular candidate;
- Effects of the coronavirus pandemic on turnout. In this case, simply believing or not believing in the pandemic swayed participation in polls.
While market research surveys are different than political polls, they do have analogies to some of the key pitfalls listed above. For instance, partisan nonresponse can be seen as an example of inadequate audience sampling. Strong audience sampling depends on two things:
- The list utilized for sampling;
- A demographic question that allows respondents to self-identify. For example, if you’re seeking the opinions of multiple audience segments and want to compare the segments against each-other, you need to ensure all segments are adequately included in your sampling list, and provide a method to identify and segment each.
There are multiple ways to guard against respondent “shyness” or reticence. First, consider offering an incentive for completing your survey, but don’t make the offer too lavish. People value their time and opinions, so the offer of a chance to win a gift card worth a nominal cash value goes a long way toward raising response and completion rates. If your respondents feel you’re offering something of value they’ll be more forthcoming and forthright in their answers.
Secondly, while best practices call for keeping surveys short, consider phrasing a single question two ways – qualitatively and quantitatively. You can tease out subtle details if you ask someone how they subjectively feel about a subject (qualitative data) and follow-up with a question asking them to rate, rank, or place a numerical value to that feeling (quantitative data).
Making your survey blind – that is, unbranded and not identifying the survey sponsor – helps to eliminate both shyness (some respondents may not wish to be viewed as “badmouthing” a company, product, or service) and bias (for example having immediate positive or negative associations with a brand name). Blind surveys are an excellent way to gauge brand enthusiasm by asking the respondent to rank and rate brand names against each other while not tipping-off which brand is funding the research.
Finally, be careful in how the survey instrument questions are worded. Some political surveys purposefully sway responses by adding bias to the questions themselves (essentially, “leading the witness”). It’s important to remove any potential for bias in credible market surveys.
“Garbage in, garbage out” is an old phrase in data management, and surveys are just another form of extracting data. Building your marketing survey with these and other best practices will help ensure unbiased data that can be used to inform your marketing strategy, tactics, and overall business approach.
Need help with a market research project? Studio/D has years of experience in creating and fielding quality research instruments that provide actionable data – give us a call to discuss your needs.
Studio/D is a full-service marketing communications firm working with mid-market industrial and manufacturing clients, together with companies in their supporting ecosystem. We’re a team of “makers” who simplify complex communication challenges with messaging that engages and drives results. Learn more about us at StudioD.agency, or contact us at 314-200-2630.