Craft Surveys That Respondents Want to Complete

The role of Insights is to understand. Our job is to listen to the voice of the people, and solve business problems based on what we learn. But to have a good conversation with people, we need to consider overhauling the typical survey experience. To better understand how to do that, we asked what people dislike about surveys so we knew what not to do.

Long is Too Long

Surveys that are too long are a very common dislike—especially surveys that are longer than advertised. It also leads to poor quality information. As one person told us: “When surveys drag on too long, then you lose interest and don’t give the answers that really mean something.”

The long survey reflects both the heritage of research design, greed for information and a disregard for the time pressures of modern life. When survey research began, interviews were face to face and long surveys were common—because it was expensive and slow to send someone out to do an in-person interview. Companies took advantage of the opportunity and crammed in every question they could. Besides, they didn’t know anything about the people they were talking to, so they started from scratch with each interview. Also, having a face to face interviewer present meant there was someone whose job it was to keep people engaged. The times have changed.

Now, the cost of doing a survey is infinitesimally less expensive, and we know a great deal about the well-profiled people who belong to our communities. There is no excuse for the long survey today.

Being Disqualified Through Screening

Another common complaint was being asked a series of survey screening questions only to be told you don’t qualify for the study. My colleague, Rob Berger of sister company Maru/Blue, covered this in an earlier article. The good news is that with careful profiling most disqualifications can be avoided because you already know the person you are inviting to the study qualifies.

Needlessly Complicated Questions

Another common complaint is long questions, often grids, which are time-consuming, repetitive, and difficult to answer.

We did another study, in which we invited people to complete a survey with a tedious set of grid questions about media habits and their perceptions of different media companies. It was difficult but, sadly, not unusual. After an average of 12 minutes, many respondents threw their hands up and dropped out. But others soldiered on. We asked the persistent to turn on their video cameras so that we could use facial coding to determine how they felt.

We saw some contempt.

And disgust.

But mostly what we saw was boredom.

Here is what people had to say about that experience.

Let’s Focus on Streamlined Survey Design Best Practices

To maintain the quality of our research, we need to step away from survey design that leads to a poor experience for respondents.

Long grids are obviously a problem. Do we really need all those brands rated on all those attributes? Our research has shown that in any big attribute bank there are many attributes that are measuring the same underlying concept. Look at the correlations, or run a factor analysis of your last study that used those attributes, and you’ll see that you can probably drop two-thirds of the items because they are duplicative. The same goes for attitude statements.

And skip the scaled ratings. Our research has shown that binary brand ratings are three times faster to answer and provide the same insights. Doing these things also makes studies more mobile friendly, the lack of which is another pet peeve—especially for younger people.

If long surveys are tiresome, we must embrace shorter surveys. A more agile way of conducting research with shorter surveys is the future. It fits better with the speed of business today. The era of “I’ll get back to you with your answer in 8 weeks” is drawing to a close. According to client-side researchers I have interviewed, time is of the essence.

We know that being disqualified is a big problem, but thankfully there is an easy answer: careful and well thought out respondent profiling. It may take a little planning, but it makes a huge difference to the people who do our surveys.

When a person doing our surveys complains that with long surveys “you lose interest and don’t give the answers that really mean something,” I take that very seriously. The whole reason to do a survey is to get answers that mean something.

Let’s heed the voice of people, and avoid long, repetitive surveys.