Scales like “definitely will buy, probably will buy, might or might not buy, probably won’t buy and definitely won’t buy” and “On a scale of 0 to 10, how likely are you to recommend product X to a colleague?” have been the bread and butter of market research since the industry blossomed following World War II. But much has changed since then, including the emergence of the internet and mobile devices.
With changing times, we need to reconsider the way we do things, including how we use scales. Here are three reasons why you should rethink scale use:
1. Scales take 3x longer to answer than binary questions, like yes/no.
In one study, for example, we collected brand image information and used a 5-point scale, which took respondents about 5 minutes to answer. In another study, we had people rate the same brands and attributes using a binary measure, which took respondents about a minute and a half to answer. That’s a huge difference, and comes at considerable cost.
2. Scales don’t offer “better” information
We conducted extensive research on research comparing different scales and binary measures in head to head comparisons. We consistently found that binary measures delivered either the same information, or information that pointed to the same conclusions.
3. People make mistakes when they use scales
We found that people would often change their answers based on which way the scale was oriented (left vs right, up vs down), sometimes in quite dramatic ways. For example, we found that some people who indicated on a scale that they were very dissatisfied with an organization gave us responses to subsequent open ended questions that stated they were, in fact, very satisfied. They had just used the wrong end of the scale.
The reason for the error is intertwined with why people take longer to answer a scale: scales make people count. When people count, they make mistakes. Research going back to the late 1800’s has consistently shown that people start to slow down and make mistakes when numbers get larger than three.
Since people make mistakes when they get past three, we’ll stop here with our reasons why you should rethink scales.
For more information on scale usage, including some of our research on research, read our free whitepaper Changing times demand rethinking old approaches: a case for quick, reliable, easy to answer questions.