With so many changes in our world since Covid-19 entered into our lives, I wanted to take a look at what is happening in survey research. Here are some of the trends that we are seeing in 2021:
Happy Methods Monday! Our clients are telling us that it is challenging to write demographic survey questions in an inclusive way. We understand. We put together guidance, including specific examples, using inclusive language to collect data on gender, sexual orientation, ethnicity, and race.
A few key points:
Lori Shelby, Ph.D., CEO & Founder
This is a holiday present for those of you who are survey research nerds like me...
Did you know that Likert is actually pronounced Lick urt? Yes, really, That is how Dr. Rensis Likert pronounced his name.
Even what a Likert scale is and is not is widely misunderstood. A five-point scale ranging from Strongly Disagree to Strongly Agree as is often seen on surveys as a way to answer a question is actually not a Likert scale. However, it is really just an ordinal scale. The misunderstanding likely stems from an oversimplification of Likert's (1932) article that has become a common misbelief over time. Likert was actually putting forward a methodology to uses a series of questions to develop a scale that includes different dimensions of a subject.He directly stated the scale labels were not important in and of themselves.
To avoid misunderstanding, it is becoming more and more popular to use the term Likert-type to refer to a 5 or 7 point scale that have a rank order and are equivalent on either side (e.g., Very Important, Important, Neutral, Unimportant, Very Unimportant). Even this is a misuse of Likert's methodology, but it is a good compromise.
Lori Shelby, Ph.D., CEO & Founder
People seem to have very strong opinions regarding how many points a survey response scale should have (4, 5, 6, 7, etc.). My advice is simple....relax. There is hardly ever only one correct way to do something in survey research, and that includes the number of points in your survey scale(s). However, there are important things to consider when choosing the best scale(s) for your survey.
Midpoints - To include or not to include?
It is becoming more popular to remove the mid-point and force respondents to choose an answer either positive or negative instead of neutral or neither.
Scenarios when it is best to include the midpoint:
Scenarios when it is best to exclude the midpoint:
Number of Points - Are more better?
Most of the arguments regarding how many points are needed come down to social psychology vs. statistics.
From a social psychology perspective, it is important for the respondents answers to be reliabile. Reliability means that if the same person was asked the same question at a later time, their answer would likely be the same (assuming no real change occurred in their thoughts/opinions). Achieving a reliabile survey is much more likely with clearly labeled scale points that correctly represent the the respondents thoughts and opinions. Consider how many scale points you can label that will have real meaning for the respondents on the survey question. If you can't come up with a meaningful label than you likely are using too many scale points. To take this to the extreme, most people don't differentiate their thoughts and opinions to the degree of strongly satisfied, moderately satisfied, less than moderately satisfied but more than neutral, neither satisfied or unsatisfied, etc.
From a statistics perspective, the more scale points the better. More scale points can increase the precision of the answers, and can result in more power to detect differences. Often when larger scales are used (e.g., 7 pt. or 9 pt.), clear labels are not included. The idea is that people can determine where they fall on a continuum without clear labels.
Although it depends on the situation, I often recommend a middle-ground, utilizing a mid-point with clearly labeled response options that reflect the likely opinions/thoughts of the respondents but still faciliate advanced analysis (i.e., 5-point scales).
Lori Shelby, Ph.D., Founder & CEO
I had the opportunity to discuss with Doug Strock, Vice President of GLTaC Inc. in Midland, MI about Translation Best Practices. It helped remind me that Translation Best Practices in Survey Research requires more attention. Here are some best practices to consider for your next on-line multi-language survey:
The Shelby Global Team
Articles and news from the Shelby Global Team.