Lori Shelby, Ph.D., CEO & Founder

Lori Shelby, Ph.D., Founder & CEO

Happy Holidays!  My family has a fun give-back tradition that we started last year, which I wanted to share today.

We choose one charity each year to donate to.  We wrap up an ornament and/or other reminder about this charity and put it under our tree. On Christmas Day, it is a great fun to open the gift and share with the family who we donated to this year. It is also a gentle reminder that Christmas is not just about giving and receiving gifts in our home.

Last year, we chose the Sheldrick Wildlife Trust (https://www.sheldrickwildlifetrust.org/), which involves a dedicated team of people rescuing elephants and rhinos in Africa. This year we chose Kiva ( https://www.kiva.org/about) another wonderful organization that crowdfunds loans to help people around the world develop their businesses, improve food security, obtain educations, etc.

We would love to hear about your give-back holiday traditions as well. Please share them in the comments. 
Note: We happen to  celebrate Christmas in our home, but this idea can  easily be adapted to Hanukkah, Kwanzaa, etc..  

Lori Shelby, Ph.D., CEO & Founder

People seem to have very strong opinions regarding how many points a survey response scale should have (4, 5, 6, 7, etc.). My advice is simple....relax. There is hardly ever only one correct way to do something in survey research, and that includes the number of points in your survey scale(s). However, there are important things to consider when choosing the best scale(s) for your survey.

Midpoints - To include or not to include?

It is becoming more popular to remove the mid-point and force respondents to choose an answer either positive or negative instead of neutral or neither.

Scenarios when it is best to include the midpoint:

• When it is reasonable that some of your survey respondents do not feel strongly or even moderately about the issue. Not giving them the choice that fits their opinion can lead to frustration for the survey respondent. 
• When you want the responses to be as accurate as possible. It is important to remember that when people respond using the mid-point of the scale, this can actually be important information. For example a mid-point on a Satisfaction scale of neither satisfied nor satisfied may well be worth following up on as the respondent was clearly not satisfied.

 
Scenarios when it is best to exclude the midpoint:

• When you want to force an opinion and are not concerned about the difference between an individual who would have answered, e.g., Disagree even when given a choice of a neutral option, and an individual who would have chosen the neutral option but made the decision to chose disagree when forced to make a choice.
• When a business is looking for actions to take to improve service, but their surveys are consistently coming back with average results near the mid-point, and follow-up with those individuals scoring items low is seen as insufficient to make significant change.

Number of Points - Are more better?
 
Most of the arguments regarding how many points are needed come down to social psychology vs. statistics.

From a social psychology perspective, it is important for the respondents answers to be reliabile. Reliability means that if the same person was asked the same question at a later time, their answer would likely be the same (assuming no real change occurred in their thoughts/opinions). Achieving a reliabile survey is much more likely with clearly labeled scale points that correctly represent the the respondents thoughts and opinions. Consider how many scale points you can label that will have real meaning for the respondents on the survey question. If you can't come up with a meaningful label than you likely are using too many scale points. To take this to the extreme, most people don't differentiate their thoughts and opinions to the degree of strongly satisfied, moderately satisfied, less than moderately satisfied but more than neutral, neither satisfied or unsatisfied, etc.
 
From a statistics perspective, the more scale points the better. More scale points can increase the precision of the answers, and can result in more power to detect differences. Often when larger scales are used (e.g., 7 pt. or 9 pt.), clear labels are not included. The idea is that people can determine where they fall on a continuum without clear labels. 

Although it depends on the situation, I often recommend a middle-ground, utilizing a mid-point with clearly labeled response options that reflect the likely opinions/thoughts of the respondents but still faciliate advanced analysis (i.e., 5-point scales).

Lori Shelby, Ph.D., Founder & CEO

I had the opportunity to discuss with Doug Strock, Vice President of GLTaC Inc. in Midland, MI about Translation Best Practices. It helped remind me that Translation Best Practices in Survey Research requires more attention. Here are some best practices to consider for your next on-line multi-language survey: 

1. Create the original survey in the native language used by the majority of the researchers, and also the native language of the survey programmer (for on-line surveys). This helps prevent errors in entering and testing multiple languages in on-line survey platforms.
2. Once the languages have been determined, create the template for translators to use to provide you with the finished translations to review.  If you are utilizing a software that faciliates uploading templates, insure you template is in the correct format. Your translations service may have also have a template they prefer that you use.
3. Meet with your Translation team to insure that context and language are clear on your template. Insure that all notes needed by the translators to produce the highest quality translations have been included in your template.
4. Use forward and back translation.  This provides you with the translated language, but also what the language is once it is translated back into the original language. It is an important step to ensure the translations are leading to equivalent meaning.
5. Have a second native speaker independently review each translation. This can be by employees of your company or the translation company. Make sure to leave plenty of time for the two translators to determine the best solution for any differences.
6. Survey research project manager should review the forward and back translations in the template. It is important to review these slowly and carefully with the full translations next to each other. On-line software typically makes this very difficult to do in the program itself due to text wrapping, hidden text, etc.
7. Once the translations are programmed into the on-line software, regardless of if they are programmed directly by the translation team, a survey programmer, or uploaded directly from the template, have all of the translators test the survey to insure it was all entered correctly.

 

​Translations are expensive and time consuming, so it is very tempting to cut corners. However, if translations are not done accurately they can render survey results difficult to interpret or even completely unusable in some cases.

For example, consider how often we are testing the statistical significance between relatively small differences in survey research (e.g., Mean = 4.3 vs. 4.5). Imagine how much of an impact it makes if your respondents were reading the question differently due to poor quality translations. Your statistically significant result showing that there is a difference between two countries, could be only because of a difference in meaning caused inadvertedly by poor quality translations.

There is another key reason why good quality translations are important...respect for your survey respondents.  Asking someone to complete a survey where the translations are not professional makes you and your company look bad. It also has the potential to cause your respondents to question your professionalism, and even to question how much your company values and respects them. Would you complete a survey where the grammar wasn't consistently correct, the language didn't flow like it should, and the wording was occasionally confusing? Your survey respondents likely will not either.

The good news is professional translators are wonderful and the profession has developed solid practices. This includes quality control processes and confidentiality standards. 

There are times when Google Translate is appropriate in survey research. I use it often to review emails and survey comments. For some studies, the comment questions will not be explored in depth. These are often only translated using Google Translate or a similar tool. However, when the comments are going to be subject to scrutiny and/or analyzed, they should also be translated professionally.

Check back next week for translation best practices.