SHELBY GLOBAL, L.L.C.
  • Home
    • How Can We Help You
  • Services
    • Project Management
    • Surveys, Interviews, & Focus Groups
    • Data & Analytics
    • Reporting
    • Strategy & Evaluation
  • Vision
  • About
    • Founder & CEO
    • Team
    • Partnerships
    • Community
    • Blog
  • Contact Us

Our Blog​

An Easy Way to Increase Reliability and Validity for your International Surveys

11/29/2020

0 Comments

 
Lori Shelby, Ph.D., Founder & CEO
When conducting surveys internationally, it is important to remember that not all cultures read left to right or utilize the same conventions for what is a logical order of response options. This issue primarily impacts surveys, which include parts of Asia and the MIddle East.

Researchers typically become aware of this issue when they see respondents reply inconsistently in a survey. For example, indicate they were Strongly Dissatisfied on a 5 point scale with a service, but then comment later how happy they were with the same service. When you see this across multiple surveys in your study, it is clear you have an issue. 


The easiest way to ensure that your survey questions are understood for international surveys is to not put the scale numbers in the questionnaire itself.  You can still code the numbers in your dataset behind the scenes. This way you remove the potential confusion and increase the reliability and validity of your survey responses.
Picture
0 Comments

The Order of Your Survey Questions Matters

11/23/2020

0 Comments

 
Picture
Lori Shelby, Ph.D., Founder & CEO
These three rules may seem like very straightforward survey construction rules, but for those new to survey research they are easy mistakes to make and worth a review:

  1. Start with your simple, interesting questions first. When you start with uninteresting or involved questions, such as demographics, it encourages your respondents to quit taking your survey immediately.
  2. Group all of the questions regarding a topic or sub-topic together (e.g., questions on placing an order, customer service questions, product quality questions, delivery questions). This makes it easier on your respondents, as they can think about one topic thoroughly, and then move on to the next topic.
  3. Put your sections in a logical order for your survey respondent (e.g., in the order of their experience as a customer). Again, this makes it easier on your respondents and encourages them to complete the survey.

Basically, put yourself in the respondent's shoes as they take your survey, and think about how to make it as interesting and easy for them as possible.


0 Comments

​Top 5 Ways to Write Better Survey Research Questions

11/16/2020

0 Comments

 
Picture
Lori Shelby, Ph.D., Founder & CEO
Here are five top ways to write better survey questions:
  1. Use simple words and avoid jargon. 
  2. Do not use acronyms without explanation.
  3. Make the questions short and focused on only 1 topic.
  4. Cover as many options as possible in the response options.  Make it quick and easy for your respondents to respond.
  5. Keep your survey questions neutral. Avoid leading questions.
0 Comments

Survey Research Tips: Don't just pre-test, engage with your pre-testers.

11/9/2020

2 Comments

 
Picture
Lori Shelby, Ph.D., Founder & CEO
What is one of the most powerful things you can do to improve your survey and increase your response rate? Pre-testing! Pre-testing is essentially a practice run of your survey that helps you determine how effective it will be.

Take the opportunity to go beyond just collecting answers to your questionnaire during pretesting. Engage with your respondents directly.  This method of pre-testing is called participating pre-testing.

Some example questions to ask your pre-testers: 


  • Which questions are potentially misleading or may result in a biased response? Why? 
  • Which questions are unclear? 
  • Did we accidentally put any jargon or advanced vocabulary in the survey?
  • Are there any questions that made you angry or made you want to stop taking the survey? 
  • Do you think we are missing something that we should be asking to understand the subject(s)?
  • Did you have any technical issues with taking your on-line survey? 
  • Did the invitation provide the information you expected? 
  • Would your invitation email entice you to click on the survey link and take the survey?

Keep an open mind and take action to improve your survey based on the answers your pre-testers give you, Your survey will almost always be improved leading to a better survey experience for your respondents and the potential for an increased response rate.
2 Comments

SHOULD I TRUST THE POLLS THIS YEAR?

11/2/2020

0 Comments

 
Picture
Lori Shelby, Ph.D., Founder & CEO
The last blog post in our series on voting methodology for #MethodsMondays takes a look at the challenges of polling accurately for U.S. presidential elections. Although the topic is of general interest to survey research geeks like myself, before I start on the details I wanted to clarify that no polling method at this historic time should be considered infallible.  It is human nature to choose not to take the time to vote when on the news they are already calling who is expected to win in your state.  I would encourage you this year to vote when you can and ignore the poll results, which may or may not be correct. Let's Vote!

Don't get me wrong, there is a great deal of science behind polling and as someone who has built a career on survey research, I believe in the research. However, we all remember Hillary Clinton was expected to win by approximately 85% in the last election. It is reasonable to infer that this suppressed voters who just assumed their vote wasn't important, thus helping President Trump to win. The American Association of Public Opinion Research released a well researched and considered report: An Evaluation of 2016 Election Polls in the U.S.   Reasons for the discrepancy included late decisions by voters on who they were going to vote for (about 14% of voters in Michigan, Wisconsin and Pennsylvania in the final week). Also, there was an overrepresentation of college graduates in the polls, which were not properly adjusted during the analysis. Most pollsters have now fixed this mistake.

However, the need for survey researchers to learn from their mistakes is concerning in an election where there is significant and unprecedented events occuring. For example, the uncertainty of Covid#19, increased voting using alternative methods, a growing fear of violence at polling places on election day, and the potential for an unprecedented voter turnout. Here is a great article explaining how the polling modeling has improved over the last 4 years and electoral college math that explains why Biden may be up in the polls, but how President Trump could still win.  The Pew Research Center released on article on what we can trust the 2020 election polls to tell us a few days ago.

In my opinion, surveys are a valuable tool. But, perhaps in this case they are of more value for understanding why people are voting as they did, and why election results happened as they did.  We will hopefully see final election results very soon and that is when we can start discussing and learning from the accuracy of the 2020 presidential election polls. 
0 Comments
<<Previous

    The Shelby Global Team

    Articles and news from the Shelby Global Team.

    Archives

    December 2020
    November 2020
    October 2020
    September 2020
    August 2020

    Categories

    All
    Data & Analytics
    International Surveys
    Interviews
    Methods Mondays
    One Tree Planted
    Project Management
    Survey Research Tips
    Surveys
    Welcome

    RSS Feed

Copyright © 2020 Shelby Global, L.L.C.
  • Home
    • How Can We Help You
  • Services
    • Project Management
    • Surveys, Interviews, & Focus Groups
    • Data & Analytics
    • Reporting
    • Strategy & Evaluation
  • Vision
  • About
    • Founder & CEO
    • Team
    • Partnerships
    • Community
    • Blog
  • Contact Us