3.3 Appropriate questions
What might seem like a simple, appropriate question, could be interpreted very differently by different people. It is good practice to test any questions being asked as part of a statistical investigation using a pilot survey. A pilot survey is a small survey done by the researcher to test the questions to be asked on a small group of people before undertaking the full survey. In this way, any issues such as misinterpretation, ambiguous wording or inadequate answer category selections can be addressed and rectified before the full survey is undertaken. To maintain consistency, survey data collected as part of the pilot survey is not used as part of the main survey data.
Activity 5 Good, or not so good, questions?
Consider the following questions and answer options.
Decide if you think it would be appropriate to ask these questions as part of a statistical investigation.
Note you are not answering the questions themselves here, rather whether you think they are appropriate questions to ask.
Dale to check functionality of activity
1. Do you have breakfast every day? □ Yes □ No
Is this question Appropriate or Inappropriate?
Comment
This question is specific and clear. The time category ‘every day’ has the same meaning for all. Most people should be able to answer ‘yes’ or ‘no’ so all possible options for answers are covered. However, it could be helpful to have a third category for a yes/no question such as ‘don’t know’ or, in this case, ‘prefer not to say’.
It might also be better to ask about a restricted time period such as ‘in the last week’. This then removes any doubt, if say, someone did not normally eat breakfast five months ago but does now.
2. How many times in a typical school week do you walk to school?
□ Always □ Sometimes □ Occasionally □ Never
Is this question Appropriate or Inappropriate?
Comment
The words ‘sometimes’ and ‘occasionally’ can mean different things to different people. ‘How many times?’ would be better answered with options such as every day, 3-4 times, 1-2 times, never.
3. Don’t you agree that it takes too long to get an appointment to see the doctor?
□ Yes □ No □ Don’t know □ Prefer not to say
Is this question Appropriate or Inappropriate?
Comment
This is an example of a biased question. A biased question is one that is deliberately favouring a particular response or leading the respondent towards a particular response. By using the phrase ‘don’t you agree’ the person asking is influencing the respondent with their own opinion. It is a known phenomenon that some respondents are more likely to give the answers they think the researcher wants to hear rather than those which reflect their own opinion.
4. How many siblings do you have?
□ 0 □ 1 □ 2 □ 3 □ 4 or more
Is this question Appropriate or Inappropriate?
Comment
If the definition of ‘siblings’ is clearly communicated, then this is a well written question with appropriate response options.
5. “Eating plenty of vegetables every day is good for you”
Tick which box best represents your opinion
□ Strongly agree □ Agree □ Neither agree nor disagree □ Disagree □ Strongly disagree
Is this question Appropriate or Inappropriate?
Comment
The statement here seems to be implying that eating vegetables is good for you so this could be deemed as a ‘leading’ question i.e. one which is leading the respondent to answer in a particular way. However, the answer options cover the full range of agreement, so this gives respondents the opportunity to disagree if they choose.
The statement itself, however, is quite vague e.g. what is meant by ‘plenty of vegetables’? There is a possibility this could be interpreted as excessive eating of vegetables and eating any one food group in excess is not recommended.
6. Approximately, how many times a week do you brush your teeth?
□ 0 □ 1-5 □ 5-10 □ 10-15 □ more than 15 □ Prefer not to say
Is this question Appropriate or Inappropriate?
Comment
Whilst this question is fine it may be deemed too personal. This is, however, covered by the ‘prefer not to say’ option. A more obvious issue with this question is that some of the categories overlap. For example, someone who brushes their teeth 10 times a week could choose 2 categories.
Bias:Tend to favour one thing over another, whether intentionally or accidently.
Leading: To have a directing influence; guiding in a particular direction.
Leading questions are biased. Thinking about the wording of the question/s that will be asked and whether it is biased in any way, vague or too personal are all considerations in the planning stages of any statistical investigation. It is difficult to know how questions will be interpreted by others until they have been tested.