5.4 Questionnaires and surveys

Questionnaires are a mainstay of much educational research. Certainly, the success of the Open University in the annual Student Satisfaction Survey is always greeted with much institutional approval.

Most practitioner research does not include surveys of the scale of the Student Satisfaction Survey, and so what is described here is the use of generally smaller-scale questionnaires, perhaps up to 100 or so students rather than thousands.

Benefits of using questionnaires

Questionnaires can be a good way of getting factual information from a group of people, particularly where there is not likely to be any confusion about the precise meaning of the questions asked. So, you might want to find out about their use of ICT, their prior learning experience, the time students spend studying, etc. As such, they are very good at proving base data, against which other data, perhaps more qualitative in focus, can be compared. Conversely, they can also be used to further test findings which are emerging from the qualitative data. As such they can be a very good method of triangulation.

Questionnaires can also be used as part of the selection of participants. It may be, for example, that you wish to interview students with a particular educational background, or who represent different levels of use of ICT. The questionnaire can help you establish which students satisfy these particular requirements.

Finally, although the focus here is on smaller-scale research, questionnaires do allow the researcher to reach a much larger group of participants than is the case in observations, interviews or even focus groups.

Using questionnaires with a tutor group
Despite their associations with large numbers, questionnaires can be used with smaller numbers of respondents. In this context, they are probably best be used for obtaining quite factual information on paper (or online) rather than asking students for that information in an interview..
Such questionnaires can be very simple in structure and are as easy to administer on paper or by email as they are electronically. Given the small numbers involved, the data they produce cannot meaningfully be used in any statistical way.

Using questionnaires with a larger group of participants
The first issue to consider when using questionnaires with larger groups of students is that of response rate. In a small tutor group, it is likely that the majority of students will respond to your request for participants, almost irrespective of the method used. For larger groups, it is worth considering that the average response rate for internal OU surveys is 25%. This is well above the average for most of education and industry but may result in smaller numbers than you perhaps originally intended. As will be discussed in the section on SRPP, you will be asked to indicate the number of students you require for your project, and the expected response rate.

The larger the number of participants you survey, the more likely you are to be able to make claims based on the data. As such, larger-scale questionnaires can be used not just for basic factual information, but for asking about people’s opinions and emotion, quite often using response scales, such as the likert scale. However, it also means that you have to be extremely careful about the questions you ask. For questions which require more subjective responses, such as the extent to which agrees or disagrees with particular statements, particular care needs to be taken in ensuring that those responding to the questionnaire have the same understanding of the questions asked, and the responses provided, as you do. Otherwise, the data you obtain, whether from ten or a 100 students, will be meaningless. As such, the questions will need to be tested and piloted before being used in earnest.

Disadvantages of using questionnaires

Perhaps the main disadvantage of using questionnaires is that, unlike questionnaires, there is no opportunity for the researcher to check the understanding of the participant or to follow up particular responses. As a result, as indicated above, there is a greater likelihood than in other methods that those completing questionnaires may be responding in a different way to that anticipated. This problem is not insurmountable but it is important for the researcher to be aware that larger numbers do not necessarily imply greater validity.

Guidance on using questionnaires as a research method

Anne Jelfs and John Richardson, from the Open University, have both written very helpful Learn About guides on this topic. Anne Jelfs' guide is on Questionnaire design and is very practical in tine.  John Richardson's guide on survey practices covers issues such as respondent bias and reliabiaity and validity and also has some links to other resources on questionnaires and surveys. They can be found through the following links:

http://epd.open.ac.uk/browseLAG.cfm?lagID=63&method=displayLAGDetails

The Plymouth University Research in Education (RESINED) website, mentioned earlier, has a good section on using questionnaires in educational research. The sections includes explanantions of questionnaire design, including the likert scale, sampling and analysis.
http://www.edu.plymouth.ac.uk/resined/QUESTS/index.htm

Examples of research which has used questionnaires

 
Jacqui Ansell, a Consultant for the Personalised Integrated Learning Support (PILS) CETL, used a small scale questionnaire with her tutor groups in her project, The Perceived Impact of Additional Study Support Material.
http://intranet.open.ac.uk/cetl/pilsintranet/pics/d90295.doc
Last modified: Tuesday, 4 Mar 2014, 16:36