2.2 Understanding feedback
Research has explored the data received as feedback from students in various ways. The findings have not always been consistent, but they do provide some useful considerations for practice.
Johnson (2003), in a summary of several studies, concluded that teachers who mark more leniently are routinely rated higher in student evaluations, and that students who receive higher grades also give more favourable feedback. Centra (2003) found that more rigorous courses received lower ratings than ‘easier’ courses. Given these factors, Parker (2013) goes on to consider three means of assessing teaching effectiveness and course quality which are less likely to fall victim to these biases:
- evidence-based teacher self-reviews.
- peer observations.
- external reviews.
If you watched the video of Professor Bart Rienties discussing learning analytics in the previous section, you might recall that research using data from The Open University does not show a correlation between attainment and satisfaction. This satisfaction data is collected through a survey delivered to students near the end of the course, but prior to the student finding out their final grade. It may be that this earlier timing of the survey removes a direct impact of grades on satisfaction. So perhaps we need to think carefully about when and how we ask for feedback.
Students perceive online courses differently from traditional courses. In comparisons of online and face-to-face MBA courses, Cao and Sakchutchawan (2011) found that whilst there was no difference in success rates between students of online courses and students of face-to-face courses, the online MBA students reported lower satisfaction with their courses. Song et al. (2004) examined feedback from online graduate students and concluded that course design and time management were crucial components to successful online learning, while lack of community and technical problems were most challenging for online learners.
The design of online courses, and the ways in which instructors act, impact on the perceptions of students towards them. Kauffman (2015) gathers together a range of studies examining the success of online learning in various contexts and concludes that ‘courses should be structured around reading materials, lectures and assignments organised into units with clear learning goals in mind’. In other words, course instructors need to ensure that there is alignment of objectives with instructional methods, learning activities and assessment methods (Blumberg, 2009). Instructors should provide timely feedback and serve as facilitators of discussion and interaction just as they do in traditional courses. Courses should provide opportunities for peer collaboration and sharing of ideas in order to develop an online community of learners, rather than feelings of isolation (Song et al., 2004). Otter et al. (2013) used questionnaires to determine differences in perception between online and face-to-face courses, among students and staff. Findings showed that students perceive online courses to be more self-directed than staff do, and that students online must be more willing to teach themselves. Students in online courses felt more disconnected from staff and fellow students than staff perceived they would feel. Students also have a lower perception of the role of the teacher in online courses than staff do.
Activity 2 Effective use of questionnaires
Create up to five questions that you think would provide useful feedback for you from your students. While doing this, think about the following:
- How would you make sure the questionnaire is not leading students to respond in a particular way?
- Does your institution have a standard feedback questionnaire that is given to students? If so, are there questions that you can take from this? Is it appropriate to online teaching?
- What type of responses would you like? (For example, closed questions on a scale, or open comments, or a mixture of both?)
- How would you analyse the results?
This activity helps you to think about one specific set of data that you would like to obtain, and how you might go about it. The questions asked need to be considered very carefully in order to ensure that the data generated is useful to you.