Skip to main content
Printable page generated Friday, 29 March 2024, 10:54 AM
Use 'Print preview' to check the number of pages and printer settings.
Print functionality varies between browsers.
Unless otherwise stated, copyright © 2024 The Open University, all rights reserved.
Printable page generated Friday, 29 March 2024, 10:54 AM

Why use eAssessment and learning outcomes

1 Why use eAssessment?

Assessment drives learning. It provides motivation and guidance. It defines standards and is the mechanism by which achievement is monitored and recognised. In order to achieve excellence in open learning, we must develop and test assessment methods and understand how excellence can be achieved consistently.

eAssessment has the potential to assist the university in tackling several strategic issues.

  • eAssessments and personalised feedback offer global delivery of quality learning opportunities.
  • eAssessments allow student profiling with links to targeted team based support.
  • eAssessments can facilitate student induction and diagnostics.
  • eAssessments can enhance efficiency and even reduce costs.

The OU's Centre for Excellence in the Open Learning of Mathematics, Science, Computing and Technology made several claims for the opportunites offered by eAssessments. The Centre also tempered those claims with several challenges.

Opportunities

eAssessments:

  • can be used across all disciplines and levels.
  • are open to place and time and can underpin targeted support systems.
  • offer long term pay back on investment.
  • engage students.
  • provide feedback instantly and can be tailored to the individual student. Feedback loops can direct the student to further questions or other scheduled activities and can embed the assessment firmly within the overall learning experience.
  • can reveal student and student cohort misunderstanding.
  • are accelerated in their adoption by faculty champions.

Challenges

eAssessments:

  • are not quick and easy to produce. Success is dependent on input from both academic authors and software developers.
  • have considerable investment costs.
  • can be fitted to pre-defined templates which lower the cost of production and improve accuracy but limit ambition.
  • require rigorous testing.
  • require a summative driver for continuing student engagement.
  • have a technical underpinning that can be a barrier to academic staff.

The Centre concluded that eAssessment for learning will be a crucial learning technology for the 21st century, particularly for those universities that aspire to a global role. Because of its exceptional pedagogical and learning systems skill base, and its position within the Open Educational Resources movement, the Open University has the opportunity to claim a central role in the further development and deployment of eAssessment for learning.

1.1 Promoting learning with instant feedback

The importance of feedback for learning has been highlighted by a number of authors, emphasising its role in fostering meaningful interaction between student and instructional materials (Buchanan, 2000), its contribution to student development and retention (Yorke, 2001), but also its time-consuming nature for many academic staff (Gibbs, 2006). In distance education, where students work remotely from both peers and tutors, the practicalities of providing rapid, detailed and regular feedback on performance are vital issues.

Gibbs and Simpson suggest eleven conditions in which assessment supports student learning (Gibbs and Simpson 2004).

  1. Assessed tasks capture sufficient study time and effort.
  2. These tasks distribute student effort evenly across topics and weeks.
  3. These tasks engage students in productive learning activity.
  4. Assessment communicates clear and high expectations to students.
  5. Sufficient feedback is provided, both often enough and in enough detail.
  6. The feedback is provided quickly enough to be useful to students.
  7. Feedback focuses on learning rather than on marks or students themselves.
  8. Feedback is linked to the purpose of the assignment and to criteria.
  9. Feedback is understandable to students, given their sophistication.
  10. Feedback is received by students and attended to.
  11. Feedback is acted upon by students to improve their work or learning.

Four of these conditions, those in italics, are particularly apposite with regard to the use of eAssessment with distance education. They are reflected in the design of OpenMark and are amplified in the rationale behind the development of the S151, Maths for Science, online assessments (Ross, Jordan and Butcher, 2006) where

  • the assessment questions provide individualised, targeted feedback, with the aim of helping students to get to the correct answer even if their first attempt is wrong
  • the feedback appears immediately in response to a submitted answer, such that the question and the student's original answer are still visible
  • students are allowed up to three attempts at each question, with an increasing amount of feedback being given after each attempt.
Buchanan, T. (2000) The efficacy of a World-Wide Web mediated formative assessment, Journal of Computer Assisted Learning, 16, 193-200
Gibbs, G. and Simpson, C. (2004), Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education, 1, pp 3-31
Gibbs, G, (2006) Why assessment is changing, in C. Bryan and K. Clegg (eds), Innovative assessment in Higher Education, Routledge
Ross, S, Jordan, S and Butcher, P (2006), Online instantaneous and targeted feedback for remote learners, in C. Bryan and K. Clegg (eds), Innovative assessment in Higher Education, Routledge
Yorke, M. (2001) Formative assessment and its relevance to retention, Higher Education Research and Development, 20(2), 115-126 Contents

1.2 Evidence that instant feedback is effective at promoting learning

Here is some raw evidence for the effect of instant feedback. The charts were prepared for questions used in a summative iCMA and we are clear that the summative nature of the test made students pay attention to and act on the feedback.

The charts show scores on an S104 question and iCMA taken by ~1,000 students.

Figure 1.1 Question 4 from iCMA49 in 2009.

Figure 1.1 shows that approximately 57% of students answered the question correctly at the first try and were awarded 3 marks. After a first hint a further ~16% gave the correct answer and were awarded two marks. And after a second more detailed hint a further 9% of students reached the correct answer and were awarded 1 mark. 18% failed to get the correct answer and scored zero.

Figure 1.2 Overall scores on S104 iCMA41 in 2008

Figure 1.2 combines all question scores across a summative iCMA to show the effect of feedback and multiple tries. If we add up the scores on the first try we arrive at the left hand chart; the sort of chart an examiner might well like to see with almost a normal distribution.

The middle chart shows the effect of adding in the scores of students who gave the correct answer on the second try and the overall effect is clearly to move the peak of the distribution to the right, to a higher score.

The right hand chart completes the story and provides clear evidence that students are acting upon the instant feedback that these iCMAs provide and are learning in the process.

2 Learning outcomes

In Hands-on Moodle Quiz you will learn:

  1. How to handle student responses containing
    1. numbers
    2. words
    3. selections
    4. 2D placements.
  2. You will learn how to give feedback to enable students to learn.
  3. You will build your questions into more substantial quizzes.
  4. And you will use performance on the quizzes to guide further study.
  5. You will also learn that writing effective eAssessment can be a time-consuming task and will be better able to judge when the input effort will be rewarded by
    1. improved student learning
    2. improved retention
    3. individualised feedback to large numbers of students
    4. 24/7/365 availability

You will end the course with your own set of ready to use questions.

N.b. The hardest task on this course is learning to use the features of Moodle to enable you to write good questions. Consequently we provide substantial time and opportunity for you to work on your own questions. But the course cannot and will not try to provide any measure of how good your questions are. To answer that you will need to put your questions in front of your own students and monitor what happens. We will show you how to use the Quiz reporting tools and suggest what you should be aiming for but it will be up to you to measure whether or not you hit those targets. Learning from your own successes and mistakes takes time too. In the long run eAssessment can improve student learning and can save you time but as with many other things you will only maximise the rewards, to your students and to yourself, by putting in sufficient effort.