Major OU contributions 2006 - 2015

1 Introduction

This section describes the major direct and indirect Open University developments of the eAssessment module in Moodle during 2006 to 2015 (and continuing). All of the developments that we have undertaken have been requested by Open University module teams or by colleagues in the OU's Examinations and Assessments area. In 2015 these developments are used daily by thousands of students and hundreds of staff. You may be assured that they are robust.

Throughout this period Tim Hunt has been the Quiz Module Lead Developer on behalf of the Moodle Community.

1.1 Tim Hunt Leading Technical Developer at the Open University and Quiz Module Lead Developer

2 Systems and Administration

• Ensure that Moodle is robust and reliable; we spent some time cleaning and refactoring swathes of code as we gradually gained confidence in the platform. To January 2008 Tim Hunt had resolved over 1,000 issues in the Moodle bug tracker, this large number reflecting that the Quiz module is a complex piece of software.
• Rewrite of the Moodle roles and permissions (Moodle 1.7).
• Support for tutor groups across the Quiz module to reflect OU structures and to ensure that tutors could see the results for only their students (enhanced roles and permissions in Moodle 1.8).
• Improve categories for grouping and sharing questions. This work was carried out in Asia, but the Quiz Module Lead Developer had a role in its specification (Moodle 1.9).
• Email confirmation on submission of summative assessment (Moodle 1.9). This was designed by the Quiz Module Lead Developer but paid for and implemented in Queensland.
• A re-written Question Engine supporting deferred feedback, interactive with multiple tries and confidence based marking styles (Moodle 2.1).
• Thousands of unit tests have been created to support the new Question Engine.
• Behat tests are used to ensure that question rendering remains consistent.
• An Online Examinations system for the Open University (based on Moodle 2.4-2.7)

2.1 An early view of the new question engine first seen in Moodle 2.1

3 Question behaviours

• Enhance authoring forms for questions to enable the provision of feedback and multiple tries together with updated run-time code to support the new features (some are in Moodle 1.6 – others are in Moodle 2.1); it was the case that Moodle 1.5 (2006) was very much a testing system and not a system geared to helping distance learners learn.
• Amend forms in which to build tests and to allow tests to run with different question behaviours.
  1. Deferred feedback: A single submission of responses to multiple qustions with feedback to all responses provided at one go either immediately or after a cut-off date (iCMA01 on this website).
  2. Interactive with multiple tries: Submission of responses to individual questions with instant feedback (iCMA02).
  3. Immediate: Like Interactive with multiple tries but without the multiple tries.
  4. Certainty Based Marking: Marks awarded are dependent on both the student's answer and their confidence in that answer ( with Tony Gardner-Medwin of University College, London).
  5. Adaptive: (a misnomer) Students can try and try again, with or without penalties (See the STACK Demonstration Quiz on this website).
  (1 has always been in Moodle; 5 was in Moodle 1.5; 2, 3 and 4 were added in Moodle 2.1).

4 Navigation

One of our first developments was to improve the navigation around a quiz with better signposting to the student of where they were and where they could go together with a visual record of where they had been. The new navigation was unashamedly lifted from Sam Marshall's navigation panel for OpenMark. The resulting Questions panel has been in use in OU Moodle since 2008 and was introduced to Moodle core in Moodle 2.0.

4.1 OU Moodle in 2008. The new navigation panel and a test configured to accept all answers before any are submitted for marking. Questions 1 and 2 have been answered (but can be revisited) and question 3 is in progress. Questions 4 – 8 have not been attempted.

The image below also shows that at the OU we had Interactive with multiple tries behaviour running some years before it was included in Moodle core.

4.2 The navigation panel in OU Moodle in 2008 with a test set up for questions to be answered one-by-one and a Moodle question which allows multiple tries. This is question 1; the panel shows that that question 2 has already been answered correctly and question 3 incorrectly. Questions 4 – 6 have not been attempted.

5 New question types

In 2012 - 2013 we have provided the following Moodle question types. These were all specified by Phil Butcher and implemented by Jamie Pratt using Tim Hunt's design for question plugins. If you have been following this site carefully you will not be surprised to read that all were already in use in OpenMark (and a rudimentary version of the algorithm behind pattern match was first used at the OU in 1976).

1. Combined - enables questions to be built from four sub-questions (variable numeric, pattern match, OU multiple response and select missing words
2. Drag and drop into text - drag words into sentences
3. Drag and drop markers - place a marker or markers on an image
4. Drag and drop onto images - both text and smaller images can be dragged
5. Opaque - for connecting to external question engines. Used on this site with our OpenMark system.
6. OU multiple response - with an OU defined scoring algorithm that we have used offline for 40 years.
7. Pattern match - provides sophisticated response matching features
8. Pattern match with JME - combines the Pattern match question type with Peter Ertl's Java Molecular Editor to support the automatic marking of drawn chemical structures of organic compounds
9. Select missing words - very similar to Drag and drop into text but utilises drop down lists. More flexible than Matching.
10. Variable numeric - supports the use of random numbers, variables and expressions within questions
11. Variable numeric set - supports variables with sets of pre-defined values
12. Variable numeric set with units - a combination of variable numeric set and pattern match

In 2011 - 2012 we were pleased to work closely with Chris Sangwin (then at Birmingham University, now at Loughborough University) to turn his STACK system into a Moodle question type. Saying that STACK is a single question type underplays what it will do. The STACK question type uses the Maxima computer algebra system to add sophisticated assessment in mathematics and related disciplines.

All of the above can be found in the Moodle Plugins database (http://moodle.org/ plugins) under Activities | Quiz | Question types. Alternatively go directly to the repositories https://github.com/ moodleou and https://github.com/ maths/ moodle-qtype_stack .

6 Improved authoring environment

As more features have been added to questions and new question types have come along the authoring forms have become long and unwieldy when there are HTML editors associated with multiple input fields. Thanks to Lancaster University Network Services from Moodle 2.5 it is possible to collapse/expand sections of Moodle forms. We have harnessed Lancaster's developments and applied them to the quiz and question authoring forms. It is also possible to show/hide the tool bars in the editors. The result is a much cleaner view of the structure of both quizzes and questions.

6.1 A combined question showing collapsed sections

From Moodle 2.7 there have been two usability improvements:

1. The standard duplicate icon is available for questions in the question bank.
2. It is possible to preview changes without having to close the editing form.

7 Reports

• The statistical analysis of question and test performance was rewritten for Moodle 2.0. The statistics provided are described elsewhere on this site.

7.1 Example statistics from a summative icma

• The reports were enhanced for Moodle 2.7 to statistics calculations to cope with questions that have internal variants e.g. Variable numeric set questions.

8 Gradebook

• A new Gradebook (Moodle 1.9).
• Conditional activities: Conditonal activities can be shown or hidden as a result of a student's mark in the gradebook (Moodle 1.9).

A centralised ‘Gradebook’ reports to students, associate lecturers, Module Teams and Examinations and Assessments administrative staff. The Gradebook also accepts data from external systems e.g. OpenMark, and exports to the university’s main student records system. The Gradebook was rewritten for Moodle 1.9 by Moodle HQ staff. The OU's contribution was to pay for it! OK, we also specified what we wanted. And as the client we most certainly were involved in testing, testing, testing during the autumn of 2008.

8.1 The Gradebook showing a tutor's view of a group of students' performances

9 In development in 2015

• A new behaviour 'Fault-tolerant' to be used with the OU's Online Examination System.
• Changes to the quiz construction form
  • addition of parts
  • addition of sequences
  • addition of question dependencies
  • enable questions to be repeated in situ.
• Improved support for running questions on mobile devices
  • a reworking of the superscript and subscript features used in the input boxes of the variable numeric set of questions and in pattern match.
  • drag and drop question support on Android (IOS and Windows mobile are already supported).