When students are using an iCMA on their Moodle website they will not see the word Quiz (unless you use it in the name of your iCMA). The Moodle Quiz is that part of the Moodle system that supports our use of iCMAs.
The key roles for course websites that hold permission for creating iCMAs are 'workflow editing' and 'activity workflow editing'.
Now 'workflow editing' is usually associated with the 'website updater' role. So anyone who is creating iCMAs must be a 'website updater'. Authors wishing to write iCMAs should request the 'website updater' role.
It is also the case that anyone who is reviewing individual questions that are in an assessed iCMA must also be a 'website updater'.
The reasons for this arrangement are associated with removing the ability to edit questions when an assessed iCMA is live. By putting the permission onto the workflow roles the workflow is able to control when authors can, and cannot, edit questions.
N.b. To create an assessed iCMA in Moodle please ask the Curriculum Manager to initiate the process using the form provided in Administering an assessed iCMA on this site.
You will need permission to edit the module in which you wish to create the iCMA. Such permission is conferred through the Website updater 'role' and through a role associated with the work flow. These roles are set up when the module is created. If you do not have the correct roles please contact your LTS Online Services team.
Once you have the roles you will find that you are allowed to turn editing on (and off). Turning editing on/off is available in the site Administration block.
Turn editing on by clicking on the link. Your module site will refresh and display the drop-down list boxes that allow you to add new resources and activities to the module. They will look like this
To add an iCMA open the 'Add an activity...' drop-down list and select iCMA from the list of displayed options.
It is useful to start by completing the iCMA definition form as this sets the context for the questions that you will write.
You will be faced with two major choices.
You will need to decide on the 'Question behaviour' which can be
The differences between the two behaviours are described and exemplified elsewhere on this website. Choosing the behaviour at the outset ensures that you will be clear about how to write your questions together with the feedback and hints that you will have to include.
You will also need to decide if your iCMA is
If it is the latter this will already have been agreed as part of the module's formal assessment strategy.
The results from all iCMAs can be displayed in the Moodle Gradebook and those for assessed iCMAs are dispatched to the Student Records system from the Moodle Gradebook. Module Teams using iCMAs will want to familiarise themselves with the Gradebook which is also described on this website.
Assessed iCMAs must have certain settings that have been agreed with Exams and Assessments. The workflow described in Administering an assessed iCMA guides authors in choosing the correct settings.
For formative iCMAs the default settings enable you to create an iCMA by completing just one or two settings.
That's all. Now press Save and you have your empty iCMA ready for you to add questions.
The General section of the iCMA form looks as follows:
Name: Supply a short name for the assessment e.g. iCMA41; Block 2 Questions. It helps to make the name short as it is used in column headings in the Gradebook and the shorter the name the smaller the on-screen space that is occupied in these reports.
Introduction: Your introduction is shown to students before they start the iCMA. You may use it to set the scene, indicate the purpose of the iCMA within the module, say how long it should take, whether it can be repeated etc.
The introduction is available to students before the iCMA is open.
It is possible to revisit this iCMA definition form any number of times and consequently it is not necessary to write the final version of your introduction at your first visit, but knowing the reasons for creating the assessment and its role in the module will help you to come to a view on the number of questions that the assessment should contain and hence the student and author time involved.
The Timing section of the iCMA form looks as follows:
Open to students: To make the assessment available from a specified date and time, check the 'Enable' box and set the start date and time. It is standard practice to open assessed iCMAs at 00.00. The assessment will be made available automatically once this time is reached.
To make the assessment available as soon as it is visible (see below) leave the 'Enable' box unchecked.
Close to students: To make the assessment close at a specified date and time, check the 'Enable' box and set the close date and time. It is standard practice to close assessed iCMAs at 23.59 hours (essentially midnight). The assessment will close automatically once this time is reached.
To leave the assessment open for the duration of the module leave the 'Enable' box unchecked.
When time expires : One of
From 13B presentations formal iCMAs will allow a grace period. Students who have started an iCMA but not submitted before the closing time will receive an email reminding them to submit if they wish to have their responses marked. This setting will be actived by the iCMA workflow.
The normal setting is 'Attempts must be submitted before time expires...'.
Submission grace period: Only used for a formal iCMA and normally 7 days.
These settings are set to the OU defaults and are rarely changed.
Time limit: Default OU setting = 0.
The Grade section of the iCMA definition form
Grade category: The category in the Gradebook under which the iCMA is listed.
Attempts allowed: For a formative assessment it is likely that you will wish students to attempt the assessment as often as they feel necessary both immediately after studying the associated text or as a revision exercise prior to examinations. Consequently for formative assessments 'Unlimited attempts' is a sensible setting.
For an assessed assessment the correct setting is '1'.
These settings are set to the OU defaults and are rarely changed.
Grading method: Default OU setting = Highest grade. This setting controls what is sent to the Moodle Gradebook. Moodle keeps the scores on all attempts and can therefore offer a variety of grading modes. For an iCMA that forms a formal part of a student’s module marks, students can make only one attempt and hence all of Moodle’s ‘Grades’ give identical results. For formative iCMAs perhaps it is best if ‘Highest grade’ is always used.
New page: We recommend a setting of 'Every question'. If you have an image that you wish to ask several questions about and you want those questions to appear on one page you will find that it is straightforward to remove page breaks when you come to add questions to the assessment on the Edit page.
Question order: Default OU setting = As shown on the edit screen. Setting this to 'Shuffled randomly' makes it harder for students to share answers, but for formative use it also makes it harder for tutors to respond with additional teaching on difficult concepts.
Navigation method: Default OU setting = Free.
How questions behave: The choices are:
Note that Essay questions are not marked by any of the above and have to be manually graded.
Certainty based marking ( http://docs.moodle.org/ 26/ en/ Using_certainty-based_marking ) asks students to reflect on their knowledge and indicate how certain they are about their answer. This certainty level is then used in assigning marks. A high degree of certainty in an incorrect answer results in a substantial penalty. Low certainty gives a lower reward but no risk of negative marking. This encourages reflection and rewards acknowledgement of uncertainty, while also retaining conventional information about student accuracy.
|Student's certainty||Mark if correct||Mark if incorrect|
Where questions are assigned weights different from the default (1.0), marks are multiplied by the weight. Partially correct answers are given the appropriate fraction of the mark for a correct answer.
Shuffle within questions: Default OU setting = Yes. When testing an iCMA it is sometimes easier for the tester if shuffling within questions is turned off temporarily. This setting overrides the setting within individual questions.
Each attempt builds on the last: Default OU setting = No.
This section controls what is displayed during and after an assessment has been submitted. The matrix offers a wide variety of settings that control the visibility of certain question elements.
During the attempt: settings for use with interactive with multiple tries and immediate feedback question behaviours
Immediately after the attempt: i.e. as soon as the assessment is submitted.
Later, while the iCMA is still open: At the OU we expect this column to be the same as the 'Immediately after the attempt' column.
After the iCMA is closed: i.e. after the closing time and date specified on this form.
The attempt: Controls whether an attempt and student responses can be reviewed. At the OU we would expect these boxes to be checked.
Whether correct: Controls the use of colours behind the navigation panel and student's response and the appearance of Correct/Partially correct/Incorrect in the status panel that is found between the navigation panel and the question. The colours are accompanied by hidden textual information for screen readers.
Marks: Controls the display of marks alongside each question. This setting also controls the display of the overall grade on the iCMA.
Specific Feedback: This controls the display of the feedback that is supplied to specific responses on the question editing form.
General feedback: This should be shown to all students irrespective of the correctness of their answer. It should contain the correct answer to a question and an explanation of why that answer is correct.
Right answer: This will usually not be checked. If checked Moodle automatically adds information on what is and is not the correct answer to each question, however how the system does this is not satisfactory for use at the OU. We strongly recommend that authors use the General feedback box to inform students what the correct answer is and why.
Overall feedback: This can be matched to the student’s overall grade on the iCMA and is completed at the bottom of the iCMA definition form. Please see the below.
All of the settings in this section are Advanced settings which should not need changing for OU use.
Show the user's picture: OU default setting = No.
Decimal places in grades: OU default setting = 2. Controls the form in which numbers are shown in reports. Because this setting has an impact in more than one place we strongly recommend that this is left at the default setting.
Decimal places in question grades: OU default setting = Same as for overall grades. Controls the form in which numbers are shown in the status panel. We recommend that this is left at the default setting.
Show blocks during quiz attempts: OU default setting = No
Students cognisance of plagiarism policy: The choices are
The latter setting results in the following being displayed when an iCMA is first started.
Please read the following message
The University has to ensure as far as possible that the work submitted by students is their own and does not contain any unreferenced material from other sources. The University statement 'What constitutes plagiarism or cheating' appears in the Plagiarism Policy Document and in your Assessment handbook and you are expected to have familiarised yourself with it. Please be aware that the University may subject your work to checks (including computerised checks) for plagiarism and in some instances may need to seek further information from you. Any cases where a basis for concern about the originality of the work is identified will be considered under the University's policy on academic conduct.
I confirm that I have read the statement 'What constitutes plagiarism or cheating' and that my answers will be my own work.
Questions are withheld until the student gives the appropriate confirmation.
This confirmation is only required on the first visit to an iCMA.
All iCMAs which are a formal part of the module assessment policy should be configured to require students to acknowledge that they must not share answers.
Require password: Default OU setting = <empty>. Access to an assessment can be controlled by setting a password here and subsequently controlling the distribution of that password.
Require network address: Default OU setting = <empty>. This setting may be useful for campus based universities that only allow access to an assessment from certain computers, perhaps under invigilated conditions in supervised rooms.
Enforced delay between 1st and 2nd attempts: Default OU setting = None.
Enforced delay between later attempts: Default OU setting = None.
Browser security: Default OU setting = None. (Note that in reality the particular security feature that is available is not very secure.)
When the assessment is submitted an overall grade will be calculated that is based on the students individual marks on each question (this is defined at the next stage of building the iCMA). On The Review options section above if the 'Marks' box is checked the student will be presented with their overall grade and if the 'Overall feedback' box is checked whatever comment you add into the appropriate feedback box above will also be displayed.
Be careful to enter consistent scores in all the Grade boundary boxes; it is perhaps advisable to stick to percentages - be sure to include the % symbol.
Visible: Set this to 'Hide' while you are building the assessment and change this setting to 'Show' when the assessment is ready for use.
ID number: This is essential for assessed iCMAs and is used for identifying results to the Student Records system. The format of the ID number is module-presentation.assignment e.g. m120-07j.icma41. Upper or lower case can be used as the interface program in the Student Records system converts all module codes, presentation codes and PIs to upper case as it reads them in.
Group mode: to ensure that tutors can see only the results for their tutor group this setting should be set to 'Separate groups'. If your module does not have tutor groups choose 'No groups'.
Grouping: Default OU setting = None. if your module has tutor groups the Grouping should be set to 'course-presentation Tutor Groups' (e.g. OU123-11B Tutor Groups). If your module does not have tutor groups choose 'None'.
Available for group members only: Default OU setting = Not checked. Leave this at the OU default setting of not checked.
Allow access from: This controls the appearance of the link to the iCMA on the page. To make the link appear from a specified date and time, check the 'Enable' box and set the start date. The link will appear automatically once this date is reached.
To make the link appear as soon as it is visible (see above) leave the 'Enable' box unchecked.
Allow access until: To remove the link from the page after a specified date check the 'Enable' box and set the until date. After this date and time has passed the link will be removed automatically.
To leave the link in place for the duration of the module leave the 'Enable' box unchecked.
Grade condition: Use this setting to tie the availability of this iCMA to the attainment of some other grade, or grades, held within the Gradebook.
User field: Restrict access based on a field from the users profile.
Activity completion condition: This setting determines any activity completion conditions which must be met in order to access the activity. Note that completion tracking must first be set before an activity completion condition can be set.
Multiple activity completion conditions may be set if desired. If so, access to the activity will only be permitted when ALL activity completion conditions are met.
While access is prevented: Two settings are available, greyed-out or hidden.
Completion checking: If enabled, activity completion is tracked, either manually or automatically, based on certain conditions.
Require view: Self explanatory.
Require grade: If enabled, the activity is considered complete when a student receives a grade. Pass and fail icons may be displayed if a pass grade for the activity has been set.
Expect completed on: This setting specifies the date when the activity is expected to be completed. The date is not shown to students and is only displayed in the activity completion report.
Please save the iCMA settings using ‘Save and display’.
Your iCMA is now ready for questions to be added. If your questions have already been created and reviewed then please carry on with the next section Build your iCMA. If you have still to write your questions please refer to the separate document How to write questions in Moodle.
This is described in a separate document, How to create questions in Moodle which you will find on this website.
At Step 1 of these instructions you created an iCMA. At that time you set the iCMA parameters but did not add any questions. Now is the time to add questions.
To do this return to the module page or sub-page containing the iCMA and click on the name of your iCMA. Then click on 'Edit quiz' in the Quiz administration block.
The screen that appears has three main sections with the list of questions in the iCMA down the centre and the Question bank to the right. To build a quiz you need to be familiar with
Category: This drop-down list box enables you to choose which of your categories of questions are displayed.
The '<<' which allows single questions to be added from the question bank to the iCMA.
The button which allows multiple, selected, questions to be added at once. And the
And the control which instructs Moodle to choose the question, at run time, from within the specified category. Should you choose to add multiple random questions, Moodle will ensure that there is no duplication in its choices.
Start with the '<<' and add a question. The screen will refresh and the question will appear in the central list of questions.
Further questions can be added in a similar manner. To save time rearranging the order of the questions later it is best to start by adding the questions in the order that you wish them to be presented. Indeed it is better still if you have already named your questions so that you can re-order them easily in the question bank before adding them. You may reorder the questions in the question bank by clicking on the headings that are found above the questions.
Within the iCMA list, questions may be reordered using the up and down arrows or by using the tools on the ’Order and paging’ tab.
Continue adding the questions until your iCMA is complete. When your iCMA is complete set the Maximum grade to be the same as the Total. (Unfortunately it is not straightforward to do this automatically and you must change the Maximum grade manually.) The iCMA reports then report scores out of the maximum e.g. 10/10. In the Gradebook which gathers grades from multiple iCMAs all grades are shown as percentages.
You are now ready to preview your iCMA to see how it works in practice. At this point you are still able to amend your iCMA.
Moodle provides various mechanisms to enable you to vary the questions which are given to students who are doing the same iCMA.
Moodle keeps a record of which student has received which question and the reports provided also contain this information.
Having added all your questions select the 'Preview' link from the Settings on the left of the window
and you will be able to interact with your iCMA usingthe settings specified on the iCMA definition form.
As you are in Preview mode the VLE will not keep long term records of your responses but it does keep sufficient records to enable you to exit and return to where you were at a later date. It is also possible to restart the preview multiple times to enable you to test different responses.
Experienced authors may choose to check their iCMAs by careful reading of the question editing forms. It is possible to review, and print, the editing forms for all questions in an iCMA at one go. Where questions are chosen randomly from a category all available questions are included.
If you alter the numbers of questions or change the types of questions at this stage of the build process please ‘Save’ between previews. Here is the reason why:
If you do not follow the above instructions you are likely to see errors when you start a preview.
Your iCMA should be thoroughly checked by a colleague before it is made live. As your iCMA will still be hidden your colleague will require the 'Website updater' and Workflow editing' roles and they should test the iCMA using Preview mode.
Please note that if your iCMA is using OpenMark questions from an OpenMark server you must move the questions to the live server and amend the question forms before you make the iCMA visible.
This can be done either at the module level by clicking on the ‘eye’ or on the iCMA definition form, in the Common Module Settings, by setting ‘Visible’ to ‘Show’.
The iCMA is now ready to be used by students and will operate according to the settings that you have made on the iCMA definition form.
You must not alter the questions in an assessed iCMA after it has gone live and we configure the VLE such that you will not be allowed to make such edits. If problems do become apparent once the test is in use you should discuss possible corrective action with Exams and Assessments.
The VLE allows you to edit questions even after students have used them. This appears to give you 'carte blanche' but, of course, things are not as straightforward as that. Take for example a question where you remove an option. How can a student return and review their responses and the feedback if an option they chose is no longer in the question? While the VLE tries to cope, what it is able to do can no longer be considered to be a faithful review and you will inevitably get queries from baffled users (I write this to warn you that sorting out such queries takes far longer than adopting the approach below).
We therefore offer the following advice as to what can be done when a question is already in use:
Beyond this if a question is creating too many issues we recommend that you:
This page is an example of what might be provided to students.
Each iCMA comprises a number of separate questions. You should attempt to answer all questions.
The navigation panel on the left-hand side shows questions that you have answered and those that are currently unanswered. How it does this depends on the type of iCMA.
Please note that if you answer a question but your answer is incomplete* the question background will remain white. * Please revisit the question to see why your response is incomplete.
You may move directly to any question by clicking on its number in the navigation panel. To move to the next question you may also click on the ‘Next’ button, which appears below the current question.
You may answer a few questions, close your browser or turn off your computer, and return later. However, the last answer you enter will only be recorded if you move to another page within the iCMA or check your answer. We recommend that you always visit the Summary to gauge your progress before exiting. You can reach the Summary page by clicking on ‘Finish attempt…’.
When you have answered all questions you should click on ‘Finish attempt...’, which will lead you to the Summary page.
All questions should be answered. You may return to any question by clicking on its number.
Once you have answered all questions you should 'Submit all and finish'. This will complete your attempt on the iCMA and your answers will be marked.
Here question 3 is ‘flagged’ (see below). Please note that if you answer a question but your answer is incomplete the Summary will report this as shown with question 7. Please revisit the question to see why your response is incomplete.
All questions should be completed. You may return to any question by clicking on its number.
Once you have completed all questions you should ‘Submit all and finish’. This will finish your attempt on the iCMA and you will receive feedback on your overall performance on the assessment (if the iCMA author has provided this).
Wherever you are presented with an answer box, please click in the box so that the cursor appears and then enter your answer.
The flag icon is provided to enable you to ‘flag’ questions that you wish to return to. Clicking on the flag turns it red and places a similar flag alongside the question number on the Summary screen.
To review your answers you will be provided with the navigation panel again. You may click on a question number to review your answer. If your answer has been marked it will be shown when you review the question. Also when the questions have been marked the question numbers on the panel will be given coloured bars; green for correct; amber for partially correct and red for incorrect.
When you click 'Submit all and finish' you complete an attempt at the iCMA. Your responses, marks and feedback for all attempts are stored and you may review them later by revisiting the iCMA. When you revisit the iCMA you will be offered the chance to start it again but you will also see a list of previous attempts.
Following the ‘Review’ link for the appropriate attempt, enables you to review that attempt.
Formative iCMAs are provided to help you to improve your knowledge and you are encouraged to repeat them as often as you wish. Summative iCMAs are assessed and you may only make a single attempt.
As soon as your students start to use your iCMA things get more interesting. Moodle provides various reports which you can use to monitor your students' performance and how your iCMA and individual questions perform. To access the reports open an iCMA and when it appears select the results link.
It is important to understand that when reviewing a student's performance module team members and tutors may see information that is not visible to students. In particular
are visible to module team members and tutors when reviewing a student's attempt.
The Results section expands to show a variety of reports. The full list is described here but what is shown to whom depends on their role on the module.
Grades: Lists students who have interacted with the iCMA together with dates and grades achieved. Students who have made multiple attempts have multiple entries beneath their name. By clicking on a date or a grade it is possible to step through the student's responses to the questions.
Responses: Provides a spreadsheet-like representation of all responses.
Statistics: Provides a various statistical calculations for the iCMA as a whole and for individual questions within it. These are described elsewhere on this site.
STACK response analysis: Provides a detailed breakdown on responses to STACK questions.
Extract responses: Provides student responses in a file which may be submitted to the university's anti-plagiarism systems.
Manual grading by question/student: Enables human markers to mark essay questions.
Review questions: Provides authors with a printable view of the question editing forms for all the questions in an iCMA.
Parameter check: Helps authors ensure that their iCMA is correctly configured.
Edit quiz settings: Enables E&A administrators to amend the opening and closing dates without having full iCMA authoring permissions.
The Grades, Responses and Statistics options of the iCMA reports provide details of students' performance on each question together with statistics that enable you to quickly identify problem questions. Problem questions fall into two categories:
Empirical evidence shows that when you have studied the responses for ~100 students you are likely to have covered the majority of possible responses. For free text responses we recommend studying the responses of 200 students or more.
Nb. For assessed iCMAs there are strict controls over when questions can be amended. For formative iCMAs we leave this to module team discretion.
Because the iCMA reports contains personal data we restrict access as follows:
|OU role||VLE 'role'||Access|
|Student||Student||Students can review their own iCMAs.|
|Associate lecturer||Tutor||ALs can review the iCMAs of all students in their tutor group.|
|Module Team||Website updater||Module team members with the Website updater role can review the iCMAs of all students on the module and access the various analyses.|
No other VLE roles provide access to the iCMA reports. If other staff wish to follow results, perhaps to check on how a set of questions is performing, they should obtain the permission of the Module Team chair to be given the Website updater role.
If the assessment is assessed and counts towards the student's module mark an email is sent when the assessment is submitted confirming the receipt. (This is configured when the iCMA is created and the workflow attached.) This is unnecessary for formative assessments.
For assessed assessments:
To prevent Glossary words being turned into links in a question it is possible to turn off the Glossary filter for the iCMA.
Choose ‘Filters’ and turn off the Glossary.