How to create an iCMA in Moodle

Introduction

1 Create an empty iCMA

N.b. To create an assessed iCMA in Moodle please ask the Curriculum Manager to initiate the process using the form provided in Administering an assessed iCMA on this site.

You will need permission to edit the module in which you wish to create the iCMA. Such permission is conferred through the Website updater 'role' and through a role associated with the work flow. These roles are set up when the module is created. If you do not have the correct roles please contact your LTS Online Services team.

Once you have the roles you will find that you are allowed to turn editing on (and off). Turning editing on/off is available in the site Administration block.

1.1 Turn editing on link in Administration block

Turn editing on by clicking on the link. Your module site will refresh and display the drop-down list boxes that allow you to add new resources and activities to the module. They will look like this

1.2

To add an iCMA open the 'Add an activity...' drop-down list and select iCMA from the list of displayed options.

1.1 Understand what is available

It is useful to start by completing the iCMA definition form as this sets the context for the questions that you will write.

You will be faced with two major choices.

You will need to decide on the 'Question behaviour' which can be

• deferred feedback - where answers to all questions are submitted simultaneously.
• interactive with multiple tries - where questions are answered one by one.

The differences between the two behaviours are described and exemplified elsewhere on this website. Choosing the behaviour at the outset ensures that you will be clear about how to write your questions together with the feedback and hints that you will have to include.

You will also need to decide if your iCMA is

• formative - assessment for learning.
• assessed - assessment for credit.

If it is the latter this will already have been agreed as part of the module's formal assessment strategy.

The results from all iCMAs can be displayed in the Moodle Gradebook and those for assessed iCMAs are dispatched to the Student Records system from the Moodle Gradebook. Module Teams using iCMAs will want to familiarise themselves with the Gradebook which is also described on this website.

1.2 Quick setup

Assessed iCMAs must have certain settings that have been agreed with Exams and Assessments. The workflow described in Administering an assessed iCMA guides authors in choosing the correct settings.

For formative iCMAs the default settings enable you to create an iCMA by completing just one or two settings.

1. You must give your iCMA a name. Do this in General | Name.
2. You must choose a question behaviour. Do this in Question behaviour | How questions behave.

That's all. Now press Save and you have your empty iCMA ready for you to add questions.

1.3 General

The General section of the iCMA form looks as follows:

1.3 The general section of the iCMA definition form

Name: Supply a short name for the assessment e.g. iCMA41; Block 2 Questions. It helps to make the name short as it is used in column headings in the Gradebook and the shorter the name the smaller the on-screen space that is occupied in these reports.

• There is a convention that iCMAs that have their results transferred to Student Records should be numbered in the range 41 - 60, so iCMA41, iCMA42 etc. Within this range 'formative but compulsory' iCMAs should be numbered between 51 and 59.
• iEMAs (End of Module Assessments) are transferred as CMEs (Computer Marked Exams) and are named iCME with a number in the range 81 - onwards e.g. iCME81, iCME82 etc.
• formative iCMAs may be named in any way that fits in with your module and we have seen Quizzes and SOFAs and more. If you wish to use iCMA please choose a number in the range 1 - 40, so iCMA1, iCMA2 etc.

Introduction: Your introduction is shown to students before they start the iCMA. You may use it to set the scene, indicate the purpose of the iCMA within the module, say how long it should take, whether it can be repeated etc.

The introduction is available to students before the iCMA is open.

It is possible to revisit this iCMA definition form any number of times and consequently it is not necessary to write the final version of your introduction at your first visit, but knowing the reasons for creating the assessment and its role in the module will help you to come to a view on the number of questions that the assessment should contain and hence the student and author time involved.

1.4 Timing

The Timing section of the iCMA form looks as follows:

1.4 The timing section of the iCMA definition form

Open to students: To make the assessment available from a specified date and time, check the 'Enable' box and set the start date and time. It is standard practice to open assessed iCMAs at 00.00. The assessment will be made available automatically once this time is reached.

To make the assessment available as soon as it is visible (see below) leave the 'Enable' box unchecked.

Close to students: To make the assessment close at a specified date and time, check the 'Enable' box and set the close date and time. It is standard practice to close assessed iCMAs at 23.59 hours (essentially midnight). The assessment will close automatically once this time is reached.

To leave the assessment open for the duration of the module leave the 'Enable' box unchecked.

When time expires : One of

• Open attempts are submitted automatically
• There is a grace period when open attempts may be submitted, but no more questions answered
• Attempts must be submitted before time expires, or they are not counted

From 13B presentations formal iCMAs will allow a grace period. Students who have started an iCMA but not submitted before the closing time will receive an email reminding them to submit if they wish to have their responses marked. This setting will be actived by the iCMA workflow.

The normal setting is 'Attempts must be submitted before time expires...'.

Submission grace period: Only used for a formal iCMA and normally 7 days.

1.5 Grade

The Grade section of the iCMA definition form

1.5 The timing section of the iCMA definition form

Grade category: The category in the Gradebook under which the iCMA is listed.

Attempts allowed: For a formative assessment it is likely that you will wish students to attempt the assessment as often as they feel necessary both immediately after studying the associated text or as a revision exercise prior to examinations. Consequently for formative assessments 'Unlimited attempts' is a sensible setting.

For an assessed assessment the correct setting is '1'.

1.6 Layout

1.6 The layout section of the iCMA definition form

New page: We recommend a setting of 'Every question'. If you have an image that you wish to ask several questions about and you want those questions to appear on one page you will find that it is straightforward to remove page breaks when you come to add questions to the assessment on the Edit page.

1.7 Question behaviour

1.7 The question behaviour section of the iCMA definition form

How questions behave: The choices are:

• deferred feedback
• interactive with multiple tries
• deferred feedback with CBM (Certainty Based Marking)
• immediate feedback (like interactive but without the multiple tries)
• immediate feedback with CBM

Note that Essay questions are not marked by any of the above and have to be manually graded.

Certainty based marking ( http://docs.moodle.org/ 26/ en/ Using_certainty-based_marking ) asks students to reflect on their knowledge and indicate how certain they are about their answer. This certainty level is then used in assigning marks. A high degree of certainty in an incorrect answer results in a substantial penalty. Low certainty gives a lower reward but no risk of negative marking. This encourages reflection and rewards acknowledgement of uncertainty, while also retaining conventional information about student accuracy.

Where questions are assigned weights different from the default (1.0), marks are multiplied by the weight. Partially correct answers are given the appropriate fraction of the mark for a correct answer.

1.8 Review options

This section controls what is displayed during and after an assessment has been submitted. The matrix offers a wide variety of settings that control the visibility of certain question elements.

1.8 The review options section of the iCMA definition form

During the attempt: settings for use with interactive with multiple tries and immediate feedback question behaviours

Immediately after the attempt: i.e. as soon as the assessment is submitted.

Later, while the iCMA is still open: At the OU we expect this column to be the same as the 'Immediately after the attempt' column.

After the iCMA is closed: i.e. after the closing time and date specified on this form.

The attempt: Controls whether an attempt and student responses can be reviewed. At the OU we would expect these boxes to be checked.

Whether correct: Controls the use of colours behind the navigation panel and student's response and the appearance of Correct/Partially correct/Incorrect in the status panel that is found between the navigation panel and the question. The colours are accompanied by hidden textual information for screen readers.

Marks: Controls the display of marks alongside each question. This setting also controls the display of the overall grade on the iCMA.

Specific Feedback: This controls the display of the feedback that is supplied to specific responses on the question editing form.

General feedback: This should be shown to all students irrespective of the correctness of their answer. It should contain the correct answer to a question and an explanation of why that answer is correct.

Right answer: This will usually not be checked. If checked Moodle automatically adds information on what is and is not the correct answer to each question, however how the system does this is not satisfactory for use at the OU. We strongly recommend that authors use the General feedback box to inform students what the correct answer is and why.

Overall feedback: This can be matched to the student’s overall grade on the iCMA and is completed at the bottom of the iCMA definition form. Please see the below.

1.9 Display

All of the settings in this section are Advanced settings which should not need changing for OU use.

1.9 The display section of the iCMA definition form

1.10 Extra restrictions on attempts

1.10 The extra restrictions on attempts section of the iCMA definition form

Students cognisance of plagiarism policy: The choices are

• not required
• must be acknowledged before starting an attempt

The latter setting results in the following being displayed when an iCMA is first started.

Questions are withheld until the student gives the appropriate confirmation.

This confirmation is only required on the first visit to an iCMA.

All iCMAs which are a formal part of the module assessment policy should be configured to require students to acknowledge that they must not share answers.

1.11 Overall feedback

1.11 The Overall feedback section of the iCMA definition form

When the assessment is submitted an overall grade will be calculated that is based on the students individual marks on each question (this is defined at the next stage of building the iCMA). On The Review options section above if the 'Marks' box is checked the student will be presented with their overall grade and if the 'Overall feedback' box is checked whatever comment you add into the appropriate feedback box above will also be displayed.

Be careful to enter consistent scores in all the Grade boundary boxes; it is perhaps advisable to stick to percentages - be sure to include the % symbol.

1.12 Common module settings

1.12 The common module section of the iCMA definition form

Visible: Set this to 'Hide' while you are building the assessment and change this setting to 'Show' when the assessment is ready for use.

ID number: This is essential for assessed iCMAs and is used for identifying results to the Student Records system. The format of the ID number is module-presentation.assignment e.g. m120-07j.icma41. Upper or lower case can be used as the interface program in the Student Records system converts all module codes, presentation codes and PIs to upper case as it reads them in.

Group mode: to ensure that tutors can see only the results for their tutor group this setting should be set to 'Separate groups'. If your module does not have tutor groups choose 'No groups'.

1.13 Restrict access

1.13 The restrict access section of the iCMA definition form

Allow access from: This controls the appearance of the link to the iCMA on the page. To make the link appear from a specified date and time, check the 'Enable' box and set the start date. The link will appear automatically once this date is reached.

To make the link appear as soon as it is visible (see above) leave the 'Enable' box unchecked.

Allow access until: To remove the link from the page after a specified date check the 'Enable' box and set the until date. After this date and time has passed the link will be removed automatically.

To leave the link in place for the duration of the module leave the 'Enable' box unchecked.

Grade condition: Use this setting to tie the availability of this iCMA to the attainment of some other grade, or grades, held within the Gradebook.

User field: Restrict access based on a field from the users profile.

Activity completion condition: This setting determines any activity completion conditions which must be met in order to access the activity. Note that completion tracking must first be set before an activity completion condition can be set.

Multiple activity completion conditions may be set if desired. If so, access to the activity will only be permitted when ALL activity completion conditions are met.

While access is prevented: Two settings are available, greyed-out or hidden.

1.14 Activity completion

1.14 The activity completion section of the iCMA definition form

Completion checking: If enabled, activity completion is tracked, either manually or automatically, based on certain conditions.

Require view: Self explanatory.

Require grade: If enabled, the activity is considered complete when a student receives a grade. Pass and fail icons may be displayed if a pass grade for the activity has been set.

Expect completed on: This setting specifies the date when the activity is expected to be completed. The date is not shown to students and is only displayed in the activity completion report.

1.15 Save and display

Please save the iCMA settings using ‘Save and display’.

Your iCMA is now ready for questions to be added. If your questions have already been created and reviewed then please carry on with the next section Build your iCMA. If you have still to write your questions please refer to the separate document How to write questions in Moodle.

2 Create your questions

This is described in a separate document, How to create questions in Moodle which you will find on this website.

3 Build your iCMA

At Step 1 of these instructions you created an iCMA. At that time you set the iCMA parameters but did not add any questions. Now is the time to add questions.

To do this return to the module page or sub-page containing the iCMA and click on the name of your iCMA. Then click on 'Edit quiz' in the Quiz administration block.

3.1 The Quiz administration block

3.2 An iCMA with questions added

The screen that appears has three main sections with the list of questions in the iCMA down the centre and the Question bank to the right. To build a quiz you need to be familiar with

Category: This drop-down list box enables you to choose which of your categories of questions are displayed.

The '

The button which allows multiple, selected, questions to be added at once. And the

And the control which instructs Moodle to choose the question, at run time, from within the specified category. Should you choose to add multiple random questions, Moodle will ensure that there is no duplication in its choices.

Start with the  '

Further questions can be added in a similar manner. To save time rearranging the order of the questions later it is best to start by adding the questions in the order that you wish them to be presented. Indeed it is better still if you have already named your questions so that you can re-order them easily in the question bank before adding them. You may reorder the questions in the question bank by clicking on the headings that are found above the questions.

Within the iCMA list, questions may be reordered using the up and down arrows or by using the tools on the ’Order and paging’ tab.

Continue adding the questions until your iCMA is complete. When your iCMA is complete set the Maximum grade to be the same as the Total. (Unfortunately it is not straightforward to do this automatically and you must change the Maximum grade manually.) The iCMA reports then report scores out of the maximum e.g. 10/10. In the Gradebook which gathers grades from multiple iCMAs all grades are shown as percentages.

You are now ready to preview your iCMA to see how it works in practice. At this point you are still able to amend your iCMA.

3.1 Varying questions between students

Moodle provides various mechanisms to enable you to vary the questions which are given to students who are doing the same iCMA.

1. Questions can be randomly chosen, by Moodle at run time, from a bank of questions. This applies to all question types.
2. Options in selection questions, such as multiple choice, multiple response, drag and drop and matching can be randomly ordered by Moodle at run time.
3. Variable numeric , Variable numeric sets and STACK questions can also be varied internally.
4. OpenMark questions can be configured to vary within the question.

Moodle keeps a record of which student has received which question and the reports provided also contain this information.

4 Preview the iCMA

Having added all your questions select the 'Preview' link from the Settings on the left of the window

4.1 The preview link is in the Quiz administration block

and you will be able to interact with your iCMA usingthe settings specified on the iCMA definition form.

As you are in Preview mode the VLE will not keep long term records of your responses but it does keep sufficient records to enable you to exit and return to where you were at a later date. It is also possible to restart the preview multiple times to enable you to test different responses.

4.1 Reviewing the questions on paper

Experienced authors may choose to check their iCMAs by careful reading of the question editing forms. It is possible to review, and print, the editing forms for all questions in an iCMA at one go. Where questions are chosen randomly from a category all available questions are included.

4.2 This link will show the contents of the editing forms of all the questions used in the iCMA

4.2 Re-structuring while previewing the iCMA

If you alter the numbers of questions or change the types of questions at this stage of the build process please ‘Save’ between previews. Here is the reason why:

1. When you preview an iCMA the system keeps track of your position.
2. If you stop, exit and then restart the preview you will find that you can carry on from where you had previously reached.
3. However as soon as you change the structure of the iCMA the information associated with your previous preview becomes obsolete and must be deleted.
4. This is done when you ‘Update the iCMA’ and ‘Save’.

If you do not follow the above instructions you are likely to see errors when you start a preview.

4.3 Ask a colleague to check the iCMA

Your iCMA should be thoroughly checked by a colleague before it is made live. As your iCMA will still be hidden your colleague will require the 'Website updater' and Workflow editing' roles and they should test the iCMA using Preview mode.

5 Make the iCMA visible

This can be done either at the module level by clicking on the ‘eye’ or on the iCMA definition form, in the Common Module Settings, by setting ‘Visible’ to ‘Show’.

The iCMA is now ready to be used by students and will operate according to the settings that you have made on the iCMA definition form.

5.1 Desist from editing questions

5.2 How to run an iCMA

Each iCMA comprises a number of separate questions. You should attempt to answer all questions.

The navigation panel on the left-hand side shows questions that you have answered and those that are currently unanswered. How it does this depends on the type of iCMA.

• In deferred feedback mode where all answers are submitted together the panel records which questions have been answered.

5.1 Navigation panel in deferred feedback mode

Please note that if you answer a question but your answer is incomplete* the question background will remain white. * Please revisit the question to see why your response is incomplete.

• In instant feedback mode, questions are answered one by one using the ‘Check’ button. The panel shows which questions have been completed.

5.2 Navigation panel in instant feedback mode

You may move directly to any question by clicking on its number in the navigation panel. To move to the next question you may also click on the ‘Next’ button, which appears below the current question.

You may answer a few questions, close your browser or turn off your computer, and return later. However, the last answer you enter will only be recorded if you move to another page within the iCMA or check your answer. We recommend that you always visit the Summary to gauge your progress before exiting. You can reach the Summary page by clicking on ‘Finish attempt…’.

• In deferred feedback mode you may revisit questions and amend your answers before submitting them for marking.
• In instant feedback mode your answer is marked when you click ‘Check’. If the question allows multiple tries you will be able to ‘Try again’ if your first answer is incorrect.

When you have answered all questions you should click on ‘Finish attempt...’, which will lead you to the Summary page.

• In deferred feedback mode the Summary page shows which questions have been answered and which have not.

  All questions should be answered. You may return to any question by clicking on its number.

  Once you have answered all questions you should 'Submit all and finish'. This will complete your attempt on the iCMA and your answers will be marked.

5.3 Summary panel in deferred feedback mode

Here question 3 is ‘flagged’ (see below). Please note that if you answer a question but your answer is incomplete the Summary will report this as shown with question 7. Please revisit the question to see why your response is incomplete.

• In instant feedback mode the Summary page shows which questions have been completed and which have not.

  All questions should be completed. You may return to any question by clicking on its number.

  Once you have completed all questions you should ‘Submit all and finish’. This will finish your attempt on the iCMA and you will receive feedback on your overall performance on the assessment (if the iCMA author has provided this).

Wherever you are presented with an answer box, please click in the box so that the cursor appears and then enter your answer.

5.4 Answer box

The flag icon is provided to enable you to ‘flag’ questions that you wish to return to. Clicking on the flag turns it red and places a similar flag alongside the question number on the Summary screen.

• Deferred feedback iCMAs are marked when you press 'Submit all and finish'. For formative iCMAs you will be able to review your overall performance, your mark on each question and the correct answer to each question immediately. For summative iCMAs, marks and feedback will be available after the iCMA closing date.
• In instant feedback iCMAs, feedback is given instantly to each question as responses are checked. Marks may be included for formative iCMAs. Overall marks and feedback are provided when you press ‘Submit all and finish’. For a formative iCMA the overall marks and feedback are available immediately; for summative iCMAs, overall marks will be available after the iCMA closing date.

To review your answers you will be provided with the navigation panel again. You may click on a question number to review your answer. If your answer has been marked it will be shown when you review the question. Also when the questions have been marked the question numbers on the panel will be given coloured bars; green for correct; amber for partially correct and red for incorrect.

5.5 Review panel

When you click 'Submit all and finish' you complete an attempt at the iCMA. Your responses, marks and feedback for all attempts are stored and you may review them later by revisiting the iCMA. When you revisit the iCMA you will be offered the chance to start it again but you will also see a list of previous attempts.

5.6 A list of previous attempts

Following the ‘Review’ link for the appropriate attempt, enables you to review that attempt.

Formative iCMAs are provided to help you to improve your knowledge and you are encouraged to repeat them as often as you wish. Summative iCMAs are assessed and you may only make a single attempt.

6 Monitor the iCMA reports

As soon as your students start to use your iCMA things get more interesting. Moodle provides various reports which you can use to monitor your students' performance and how your iCMA and individual questions perform. To access the reports open an iCMA and when it appears select the results link.

6.1 Access the quiz reports here

6.2 Expanded Results section

The Results section expands to show a variety of reports. The full list is described here but what is shown to whom depends on their role on the module.

Grades: Lists students who have interacted with the iCMA together with dates and grades achieved. Students who have made multiple attempts have multiple entries beneath their name. By clicking on a date or a grade it is possible to step through the student's responses to the questions.

Responses: Provides a spreadsheet-like representation of all responses.

Statistics: Provides a various statistical calculations for the iCMA as a whole and for individual questions within it. These are described elsewhere on this site.

STACK response analysis: Provides a detailed breakdown on responses to STACK questions.

Extract responses: Provides student responses in a file which may be submitted to the university's anti-plagiarism systems.

Manual grading by question/student: Enables human markers to mark essay questions.

Review questions: Provides authors with a printable view of the question editing forms for all the questions in an iCMA.

Parameter check: Helps authors ensure that their iCMA is correctly configured.

Edit quiz settings: Enables E&A administrators to amend the opening and closing dates without having full iCMA authoring permissions.

The Grades, Responses and Statistics options of the iCMA reports provide details of students' performance on each question together with statistics that enable you to quickly identify problem questions. Problem questions fall into two categories:

• questions that have identified student learning difficulties. Incorrect responses which are repeated by multiple students probably need specific feedback to correct widespread misunderstandings.
• questions that are not coping adequately with student responses. You will need to ensure that your response matching has coped with all the responses that your students have given. Clearly you will wish to ensure that all correct responses have been marked accordingly.

Empirical evidence shows that when you have studied the responses for ~100 students you are likely to have covered the majority of possible responses. For free text responses we recommend studying the responses of 200 students or more.

Nb. For assessed iCMAs there are strict controls over when questions can be amended. For formative iCMAs we leave this to module team discretion.

Because the iCMA reports contains personal data we restrict access as follows:

No other VLE roles provide access to the iCMA reports. If other staff wish to follow results, perhaps to check on how a set of questions is performing, they should obtain the permission of the Module Team chair to be given the Website updater role.

7 When the assessment is completed and closed

8 Tips and Tricks

8.1 How to hide glossary entries in questions

To prevent Glossary words being turned into links in a question it is possible to turn off the Glossary filter for the iCMA.

8.1 The iCMA settings

Choose ‘Filters’ and turn off the Glossary.