How to create questions in Moodle

Introduction

1 The Question bank

The Question bank can be accessed from the site Administration block.

1.1 The link to the question bank

There are four question bank actions for creating and editing questions, creating categories in which to store the questions, import (from a Moodle XML format file) and export (to a Moodle XML format file).

Understanding these tools will enable you to store and retrieve your questions more efficiently.

Questions: Where the questions are held in the VLE. Your questions will normally be held within your module where they will be visible to all your colleagues who are 'Website updaters' on this module. When the module is rolled forwards for the next year the questions will be included. It is also possible to share questions across modules by storing the questions in categories at the Faculty or School level - please read on.

Categories: By creating categories for different topics, books or blocks of the module you are creating a logical storage structure so that you and all your colleagues will be able to locate all the questions on that topic in one area. It is also possible to create categories that span Faculties or Schools.

Import: It is possible to import files of questions in Moodle XML format.

Export: It is possible to export questions for import into another module. Please ensure that you use ‘Moodle XML format’.

1.1 Categories

Click on 'Categories' and you will be taken to the following screen where you can create your own categories.

1.2 Accessing categories from the site Administration block

Possible structures can be based on Blocks, Books, Chapters or Units. We anticipate that most questions will be used within individual modules and consequently we anticipate that most question categories will be created within the module. In the example above there are three main categories of questions for iCMA01, iCMA02 and iCMA22.

Please note that it is also possible to reach the question bank from the Quiz administration block and if you do this you will also find a category for this particular quiz.

1.3 Accessing categories from the Quiz administration block

Please be aware that entering the question bank from the site Administration block (Figure 1.2) does not show the categories associated with individual quizzes. Please compare and contrast Figures 1.2 and 1.3.

1.2 Import and export

2 Question types

Having created your categories click on the Questions link in the administration block to show:

2.1 An empty question bank.

In the Question bank select the category that you wish to store your question in and then choose one of the question types from 'Create a new question…'. This will bring up the ‘Choose a question type to add’ dialogue box.

2.2 The question types available in December 2013

These Moodle question types can be categorised by the type of interaction that they support.

1. Numeric response
2. Text response
3. Selection
4. 2D - i.e some form of spatial input
5. Mathematical response
6. External

There is also a seventh category, Description, which enables you to include blocks of information within an assessment.

2.1 Numeric response

There are a variety of question types that accept numeric responses.

2.1.1 Numerical

2.3 The Question text, Default mark and General feedback. The text surrounded by two $ symbols is LaTeX which will be interpreted by the maths filter and displayed in mathematical format.

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

You may also include formatted mathematics using TeX or MathML. The text surrounded by two $ symbols in this example will be interpreted by the TeX filter and converted to nicely formatted mathematics. Please see the Including TeX and MathML section below.

Default mark: Decide on how to score your questions and be consistent.

General feedback: We recommend that all questions should have this box completed with the correct answer and a fully worked explanation. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect. We do not recommend that authors rely on using the machine generated 'Right answer' (from the iCMA definition form) as this is:

• written by a computer
• frequently not the full right answer
• not necessarily in the correct format.

2.4 The answers section

Answer: an integer or decimal number. Use 'E' format for very large or very small values. '*' matches all other answers.

Error: Only rely on exact values for integers, for all other values specify a suitable range.

Grade: The percentage of the mark to be allocated for this answer. The correct answer must be graded at 100%.

Feedback: Feedback that is specific to the answer.

Responses are matched in the order that they are entered into the form.

2.5 Including units

The Numerical question type does not allow students to input powers and the Unit handling and Units sections provide ways around this. An alternative approach is provided in the Variable numeric with units question type.

Unit handling: a choice of three options:

• Units are not used at all. Only the numerical value is graded.
• Units are optional. If a unit is entered, it is used to convert the response to Unit 1 before grading
• The unit must be given and will be graded.

Unit penalty:

Units are input using: A choice of three options:

• the text input element
• a multiple choice selection
• a drop down menu

Units go: left or right.

2.6 Specifying units

The multiplier is the factor by which the correct numerical response will be multiplied.

The first unit (Unit 1) has a default multiplier of 1. Thus if the correct numerical response is 5500 and you set W as unit at Unit 1 which has 1 as default multiplier, the correct response is 5500 W.

If you add the unit kW with a multiplier of 0.001, this will add a correct response of 5.5 kW. This means that the answers 5500W or 5.5kW would be marked correct.

Note that the accepted error is also multiplied, so an allowed error of 100W would become an error of 0.1kW.

2.7 Hints for use in Interactive with multiple tries behaviour

If you are configuring your question to run in 'Interactive with multiple tries' mode such that students can answer questions one by one you can also provide hints and second (and third) tries by entering appropriate feedback into the boxes provided in the 'Multiple tries' section of the question definition form.

To use this section of the question form please ensure that on the iCMA definition form you have chosen 'How questions behave' = 'Interactive with multiple tries'.

Penalty for each incorrect try: The available mark is reduced by the penalty for second and subsequent tries. In the example above a correct answer at the second try will score 0.6666667 of the available marks and a correct answer at the third try will score 0.3333334 of the available marks.

Hint: You can complete as many of these boxes as you wish. If you wish to give the student three tries at a question you will need to provide two hints. At runtime when the hints are exhausted the question will finish and the student will be given the general feedback and the question score will be calculated.

N.b. If you create a hint and then decide to delete it such that there is one less hint to the question you must ensure that all HTML formatting is also removed. To do this click on the 'HTML' icon in the editor toolbar which will display the HTML source. If any HTML formatting (in '< >') is present delete it and click on the HTML icon again to return to normal view.

2.1.2 Variable numeric

Variable numeric questions use numeric values and calculations that are performed at runtime following rules laid down in the question. As such it is straightforward to produce numerous variations on a question. However this places an onus on the author to be sure that all possible variations will function properly and not inadvertently run into ‘division by zero’ or ‘square root of a minus number’ errors. Please see Variable numeric sets questions for a ‘safer’ alternative.

2.8 The question, marks and general feedback for a Variable numeric question

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

You may also include formatted mathematics using TeX or MathML. The text surrounded by two $ symbols in this example will be interpreted by the TeX filter and converted to nicely formatted mathematics. Please see the Including TeX and MathML section below.

General feedback: We recommend that all questions should have this box completed with the correct answer and a fully worked explanation. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect. We do not recommend that authors rely on using the machine generated 'Right answer' (from the iCMA definition form).

String to act as seed for randomisation: Sometimes it is desirable to base a sequence of questions on the same set of numbers. To do this use the same string e.g. ‘mystring’ as the ‘seed for randomisation’ in each question in the sequence.

Please note that this then generates the same set of numbers across a sequence of questions for a student but different students get different sets of numbers.

2.9 The Variables section

The formula for all calculations should start with a variable name followed by an equals (=) sign and may use common mathematical operators +, -, *, /, brackets, (, ) and other variables defined within the question.

The function rand_int(n,m)provides a random integer within the range n to m inclusive. rand_int(n,m)uses the seed described above.

The mathematical processing that is available is similar to that provided within the Moodle Gradebook and includes

• average(a, b…): Returns the average of a list of arguments.
• max(a, b...): Returns the maximum value in a list of arguments
• min(a,b...): Returns the minimum value in a list of arguments
• mod(dividend, divisor): Calculates the remainder of a division.
• ln(number): Returns the natural logarithm of number.
• log(number): Returns the natural logarithm of number.
• pi(): Returns the value of the number Pi.
• power(base,exponent): Calculates the value of the first argument raised to the power of the second argument.
• round(number, count): Rounds a number to a predefined accuracy.
• sqrt, abs, exp
• sin, sinh, arcsin, asin, arcsinh, asinh
• cos, cosh, arcos, acos, arccosh, acosh
• tan, tanh, arctan, atan, arctanh, atanh

Recalculate now: With all variables specified the ‘Recalculate now’ button will calculate a set of values within the greyed out boxes.

2.10 The answers section

Require scientific notation:It is possible to require the student to enter their response using scientific notation. The input box is provided with a superscript facility and all of the following formats are accepted: 1.234e5, 1.234x10⁵, 1.234X10⁵ and 1.234*10⁵.

For keyboard users the superscript may be accessed with the up-arrow key. Down-arrow returns entry to normal. You may wish to include this information within your question. If you do please note that the up-arrow and down-arrow provided in the HTML editor’s ‘insert custom characters’ list are not spoken by a screen reader and you should also include the words ‘up-arrow’ and ‘down-arrow’.

Answer: This may be a numeric value, a variable as above or a mathematical expression.

Accepted error: Computers store numbers to a finite accuracy e.g. 1/3 is stored as a finite number of recurring 3s in 0.33333… When used in calculations these minor infelicities can soon propagate and as such we strongly recommend that answers are matched within a suitable numerical range as shown here.

Significant figures: It is possible to request that responses be given to a specified number of significant figures.

In all of these four cases specific feedback can be supplied using these settings.

2.11 Hints for use in Interactive with multiple tries behaviour

If you are configuring your question to run in 'Interactive with multiple tries' mode such that students can answer questions one by one you can also provide hints and second (and third) tries by entering appropriate feedback into the boxes provided in the 'Multiple tries' section of the question definition form.

To use this section of the question form please ensure that on the iCMA definition form you have chosen 'How questions behave' = 'Interactive with multiple tries'.

Penalty for each incorrect try: The available mark is reduced by the penalty for second and subsequent tries. In the example above a correct answer at the second try will score 0.6666667 of the available marks and a correct answer at the third try will score 0.3333334 of the available marks.

Hint: You can complete as many of these boxes as you wish. If you wish to give the student three tries at a question you will need to provide two hints. At runtime when the hints are exhausted the question will finish and the student will be given the general feedback and the question score will be calculated.

N.b. If you create a hint and then decide to delete it such that there is one less hint to the question you must ensure that all HTML formatting is also removed. To do this click on the 'HTML' icon in the editor toolbar which will display the HTML source. If any HTML formatting (in '< >') is present delete it and click on the HTML icon again to return to normal view.

2.1.3 Variable numeric sets

Variable numeric sets questions are similar to Variable numeric questions but are limited to ‘sets’ of pre-specified values. As such Variable numeric sets questions are safer as all possible variations can be tested before the question is released to students.

Variable numeric sets questions use numeric values and calculations that are performed at runtime following rules laid down in the question.

2.12 The question, marks and general feedback for a variable numeric sets question

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

You may also include formatted mathematics using TeX or MathML. The text surrounded by two $ symbols in this example will be interpreted by the TeX filter and converted to nicely formatted mathematics. Please see the Including TeX and MathML section below.

General feedback: We recommend that all questions should have this box completed with the correct answer and a fully worked explanation. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect. We do not recommend that authors rely on using the machine generated 'Right answer' (from the iCMA definition form).

String to act as seed for randomisation: Sometimes it is desirable to base a sequence of questions on the same set of numbers. To do this use the same sets of numbers and the same string e.g. ‘mystring’ as the ‘seed for randomisation’ in each question in the sequence.

Please note that this then selects the same set of numbers across a sequence of questions for a student but different students get different sets of numbers.

2.13 Two pre-specified sets of values and four calculated sets

The formula for all calculations should start with a variable name followed by an equals (=) sign and may use common mathematical operators +, -, *, /, brackets, (, ) and other variables defined within the question.

The mathematical processing that is available is similar to that provided within the Moodle Gradebook and includes

• average(a, b…): Returns the average of a list of arguments.
• max(a, b...): Returns the maximum value in a list of arguments
• min(a,b...): Returns the minimum value in a list of arguments
• mod(dividend, divisor): Calculates the remainder of a division.
• ln(number): Returns the natural logarithm of number.
• log(number): Returns the natural logarithm of number.
• pi(): Returns the value of the number Pi.
• power(base,exponent): Calculates the value of the first argument raised to the power of the second argument.
• round(number, count): Rounds a number to a predefined accuracy.
• sqrt, abs, exp
• sin, sinh, arcsin, asin, arcsinh, asinh
• cos, cosh, arcos, acos, arccosh, acosh
• tan, tanh, arctan, atan, arctanh, atanh

Recalculate now: With all variables specified the ‘recalculate now’ button will calculate a set of values within the greyed out boxes.

2.14 The answers section

Require scientific notation: It is possible to require the student to enter their response using scientific notation. The input box is provided with a superscript facility and all of the following formats are accepted: 1.234e5, 1.234x10⁵, 1.234X10⁵ and 1.234*10⁵.

For keyboard users the superscript may be accessed with the up-arrow key. Down-arrow returns entry to normal. You may wish to include this information within your question. If you do please note that the up-arrow and down-arrow provided in the HTML editor’s ‘insert custom characters’ list are not spoken by a screen reader and you should also include the words ‘up-arrow’ and ‘down-arrow’.

Answer: This may be a numeric value, a variable as above or a mathematical expression. For the example shown here it would have been quite acceptable to place ‘a/(-b)’ in the Answer box.

Accepted error: Computers store numbers to a finite accuracy e.g. 1/3 is stored as a finite number of recurring 3s in 0.33333… When used in calculations these minor infelicities can soon propagate and as such we strongly recommend that answers are matched within a suitable numerical range as shown here.

Significant figures: It is possible to request that responses be given to a specified number of significant figures.

In all of these four cases specific feedback can be supplied using these settings.

2.15 Hints for use in Interactive with multiple tries behaviour

If you are configuring your question to run in 'Interactive with multiple tries' mode such that students can answer questions one by one you can also provide hints and second (and third) tries by entering appropriate feedback into the boxes provided in the 'Multiple tries' section of the question definition form.

To use this section of the question form please ensure that on the iCMA definition form you have chosen 'How questions behave' = 'Interactive with multiple tries'.

Penalty for each incorrect try: The available mark is reduced by the penalty for second and subsequent tries. In the example above a correct answer at the second try will score 0.6666667 of the available marks and a correct answer at the third try will score 0.3333334 of the available marks.

Hint: You can complete as many of these boxes as you wish. If you wish to give the student three tries at a question you will need to provide two hints. At runtime when the hints are exhausted the question will finish and the student will be given the general feedback and the question score will be calculated.

N.b. If you create a hint and then decide to delete it such that there is one less hint to the question you must ensure that all HTML formatting is also removed. To do this click on the 'HTML' icon in the editor toolbar which will display the HTML source. If any HTML formatting (in '< >') is present delete it and click on the HTML icon again to return to normal view.

2.1.4 Variable numeric sets with units

This is an extension of Variable numeric sets by the addition of a Pattern match response matching field to match the units.

The description below deals only with the differences from Variable numeric sets questions.

General

2.16 Relative weightings

The grade for the numeric value and the unit can have different weights.

Units

2.17 The field for matching the units

The question type splits the numeric value from the units before matching the units using the Pattern match algorithm.

The full capabilities for the Pattern match question is described in section 2.4.1.

2.1.5 Combined

The Combined question enables multiple numeric responses to be matched.

The full capabilities of the Combined question are described in section 2.4.1.

2.18 A combined question with two numeric input fields

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

General feedback: We recommend that all questions should have this box completed with the correct answer and a fully worked explanation. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect. There is no system generated 'Right answer' (from the iCMA definition form) for Combined questions.

2.19 The two numeric response matching fields

Thie Combined question uses a restricted form of the Variable numeric response matching.

Weighting. Different responses can have different percentages of the total mark. The weightings must add to 100%.

Answer: an integer or decimal number. Use 'E' format for very large or very small values.

Accepted error: Only rely on exact values for integers, for all other values specify a suitable range.

Scientific notation:It is possible to enable the student to enter their response using scientific notation. The input box is provided with a superscript facility and all of the following formats are accepted: 1.234e5, 1.234x10⁵, 1.234X10⁵ and 1.234*10⁵.

Feedback for any incorrect response: Where some fields are answered correctly and others incorrectly the intention is that the feedback associated with the field is used to say what is wrong. Correct answers can be counted up using the the option to 'Show the number of correct responses' in the Combined feedback and Hints fields.

2.20 Themultiple tries section

If you are configuring your question to run in 'Interactive with multiple tries' mode such that students can answer questions one by one you can also provide hints and second (and third) tries by entering appropriate feedback into the boxes provided in the 'Multiple tries' section of the question definition form.

To use this section of the question form please ensure that on the iCMA definition form you have chosen 'How questions behave' = 'Interactive with multiple tries'.

Penalty for each incorrect try: The available mark is reduced by the penalty for second and subsequent tries. In the example above a correct answer at the second try will score 0.6666667 of the available marks and a correct answer at the third try will score 0.3333334 of the available marks.

Hint: You can complete as many of these boxes as you wish. If you wish to give the student three tries at a question you will need to provide two hints. At runtime when the hints are exhausted the question will finish and the student will be given the general feedback and the question score will be calculated.

2.2 Text response

Four question types will accept typed textual responses.

2.2.1 Pattern match

The Pattern match question type is used to test if a short free-text student response matches a specified response pattern.

Pattern match is a more sophisticated alternative to the Short answer question type and offers:

• the ability to cater for misspellings, with and without an English dictionary
• specification of synonyms and alternative phrases
• flexible word order
• checks on the proximity of words.

For certain types of response it has been shown to provide an accuracy of marking that is on a par with, or better than, that provided by a cohort of human markers.

Pattern match works on the basis that you have a student response which you wish to match against any number of response matching patterns. Each pattern is compared in turn until a match is found and feedback and marks are assigned.

The key to using Pattern match is in asking questions that you have a reasonable hope of marking accurately. Hence writing the question stem is the most important part of writing these questions.

2.21 The Question text, Grade and General feedback for a Pattern match question.

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

Default mark: Decide on how to score your questions and be consistent.

General feedback: We recommend that all questions should have this box completed with the correct answer and a fully worked explanation. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect. For the Pattern match question type you cannot rely on using the machine generated 'Right answer' (from the iCMA definition form).

2.22 Options for entering answers

Case sensitivity: No or yes.

Allow use of subscript/superscript: No or Yes. Any subscripts entered by the student are contained in their response between the standard tags of _and or ^and . For example:

• The formula for water is H₂O
• The speed of light is approximately 3x10⁸ m s^-1

At run time keyboard users may move between normal, subscript and superscript by using the up-arrow and down-arrow keys. You may wish to include this information within your question. If you do please note that the up-arrow and down-arrow provided in the HTML editor’s ‘insert custom characters’ list are not spoken by a screen reader and you should also include the words ‘up-arrow’ and ‘down-arrow’.

If answer is more than 20 words: We strongly recommend that you limit responses to 20 words. Allowing unconstrained responses often results in responses that are both right and wrong – which are difficult to mark consistently one way or the other.

Check spelling of student: How many ways do you know of to spell ‘temperature’? We’ve seen 14! You will improve the marking accuracy by insisting on words that are in Moodle system dictionary.

Add these words to dictionary: When dealing with specialised scientific, technical and medical terms that are not in a standard dictionary it is most likely that you will have to add them by using this field. Enter your words leaving a space between them.

Convert the following characters to space: In Pattern match words are defined as sequences of characters between spaces. The exclamation mark and question mark are also taken to mark the end of a word. The period is a special case; as a full stop it is also a word delimiter but as the decimal point it is not. All other punctuation is considered to be part of the response but this option lets you remove it.

2.23 Fields for entering synonyms (not associated with the SI units question)

Words and synonyms: All words and synonyms are specified as they are to be applied by the response matching. They do not have to be full words but can be stems with a wildcard.

Synonyms may only be single words i.e. the ability to specify alternate phrases in synonym lists is not allowed.

From the example above any occurrence of the word oil in the response match will be replaced by oil|glycer*|paraf* before the match is carried out.

2.24 A response match

Answer: This example shows that Pattern match will support complex response matching.

Take the first answer field match_o(ampere candela kelvin mole) is the exact match for the four words with the additional feature that the matching option 'o' allows the words to be given in any order.

The answer field match_ow(ampere candela kelvin mole) requires the same four words, again in any order, but also allows other words.

The third answer field match_mow(ampere candela kelvin mole) allows for misspellings which are still in the dictionary e.g. mule instead of mole.

Please see the section on Pattern match syntax for a full description.

Grade: Between ‘none’ and 100%. At least one response must have a mark of 100%.

Feedback: Specific feedback that is provided to anyone whose response is matched by the response matching rule in Answer.

2.25 Feedback to all non-matched responses

The feedback for all non-matched responses should go into the ‘Any other answer’ field.

2.26 The multiple tries section

________________________________________________________________________

2.27

The basic unit of the student response that Pattern match operates on is the word, where a word is defined as a sequence of characters between spaces. The full stop (but not the decimal point), exclamation mark and question mark are also treated as ending a word.

Numbers are special instances of words and are matched by value and not by the form in which they are given. match_w(25 ms^-1)will match the following correct responses; 25 ms^-1, 2.5e1 ms^-1, 2.5x10^-1 ms^-1

With the exception of numbers and the word terminators (? and !) Pattern match matches what it is given. Whether case does or does not matter is left to the author to decide as is the significance of punctuation such as ,;: and others.

The response is treated as a whole with the exception that words that are required to be in proximity must also be in the same sentence.

2.28

Pattern match syntax

The match syntax can be considered in three parts.

1. the matching options e.g. mow
2. the words to be matched e.g.tom dick harry together with the in-word special characters.
3. and, or and not combinations of matches e.g. match_any()

2.29

2.1 The matching options

Matching option	Symbol	Description
allowExtraChars	c	Extra characters can be anywhere within the word. Authors are expected to omit allowExtraChars when using the misspelling options below.
allowAnyWordOrder	o	Where multiple words are to be matched they can be in any order.
allowExtraWords	w	Extra words beyond those being searched for are accepted.
misspelling: allowReplaceChar	mr	Will match a word where one character is different to those specified in the pattern. The pattern word must be 4 characters or greater, excluding wildcards, for replacement to kick in. Authors are expected to omit allowExtraChars when using this option.
misspelling: allowTransposeTwoChars	mt	Will match a word where two characters are transposed. The pattern word must be 4 characters or greater, excluding wildcards, for transposition to kick in. Authors are expected to omit allowExtraChars when using this option.
misspelling: allowExtraChar	mx	Will match a word where one character is extra to those specified in the pattern. The pattern word must be 3 characters or greater, excluding wildcards, for extra to kick in. Authors are expected to omit allowExtraChars when using this option.
misspelling: allowFewerChar	mf	Will match a word where one character is missing from those specified in the pattern. The pattern word must be 4 characters or greater, excluding wildcards, for fewer to kick in. Without this 'no' would be reduced to just matching 'n' or 'o'. Authors are expected to omit allowExtraChars when using this option.
misspelling	m	This combines the four ways of misspelling a word described above i.e. m is equivalent to mxfrt. Authors are expected to omit allowExtraChars when using this option.
misspellings	m2	Allows two misspellings, as defined by option 'm', in pattern words of 8 characters or more, excluding wildcards. Authors are expected to omit allowExtraChars when using this option. N.b. use this option sparely. It introduces a huge number of possible acceptable spellings all of which have to be checked and the time taken for the match to complete goes up and up.
allowProximityOf0	p0	No words, or full stops, are allowed in between any words specified in the proximity sequence.
allowProximityOf1	p1	One word is allowed in between any two words specified in the proximity sequence. The words must not span sentences.
allowProximityOf2	p2	(Default value) Two words are allowed in between any two words specified in the proximity sequence. The words must not span sentences.
allowProximityOf3	p3	Three words are allowed in between any two words specified in the proximity sequence. The words must not span sentences.
allowProximityOf4	p4	Four words are allowed in between any two words specified in the proximity sequence. The words must not span sentences.

The matching options are appended to the word match with an intervening underscore and may be combined. A typical match combines the options ‘mow’ to allow for

1. misspellings
2. any word order
3. any extra words

and is written match_mow(words to be matched).

________________________________________________________________________

2.30

Within a word 'special characters' provide more localised control of the patterns.

2.2 Special characters

Special character	Symbol	Description
Word AND	space	'space' delimits words and acts as the logical AND.
Word OR	|	| between words indicates that either word will be matched. | delimits words and acts as the logical OR.
Proximity control	_	Words must be in the order given and with no more than n (where n is 0 - 4) intervening words. All words under the proximity control must be in the same sentence. _ delimits words and also acts as logical 'AND'. Other words included in the match that are not under proximity control must be outside the words under the proximity control in the response for them to be matched.For example: 'match(abcd_ffff ccc)'does match the response 'abcd ffff ccc' but does not match the response 'abcd ccc ffff'
Word groups	[ ]	[ ] enables multiple words to be accepted as an alternative to other single words in OR lists. [ ] may not be nested. Single words may be OR'd inside [ ]. Where a word group is preceded or followed by the proximity control the word group is governed by the proximity control rule that the words must be in the order given.
Single character wildcard	?	Matches any single character.
Multiple character wildcard	*	Matches any sequence of characters including none.

It is possible to match some of the special characters by ‘escaping’ them with the ‘\’ character. So match(\|) will match ‘|’. Ditto for _, [, ], and *. And if you wish to match round brackets then match(\(\)) will match exactly ‘()’.

________________________________________________________________________

2.31

2.3 Combining matches

match_all()	All matches contained within the brackets must be true.	match_all( match_mow(first) match_mow(second) )
match_any()	Just one of the matches contained within the brackets must be true.	match_any( match_mow(first) match_mow(second) )
not()	The match within the brackets must be false.	not ( match_any( match_mow(first) match_mow(second) ) )

match_all(), match_any()and not() may all be nested.

________________________________________________________________________

2.32

2.4 Pattern match examples

Student response	Matching options	Pattern match	match method return
tom dick harry	empty	tom dick harry	True. This is the exact match.
thomas	c	tom	True. Extra characters are allowed anywhere within the word.
tom, dick and harry	w	dick	True. Extra words are allowed anywhere within the sentence.
harry dick tom	o	tom dick harry	True. Any order of words is allowed.
rick	m	dick	True. One character in the word can differ.
rick and harry and tom	mow	tom dick harry	True.
dick and harry and thomas	cow	tom dick harry	True.
arthur, harry and sid	mow	tom|dick|harry	True. Any of tom or dick or harry will be matched.
tom, harry and sid	mow	tom|dick harry|sid	True. The pattern requires either tom or dick AND harry or sid. Note that 'tom,' is only allowed because m allows the extra character, the comma, in 'tom,'.
tom was mesmerised by maud	mow	[tom maud]|[sid jane]	True. The pattern requires either (tom and maud) or (sid and jane).
rick	empty	?ick	True. The first character can be anything.
harold	empty	har*	True. Any sequence of characters can follow 'har'.
tom married maud, sid married jane.	mow	tom_maud	True. Only one word is between tom and maud.
maud married tom, sid married jane.	mow	tom_maud	False. The proximity control also specifies word order and over-rides the allowAnyWordOrder matching option.
tom married maud, sid married jane.	mow	tom_jane	False. Only two words are allowed between tom and jane.
tom married maud	mow	tom|thomas marr* maud	True.
maud marries thomas	mow	tom|thomas marr* maud	True.
tom is to marry maud	mow	tom|thomas marr* maud	True.
tempratur	m2ow	temperature	True. Two characters are missing.
temporatur	m2ow	temperature	True. Two characters are incorrect; one has been replaced and one is missing.

________________________________________________________________________

2.33

How can you possibly guess the multiplicity of phrases that your varied student cohort will use to answer a question? Of course you can’t, but you can record everything and over time you will build a bank of student responses on which you can base your response matching. And gradually you might be surprised at how well your response matching copes.

Before we describe the response matching it’s worth stressing:

1. The starting point is asking a question that you believe you will be able to mark accurately.
2. How you phrase the question can have a significant impact.
3. Pattern match works best when you are asking for a single explanation that you will mark as simply right or wrong. You will find that dealing with multiple parts in a question or apportioning partial marks is a much harder task that will quickly turn into a research project.
4. A bank of marked responses from real students is an essential starting point for developing matching patterns. Consider using the question in a human marked essay to get your first bank of student responses. Alternatively first use the question in a deferred feedback test where marks are allocated once all responses are received.
5. This question type possibly more than all others demands that you monitor student responses and amend your response matching as required. There will come a point at which you believe that the response matching is ‘good enough’ and might wonder ‘how many responses might I need to check?’ Unfortunately there is no easy answer to this question. Sometimes 200 responses will be sufficient. At other times you may have to go much further than this. If you do bear in mind that
   1. Asking students to construct their response, as opposed to choosing it from a list, asks more of the student i.e. it is worth your while to do this.
   2. Human markers following a mark scheme are fallible too. Ditto.
6. Stemming is the accepted method of catering for different word endings e.g. mov* will cater for ‘moved’ and ‘moves’ and ‘moving’ etc.
7. Students do not type responses that are deliberately wrong. Why should they? This is in contrast to academics who try to defeat the system. Consequently you will not find students entering the right response but preceding it with ‘it is not..’ And consequently you do not have to match deliberate errors.
8. The proximity control enables you to link one word to another.
9. It is often the case that marking obvious wrong responses early in your matching scheme will improve your overall accuracy.
10. Dealing with responses that contain both right and wrong answers requires you to take a view. You will either mark all such responses right or wrong. The computer will carry out your instructions consistently c.f. asking a group of human markers to apply your mark scheme consistently.
11. You should aim for an accuracy of >95% (in our trials our human markers were in the range 92%-96%). But, of course, aim as high as is reasonable given the usual diminishing returns on continuing efforts.

2.2.2 Short answer

The response matching in the Short answer question type is very limited. Consider using the Pattern match question type instead.

2.34 The Question text, Grade and General feedback

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

Default mark: Decide on how to score your questions and be consistent.

General feedback: We recommend that all questions should have this box completed with the correct answer and a fully worked explanation. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect. We do not recommend that authors rely on using the machine generated 'Right answer' (from the iCMA definition form).

2.35 The response matching fields

Case sensitivity: The author can specify whether or not the case of the response is important.

Answer: The Short answer question type will search for a sequence of consecutive characters within a response. The wildcard character '*' allows other words and characters, including space, to be present.

Beyond this wildcard there are no other response handling facilities. The Pattern match question type offers more sophisticated features. For example if you wished to look for ‘Tom’ and ‘Dick’ and ‘Harry’ in any order then Pattern match allows you to do this easily but Short answer requires you to specify all possible sequences.

The wildcard '*' on its own in the 'answer' field will match 'any other answer'.

Feedback: Use this to give feedback to the responses that are matched by the current response match.

2.2.3 Combined

The Combined question enables multiple text responses to be matched.

The full capabilities for the Combined question is described in section 2.4.1.

2.36 A combined question with three Pattern match input fields

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

General feedback: We recommend that all questions should have this box completed with the correct answer and a fully worked explanation. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect. There is no system generated 'Right answer' (from the iCMA definition form) for Combined questions.

2.37 The three Pattern match response matching fields

Thie Combined question uses a restricted form of the Pattern match response matching.

Weighting. Different responses can have different percentages of the total mark. The weightings must add to 100%.

Allow use of subscript/superscript: for students to include powers or chemical formulae.

Case sensitivity: Yes or no.

Check spelling of student: Yes or no.

Answer: Use Pattern match syntax.

Feedback for any incorrect response: Where some fields are answered correctly and others incorrectly the intention is that the feedback associated with the field is used to say what is wrong. Correct answers can be counted up using the the option to 'Show the number of correct responses' in the Combined feedback and Hints fields.

2.38 The combined feedback section

2.39 Hints for use in Interactive with multiple tries behaviour

If you are configuring your question to run in 'Interactive with multiple tries' mode such that students can answer questions one by one you can also provide hints and second (and third) tries by entering appropriate feedback into the boxes provided in the 'Multiple tries' section of the question definition form.

To use this section of the question form please ensure that on the iCMA definition form you have chosen 'How questions behave' = 'Interactive with multiple tries'.

Penalty for each incorrect try: The available mark is reduced by the penalty for second and subsequent tries. In the example above a correct answer at the second try will score 0.6666667 of the available marks and a correct answer at the third try will score 0.3333334 of the available marks.

Hint: You can complete as many of these boxes as you wish. If you wish to give the student three tries at a question you will need to provide two hints. At runtime when the hints are exhausted the question will finish and the student will be given the general feedback and the question score will be calculated.

2.2.4 Essay

Essay questions are not marked automatically unlike other questions in the iCMA. If an Essay question is to be marked this has to be undertaken by an appointed marker who is given appropriate permission by Exams and Assessments; the Tutor role in OU Moodle does not have permission to mark questions.

We recommend that essay questions are shown one per page.

2.40 An essay question

Question text: You may use the full functionality of the editor to state the question.

General feedback: Perhaps use this for a model answer.

Response format: With or without the normal on-screen editor.

Input box size

Allow attachments: Any text entered here will be displayed in the response input box when a new attempt at the question starts.

Response template: Any text entered here will be displayed in the response input box when a new attempt at the question starts.

Information for graders: The mark scheme can be placed here.

2.3 Selection questions

Questions in this category provide the correct answers together with distractors and ask the student to choose or place their choices.

2.3.1 Drag and drop into text

Drag and drop questions for completing texts are a popular form of 'selection' question consisting of words which can be dragged from a list and dropped into pre-defined gaps in the text.

2.41 The Question text, mark and General feedback of a Drag and drop into text question

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

The Question text is written with two sets of square brackets '[[n]]' indicating the positioning of gaps and a number 'n' inside the brackets indicating the correct choice from a list which follows the question.

It is allowed to place the drop zones in lists and tables.

General feedback: We recommend that all questions should have this box completed with the correct answer and a fully worked explanation. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect. We do not recommend that authors rely on using the machine generated 'Right answer' (from the iCMA definition form).

2.42 The choices for a drag and drop words into text question

Limited formatting of text using _{, ^{, , , and is allowed.}}

Choices that are within the same Group are colour coded and may only be dropped in a gap with the corresponding colour.

Choices that are marked as 'infinite' may be used in multiple locations.

It is not possible to have drag boxes containing multiple lines. If you want to drag long sentences - don't. Give each a label and drag the label.

The resulting question when run in 'interactive with multiple tries' style looks as follows:

2.43 How a drag and drop question might look at runtime

2.44 The combined feedback section of the question editing form

The display of Combined feedback is governed by the Specific feedback setting on the iCMA definition form.

In interactive with multiple tries mode Combined feedback is shown after every try as well as when the question completes.

The option ‘Show the number of correct responses’ is over-ridden in interactive with multiple tries mode by the same settings in the ‘Settings for multiple tries’ section of the editing form.

2.45 The multiple tries section of the question editing form

Penalty for each incorrect try: The available mark is reduced by the penalty for second and subsequent tries. In the example above a correct answer at the second try will score 0.6666667 of the available marks and a correct answer at the third try will score 0.3333334 of the available marks.

If the question is used in 'interactive with multiple tries' behaviour the marking is modified as follows:

1. The mark is reduced for each try by the penalty factor.
2. Allowance is made for when a correct choice is first chosen providing it remains chosen in subsequent tries.

Hint: You can complete as many of these boxes as you wish. If you wish to give the student three tries at a question you will need to provide two hints. At runtime when the hints are exhausted the question will finish and the student will be given the general feedback and the question score will be calculated.

Clear incorrect responses: When ‘Try again’ is clicked incorrect choices are cleared.

Show the number of correct responses: Include in the feedback a statement of how many choices are correct.

2.3.2 Drag and drop onto image

Drag and drop questions for labelling images are a popular form of 'selection' question consisting of images or words which can be dragged from a list and dropped into pre-defined gaps on the base image.

2.46 The Question text, mark and General feedback of a Drag and drop onto image question

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

General feedback: We recommend the correctly labelled image be included in this box. Students who did not answer completely correctly can then compare and contrast to see where they made an error. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect.

2.47 Defining the background image and placing the drop areas

Start by uploading a background image. There is a maximum size of 600 x 400 pixels for this image. Please note that we recommend a maximum image width of 570 pixels so as to fit all parts of an iCMA into a 1024 wide window. When you have done this it will appear in the preview area.

2.48 Defining the items, text or images, to be dragged

Next create the drag items. These may be text items as shown here, or images. There is a maximum import size for images of 150 x 100 and images larger than this will be scaled on import. If you choose to use draggable images please ensure that you add a short text descriptor. For each item having entered the text please press Enter to complete the addition of the draggable item and register it in the drop-down lists of Drop zones.

Now place the drag items onto the background image by completing the Drop zones section above. Once you have established the draggable item for a drop zone the item will appear beneath the background image. At this point you may position the item either by dragging it or by completing the Left and Top boxes for the item.

The drag item will be placed at the top and left coordinates as measured in pixels from the top left of the background image.

Within draggable text items limited formatting of text using _{, ^{, , , and is allowed.
is also allowed.}}

Choices that are within the same Group are colour coded and may only be dropped on a drop zone with the corresponding colour. Choices that are marked as 'infinite' may be used in multiple locations.

The resulting question when run in 'interactive with multiple tries' style looks as follows:

2.49 How a drag and drop question might look

2.50 The combined feedback section

Whether or not Combined feedback is shown to students is governed by the Specific feedback setting on the iCMA definition form.

2.51 The multiple tries section

Penalty for each incorrect try: The available mark is reduced by the penalty for second and subsequent tries. In the example above a correct answer at the second try will score 0.6666667 of the available marks and a correct answer at the third try will score 0.3333334 of the available marks.

If the question is used in 'interactive with multiple tries' style the marking is modified as follows:

1. The mark is reduced for each try by the penalty factor.
2. Allowance is made for when a correct choice is first chosen providing it remains chosen in subsequent tries.

Hint: You can complete as many of these boxes as you wish. If you wish to give the student three tries at a question you will need to provide two hints. At runtime when the hints are exhausted the question will finish and the student will be given the general feedback and the question score will be calculated.

Clear incorrect responses: When ‘Try again’ is clicked incorrect choices are cleared.

Show the number of correct responses: Include in the feedback a statement of how many choices are correct.

2.3.3 Matching

Matching questions are created by the author supplying several questions and their correct answers. The computer lays out the questions and shuffles the answers in a drop-down list. Matching questions raise the same issues as multiple choice in that all answers are supplied and students are 'matching' answers rather than providing them. If the same number of matching options is provided as there are questions then it is the case that students do not have to know all the correct answers to obtain full marks; if there is one answer they do not know then it is clearly the one that is left over after all other questions have been answered. However it is possible to add extra answers as distractors to make the question more challenging.

2.52 The Question text, mark and General feedback of a Matching question

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

General feedback: We recommend the correct answer be included in this box. Students who did not answer completely correctly can then compare and contrast to see where they made an error. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect.

2.53 The Answers section

The answer fields in Matching questions can only contain plain text.

2.54 The combined feedback section

Whether or not Combined feedback is shown to students is governed by the Specific feedback setting on the iCMA definition form.

2.55 The multiple tries section

Hint: You can complete as many of these boxes as you wish. If you wish to give the student three tries at a question you will need to provide two hints. At runtime when the hints are exhausted the question will finish and the student will be given the general feedback and the question score will be calculated.

Clear incorrect responses: When ‘Try again’ is clicked incorrect choices are cleared.

Show the number of correct responses: Include in the feedback a statement of how many choices are correct.

2.3.4 Multiple choice

The Multiple Choice question type is the most common form of question used in the OU's paper-based CMA system which was used by large numbers of modules in the 1970s, 1980s and 1990s and remains in use today.

Typically the student is provided with a question and a range of possible answers and asked to choose one. The skill in writing multiple choice questions is in providing questions that expose misunderstanding of the materials under test such that suitable distractors can be written. The response to the distractors can then be used to provide remedial feedback to help students overcome their misunderstanding.

A problem with all forms of multiple choice questions is that students can guess the answer from the range of options provided. To discourage this various penalty mechanisms have been suggested to try and prevent students from simply guessing. In its current form Moodle allows the author to attach negative marks to options though most users at the OU do not use this facility and score the correct choice at 100% and incorrect choices at 0%.

2.56 The Question text, mark and General feedback

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

General feedback: We recommend the correctanswer be included in this box. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect.

2.57 The available choices

One or multiple answers? It is possible to use the Multiple choice question to create a Multiple response question that requires the student to choose multiple options. However marking these questions can become complex. If for example a correct response requires that options (a), (b) and (c) be chosen then each could be allocated 33.3% of the marks, but what marks are allocated to options (d), (e) and (f)? Should it be zero or a negative value; if the latter at what level, -33.3%, -100%? This can become difficult and because of this we strongly recommend that authors wishing to create Multiple Response questions use the OU Multiple Response question type which incorporates the mark schemes used in the original paper-based CMA system.

2.58 The Combined section

2.59 The multiple tries sections

Penalty for each incorrect try: The available mark is reduced by the penalty for second and subsequent tries. In the example above a correct answer at the second try will score 0.6666667 of the available marks and a correct answer at the third try will score 0.3333334 of the available marks.

Hint: You can complete as many of these boxes as you wish. If you wish to give the student three tries at a question you will need to provide two hints. At runtime when the hints are exhausted the question will finish and the student will be given the general feedback and the question score will be calculated.

Clear incorrect responses: When ‘Try again’ is clicked incorrect choices are cleared.

Show the number of correct responses: Include in the feedback a statement of how many choices are correct. Clearly this should not be used for questions with single choices.

2.3.5 OU multiple response

Along with the Multiple Choice question type, the Multiple Response question type is the most common form of question used in the OU's paper-based CMA system. The Multiple Response question type requires the student to choose multiple options.

2.60 The Question text, mark and General feedback

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

General feedback: We recommend the correct answer be included in this box. Students who did not answer completely correctly can then compare and contrast to see where they made an error. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect.

2.61 The available choices

The OU Multiple Response question type simplifies the authoring process by incorporating the marking scheme developed for the OU paper-based CMA system. In this marking scheme

• Choices are marked as correct or incorrect. If a question has n correct choices each correct choice is given a mark of (Question grade)/n.
• If a student selects more than n choices the marks are based on the worst n choices.
• It follows that the number of incorrect choices should equal or exceed the number of correct choices (otherwise a student choosing all choices will receive marks).

Example: A question has five choices of which only 'a' and 'b' are correct. How student responses are marked:

• a b: 100% correct
• a c: 50% correct
• a b c: 50% correct - only the two worst choices are marked
• a b c d: 0% correct - only the two worst choices are marked

2.62 The Combined feedback section

Whether or not Combined feedback is shown to students is governed by the Specific feedback setting on the iCMA definition form.

2.63 The multiple tries section

Penalty for each incorrect try: The available mark is reduced by the penalty for second and subsequent tries. In the example above a correct answer at the second try will score 0.6666667 of the available marks and a correct answer at the third try will score 0.3333334 of the available marks.

If the question is used in interactive with multiple tries mode the marking is modified as follows:

1. At each try only the worst n choices are scored.
2. The mark is reduced for each try by the penalty factor.
3. Allowance is made for when a correct response is first chosen providing it remains chosen in subsequent tries.

Hint: You can complete as many of these boxes as you wish. If you wish to give the student three tries at a question you will need to provide two hints. At runtime when the hints are exhausted the question will finish and the student will be given the general feedback and the question score will be calculated.

Clear incorrect responses: When ‘Try again’ is clicked incorrect choices are cleared.

Show the number of correct responses: Include in the feedback a statement of how many choices are correct.

Show the feedback for the selected responses: This setting controls the feedback to individual choices however if too many choices are made it is suppressed.

2.3.6 Select missing words

The Select missing words question type is very similar to the Drag and drop words into text question type. It is useful for questions where the on-screen space is insufficient to provide space for both the question and the drag items – if the drag items are off-screen it’s rather hard to drag them anywhere.

This question type will therefore most likely be used when the question author has a large text that they wish the student to label or complete by selecting missing words from a drop-down list. Though, of course, some authors might prefer to use it to the drag and drop alternative.

2.64 The Question text, mark and General feedback

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

The Question text is written with two sets of square brackets '[[n]]' indicating the positioning of gaps and a number 'n' inside the brackets indicating the correct choice from a list which follows the question.

General feedback: We recommend the correct answer be included in this box. Students who did not answer completely correctly can then compare and contrast to see where they made an error. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect.

2.65 The choices for a missing words question

Answer: Unlike the Drag and drop words into text question type the Select missing words question allows no formatting of the words to be selected.

Group: Choices that are within the same Group appear in the same drop-down lists.

The resulting question when run in 'interactive with multiple tries' style looks as follows:

2.66 A select missing words question

2.67 The combined feedback section

Whether or not Combined feedback is shown to students is governed by the Specific feedback setting on the iCMA definition form.

2.68 The multiple tries section

Penalty for each incorrect try: The available mark is reduced by the penalty for second and subsequent tries. In the example above a correct answer at the second try will score 0.6666667 of the available marks and a correct answer at the third try will score 0.3333334 of the available marks.

If the question is used in 'interactive with multiple tries' style the marking is modified as follows:

1. The mark is reduced for each try by the penalty factor.
2. Allowance is made for when a correct choice is first chosen providing it remains chosen in subsequent tries.

Hint: You can complete as many of these boxes as you wish. If you wish to give the student three tries at a question you will need to provide two hints. At runtime when the hints are exhausted the question will finish and the student will be given the general feedback and the question score will be calculated.

Clear incorrect responses: When ‘Try again’ is clicked incorrect choices are cleared.

Show the number of correct responses: Include in the feedback a statement of how many choices are correct.

2.3.7 True/false

The True/false question is the simplest question type. There is one right answer, worth 100% and one wrong answer worth 'none'.

2.69 The True/false editing form

2.3.8 Combined

The Combined question enables multiple OU multiple response and Select missing words responses to be matched.

The full capabilities for the Combined question is described in section 2.4.1.

2.70 A combined question with various select fields

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

General feedback: We recommend that all questions should have this box completed with the correct answer and a fully worked explanation. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect. There is no system generated 'Right answer' (from the iCMA definition form) for Combined questions.

2.71 Four Selectmenu and multireponse response matching fields

These are restricted forms of the Selectmenu and OU multiple response response matching fields.

2.72 The combined feedback section

2.73 The multiple tries section

Where some fields are answered correctly and others incorrectly the intention is that the feedback associated with the field is used to say what is wrong. Correct answers can be counted up using the the option to 'Show the number of correct responses' in the Combined feedback and Hints fields.

2.4 Multiple inputs

There are two question types that accept responses to multiple inputs.

2.4.1 Combined

The Combined question incorporates features of four existing question types into one composite question. The four question types that are included are:

• Numeric (n.b. based on Variable numeric and not Numerical)
• Pattern match
• OU multiple response
• Select missing words.

2.74 A combined question with four input fields

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

General feedback: We recommend that all questions should have this box completed with the correct answer and a fully worked explanation. The contents of this box will be shown to all students irrespective of whether their response was correct or incorrect. There is no system generated 'Right answer' (from the iCMA definition form) for Combined questions.

2.75 Four response matching fields

These are restricted forms of the Variable numeric, Pattern match, OU multiple response and Selectmenu response matching fields.

Weighting. Different responses can have different percentages of the total mark. The weightings must add to 100%.

Where some fields are answered correctly and others incorrectly the intention is that the feedback associated with the field is used to say what is wrong. Correct answers can be counted up using the the option to 'Show the number of correct responses' in the Combined feedback and Hints fields.

2.76 The combined feedback field

2.77 The multiple tries field

The resulting question when run in interactive with multiple tries behaviour looks as follows:

2.78 How a Combined question might look.

2.5 2D questions

2.5.1 Drag and drop marker

Drag and drop marker questions allow for multiple markers to be placed onto an image and the positioning of each marker to be assessed.

Drag and drop marker questions differ from Drag and drop onto image questions in that there are no predefined areas on the underlying image that are visible to the student.

2.79 The Question stem, Default mark and General feedback section

Question name: A descriptive name is sensible. This name will not be shown to students.

Question text: You may use the full functionality of the editor to state the question.

Highlight drop zones which have not had the correct marker dropped on them: Does what it says when the General feedback is shown.

There is a short video on this on YouTube if that's your thing...https://www.youtube.com/ watch?v=Tvl4GuGvtxc

2.80 The preview section

Start by uploading a background image. There is a maximum size of 600 x 400 pixels for this image. Please note that we recommend a maximum image width of 570 pixels so as to fit all parts of an iCMA into a 1024 wide window. When you have done this it will appear in the preview area.

2.81 Placing the drop zones

Next create markers by assigning a text label. And for each label set the allowed drop zone on the background image. Zones may have one of three basic shapes:

• Circle – provide the x and y coordinates of the centre and the radius. The format is x,y;r
• Rectangle – provide the x and y coordinates of the top left corner and the width and height. The format is x,y;w,h
• Polygon – provide the x and y coordinates of a range of points that define the corners of the polygon. Straight lines joining these corners will define the edges of the polygon. The format is x₁,y₁;x₂,y₂;x₃,y₃;…x_n,y_n

  This online image map tool provides an easy interface for plotting the points of complex polygons though you will have to amend the syntax of the coordinates produced for inclusion in the question.

All x dimensions are in pixels from the left of the image and all y dimensions are in pixels from the top of the image.

Within marker text limited formatting using , , and is allowed.
is also allowed.

Markers that are marked as 'infinite' may be associated with multiple drop zones.

The centre of each marker is defined as the centre of the small circle to the top left of each label.

It is possible to provide more markers than are needed to complete the question thereby providing an extra level of distraction.

2.82 The combined feedback section

2.83 The multiple tries section

If the question is used in 'interactive with multiple tries' style the marking is modified as follows:

1. The mark is reduced for each try by the penalty factor.
2. Allowance is made for when a correct choice is first chosen providing it remains chosen in subsequent tries.

Show the number of correct responses: Include in the feedback a statement of how many choices are correct.

State which markers are incorrectly placed: Include in the feedback a list of markers which are incorrectly placed.

Move incorrectly placed markers back to default start position below image: When ‘Try again’ is clicked incorrectly positioned labels are removed.

The resulting question when run in interactive with multiple tries behaviour looks as follows:

2.84 How a Drag and drop marker question might look

The centre of each marker is defined as the centre of the small circle to the top left of each label.

2.5.2 Pattern match with JME

Pattern match with JME combines the Pattern match algorithm with Peter Ertl’s JavaScript Molecular Editor (http://peter-ertl.com/ jsme/ ) to provide questions that allow students to draw their answer.

The JavaScript Molecular Editor is able to convert 2D (really 3D) molecular structures into SMILES strings (http://www.daylight.com/ dayhtml/ doc/ theory/ theory.smiles.html ) which may then be readily matched using Pattern match. A variety of specialised feedback options are provided.

The Question stem, Default mark and General feedback sections are as usual.

2.85 The Question text and General feedback for a Pattern match with JME question

2.86 Controls of how stereochemistry is handled by the JME.

1,2-Difluoroethene has the formula CHF=CHF. Stereochemically there are two possibilities E and Z.

Z-difluoroethene and E-difluoroethene

With stereochemistry turned on the JME will return F/C=C\F for the Z isomer and F/C=C/F for the E isomer.

Please be aware that the ‘\’ character is the ‘escape’ character in the pattern match algorithm that enables ‘special characters’ to be matched. For example match(\|) will match ‘|’. Consequently in the response match for Z-difluoroethene the ‘\’ character is used to ‘escape’ itself and the response match must be written as match(F/C=C\\F).

2.87 Synonyms can be defined

2.88 A response match of a SMILES string

Answer: The first answer must be the SMILES string for the correct answer as this is used as the basis for the Atom count feedback. In this case the correct answer for acetic acid is ‘CC(=O)O’. Please note that brackets are special characters that have to be ‘escaped’ as shown in the example.

Grade: Between ‘none’ and 100%. At least one response must have a mark of 100%.

Feedback: Specific feedback that is provided to anyone whose response is matched by the response matching rule in Answer.

Atom count feedback: If this is ticked the system generates feedback based on a comparison of the student’s response and the correct answer given in Answer 1

2.89 Feedback to all non-matched responses

The feedback for all non-matched responses should go into the ‘Any other answer’ field.

Atom count feedback: The student’s response is compared against the correct answer given in ‘Answer 1’ and feedback is generated on the atoms that are in both the student’s response and in the correct answer

The resulting question when run in 'interactive with multiple tries' style looks as follows:

2.90 How a Pattern match with JME question might look

The student draws their response using the tools provided by the JME.

2.6 STACK

System for Teaching and Assessment using a Computer algebra Kernel has been developed by Christopher Sangwin (ex. Birmingham University now Loughborough University) and integrated as a Moodle question type by Tim Hunt at the OU. It uses the features of the Maxima computer algebra system to support the setting of questions with a mathematical basis and the analysis of responses that are provided in the form of numbers and functions.

A large set of example STACK questions are available on the main STACK site at http://stack.bham.ac.uk.

2.91 An example of a STACK question

When writing mathematics in a STACK question please note the following:

1. Do not include unicode characters for Greek characters or symbols but instead use the TeX form e.g. \lambda. See Supported TeX commands under MathML in the OLS-Guide.
2. Do not use $...$ around TeX in STACK questions. Instead use
   • $...$ for inline maths
   • $...$ for display maths
   • @...@ to display the values of STACK variables. It is not necessary to wrap \(\) or \[\] around it. @...@ displays as inline maths.

   Following this convention enables us to share STACK questions with non-OU sites.

3. Do not use en-dash to emphasise the minus sign. The maths renderer will convert your keyboard minus into the more prominent minus sign (U+2212) for display.
4. Do not put CASText into the Options fields (which hold the standard feedback for correct, partially correct, incorrect). These fields are currently configured to hold text only.

To learn more please use the standard STACK documentation which can be found here http://stack.bham.ac.uk.

STACK uses a specially configured Maxima environment and the STACK documentation details how to configure your desktop version of Maxima to work in this environment. You will need a copy of the configuration file maxima.mac.

2.7 OpenMark

OpenMark questions are made available to Moodle iCMAs through the Opaque question type.

2.92 An Opaque question form pointing at an OpenMark question

Question name: A descriptive name is sensible. This name will not be shown to students.

Question engine: The OpenMark server that is handling the question. Developments should be undertaken on the development server. When all testing is complete the question should be moved to the 'live servers' ready for use by students and this entry should be changed to match.

Question id: The name of the question on the OpenMark server e.g. mu120.module5.question05

Question version: OpenMark supports version numbers of questions usually starting at 1.0. Provide the version number here. Please note that should you alter the version number in 'mid-test', Moodle will start to use the new version for all students immediately. Never alter the logic of a question in 'mid-test'.

2.7.1 No. of tries and marking

The number of tries that a student is allowed at a question and the marking of the question is carried out entirely within OpenMark. The question author must ensure that they are 'extending' a suitably configured OpenMark class when writing their questions, for example SimpleQuestion1. OpenMark will share appropriate information with Moodle.

OpenMark questions that are to be used in Deferred feedback iCMAs must extend the OpenMark class DeferredFeedbackQuestion().

The following OpenMark functions and XML components control the number of tries* and the marking and supply this information to Moodle

• 3 in the question XML will set the maximum mark. Please be aware that OpenMark will only export integers. However it is possible to scale these once they are in Moodle e.g. if a question has a maximum mark of 3 and a mark of 1 is achieved this can be reported in Moodle as 0.33.
• protected void setMaxAttempts(int iMaxAttempts) - to set the number of tries.
• getResults().setScore(int iMarks, int iAttempts) - to set the score and whether the question has been answered correctly, partially correctly or incorrectly. The parameter iAttempts is used as follows.
  • if the question has been answered correctly provide the attempt at which the correct answer was given.
  • if it is partially correct after the maximum number of tries use the constant ATTEMPTS_PARTIALLYCORRECT.
  • if it is incorrect after the maximum number of tries use the constant ATTEMPTS_WRONG.

For examples of the partial scoring please consult the SDK125 interactive media developers who have developed SimpleQuestion3 that allows two tries with partial scoring after the second try.

2.7.2 Storing student data

Student data associated with OpenMark questions that are included in a Moodle iCMA are stored in the Moodle database. As is usual with OpenMark it is the responsibility of the interactive media developer to decide what information should be stored.

The OpenMark question developer may use

• getResults().setQuestionLine("Variant=" + iVariant);

to store any information on the variant of the question in the Moodle database. Using the above form for setQuestionLine() enables Moodle to calculate the usual statistics for each variant of a question as well as the overall statistics for all variants of the question. This information can then be extracted by the author using the Statistics report. Use

• getResults().setAnswerLine(responseString)

to store the student's responseString in the Moodle database. This information can then be extracted by the author using the Responses report.

Moodle does not use

• getResults().appendActionSummary()

2.7.3 iCMA reports

Marks and responses stored in the Moodle database can be queried with the Grades and Responses features from the Results page for the iCMA. The Statistics feature also includes OpenMark questions from the December 2014 release.

2.8 Descriptions

Descriptions enable you to include blocks of information within an iCMA. They carry no score and are labelled in the navigation panel by the letter 'i'. (Think of the i for tourist information centres.)

2.93 A Description editing form

2.9 Including images and audio-visual files

2.9.1 Images

Images may be included by clicking on the icon. This icon allows you to upload images from your hard disk where they will be stored with the question. However we recommend that images that appear within questions should not exceed 570 pixels wide. At this width the navigation panel, the status panel and the image will all fit in a screen that is 1024 pixels wide.

2.9.2 Large images

On occasion students are asked to read information off an image. If an image that is 570 pixels wide is not large enough to show the information in sufficient detail we recommend the following:

1. upload the image at 570 pixels wide and show it in the question
2. prepare a pdf file from an original, larger, version of the image
3. below the image in the question write ‘right-click to view enlarged image in a separate tab’
4. highlight ‘enlarged image’ and click on the ‘Insert/edit link’ icon in the editor toolbar.
5. in the ‘Insert/edit link’ dialogue select the’ browse’ icon
6. browse to the pdf on your hard-disk and upload the pdf; this associates the pdf with the question

Students may then be instructed to open the enlarged image in a separate tab and inspect it closely before answering the question.

2.9.3 Audio-visual files

You can include the following within your question:

• flash animations - files ending in .swf.
• audio - files ending in .mp3.
• video – MPEG4 (ending in .mp4) or Flash (ending in .flv) video files.

2.10 Including TeX and MathML

The Maths On-Line Project's preferred way of displaying mathematical equations in the VLE is to use MathML. However writing MathML is not so easy. The alternative is to use TeX.

Within the VLE both TeX and MathML are displayed through a Mathematics service which will also convert between the two, i.e. authors can enter TeX and the mathematics service will create the corresponding MathML. Students may choose to receive their mathematics as images from TeX or as MathML that is interpreted by their browser.

To see a rendition of your maths as you type it use the 'insert equation' tool . Choose the appropriate tab and enter your expression. Here are a couple to try.

This following TeX will render as one half.

2.94 TeX typed into the editor

The equivalent MathML is:

Note the annotation field. This is used for building the TeX based Alt-text.

Should you wish to include MathML here is a paper on generating MathML with MathType. http://www.dessci.com/ en/ reference/ white_papers/ TUG_paper.pdf .

3 Preview your questions

Having written your question you will want to know how it looks to the student. Alongside the question name in the question bank under each category are a series of icons.

The preview icon will allow you to try the question. The system tries to apply 'intelligent' initial settings for you to test your question but you are given full control of how you preview the question using the 'Change options' settings.

3.1 Question preview options

The Question variant field is also available for the Variable numeric trio of questions.

If you wish to amend your question you may go back to the question editing form by clicking on the edit icon .

And if your question isn't at all what you intended the delete icon will remove it.

Please note:

• if you accidentally delete a question that is part of an iCMA, Moodle will hide the question and not delete it - you can show it again with 'Also show old questions'.
• Moodle is backed up daily but the backup is really in case there is a catastrophic emergency and we have to build our systems again somewhere else.
• if you're worried about the odd question or two being accidentally deleted the easiest reassurance is to take your own copy. It's straightforward to export your questions to a file. The export link is in the Question bank links in the Website administration block.Be sure to choose Moodle XML format, download the xml file that is created and store it safely.

4 Tips and Tricks

4.1 A word on the Moodle editor

1. In general the editor is intended to allow you to enter simple pieces of text such as forum posts, blog entries, news messages, and longer pieces of structured text such as wiki entries, but it is not intended to allow you to style content in any way.
2. Using the GUI editor controls should mostly work. The exception is GUI controls inside various of the dialogue boxes, detailed table options and detailed image options.
3. Editing HTML source directly is unlikely to give you exactly what you wanted as the editor tries to tidy up your HTML. However users can turn off the editor in their own Moodle profile.

4.2 Showing computing code fragments

The HTML tag is an inline tag that can be used to indicate snippets of computer code within sentences. However to use this tag authors must switch to HTML mode to enter it.

The ‘preformatted’ option in the format menu of the editor is an alternative for showing blocks of code in a different font.

However the font for

 and  is different. If you wish to mix inline and block code snippets and keep the font consistent you will probably have to amend preformatted blocks with 
…
 again by putting the editor into HTML mode.

4.3 More table controls

Table cells may be individually controlled.

4.4 Entering fractions with the combined question type

In a combined question it is possible to arrange two input boxes such that they look like the numerator and denominator in a fraction using the following HTML.