Design: Tips and Checklist

3. Checklist of e-Assessment Tools

VLE e-Assessment Tools

1. Objective / MCQ style tests - interactive multiple-choice, short-answer, jumbled-sentence, crossword, matching/ordering, drag and drop, and gap-fill exercises are some of the options.

2. Assignment Submission box - typically used for essays and reports (the assignment / report can be authored by individuals or groups) - the classic VLE ‘dropbox’ as it is often called. The assignment activity module enables a teacher to communicate tasks, collect work and provide grades and feedback. Students can submit any digital content (files), such as word-processed documents, spreadsheets, images, or audio and video clips. Alternatively, or in addition, the assignment may require students to type text directly into a text editor. An assignment can also be used to remind students of 'real-world' assignments they need to complete offline, such as practical work, and thus not produce any digital content. One benefit of this is that the teacher can use the Assignment grading tools to mark ‘real’ world practical exercises that involve the creation physical artefacts. Similarly, Assignments are often used to mark the digital contents of student e-Portfolios that exist outside the VLE. Students can submit work individually or as a member of a group. When reviewing assignments, teachers can leave feedback comments and upload files, such as marked-up student submissions, documents with comments or spoken audio feedback. Assignments can be graded using a numerical or custom scale or an advanced grading method such as a rubric. Final grades are recorded in the Gradebook.

3. Assignment Submission Box (Turnitin) – The online plagiarism detection service Turnitin is widely used in FE and HE to generate ‘similarity’ reports that provide information about what parts of a student’s work are similar to work produced elsewhere. It is usually integrated into a VLE as an option. What is less widely known about the service is that it also includes a powerful grading toolkit that is generally more user friendly that those in VLEs (although that is changing). Drawbacks to using this service include; learning another system and interface, a student cannot make multiple submissions to same assessment – each submission overwrites the previous one.

4. Rubric– in Moodle / Turnitin / Blackboard etc. rubrics are associated with the Assignment Submission Box. It’s a grading form that uses a table structure, containing a set of criteria (usually down the left hand side) with specified levels of performance to the right of each criterion that form the ‘boxes’ of the table. Clicking on a rubric box for each criterion will automatically create a grade for a student and generate consistent feedback by using the criteria components and their performance level. Personalized feedback can be created as well and added in addition to the auto feedback. When the marks are released the student will see in their view of the rubric the marks and feedback. This has the potential to speed up marking and make it more consistent. Note you can set up a rubric but don’t have to use the associated essay style submission. So, this could be used for other types of assessment activity other than essays - like practice based assessment in order to produce marks and feedback based on the rubric. This may be more useful for graded units in a qualification, although it could be used with ‘pass or fail’ units as well.

5. Marking Guide – in Moodle – This is a form that that contains the criteria, a space for comments and a space to manually enter the marks for that criterion. It will store frequently used comments as well making feedback easier It is simpler than a rubric and may be more suitable for using when grading ‘Pass or Fail’ SQA units.

6. Chat – a chat room for real-time text chat by students and staff. Can be good for recording discussions and planning sessions. Contributions can be marked by lecturers and peer assessed by students

7. Choice- enables a teacher to ask a single question and offer a selection of possible responses. A bit like a single question MCQ. Can be used for formative assessment in order to decide the direction of teaching for individuals and groups

8. Checklist– in Moodle - of activities for students and staff – can be good for self-assessment and peer assessment and teachers can comment on students work and it can be linked to the marking managing system (Gradebook in Moodle)

9. Database – in Moodle - The database activity module enables participants to create, maintain and search a collection of entries (i.e. records). The structure of the entries is defined by the teacher as a number of fields. Field types include checkbox, radio buttons, dropdown menu, text area, URL, picture and uploaded file. The teacher can grade student work in this tool and have it recorded in the Gradebook system. This can be a useful alternative to a ‘traditional’ online essay submission box, but make sure you use it for the database features. This can be a very useful tool to create learning resources from students work for future use. The fields the teacher creates for the student to fill in / upload can support and guide the students in their activities. The tool can support peer assessment by allowing students to grade each other’s work.

10. Feedback – a restricted version of objective / MCQ style test that can be inserted anywhere in a course with multiple choice, yes/no or text input. It is linked to the Gradebook system. It’s really intended for use by teachers to get feedback from students about the course (as the name implies), so is not for peer assessment. It can be used for diagnostic and formative assessment as well

11. Forum / Discussion Board - The forum activity module enables participants to have asynchronous discussions i.e. discussions that take place over an extended period of time. A teacher can allow files to be attached to forum posts. Attached images are displayed in the forum post. Forum posts can be rated by teachers or students (peer evaluation). Ratings can be aggregated to form a final grade, which is recorded in the Gradebook.

12. Glossary - The glossary activity module enables participants to create and maintain a list of definitions, like a dictionary, or to collect and organise resources or information. A teacher can allow files to be attached to glossary entries. Attached images are displayed in the entry. Entries can be searched or browsed alphabetically or by category, date or author. Entries can be approved by default or require approval by a teacher before they are viewable by everyone. A teacher can allow comments on entries. Entries can also be rated by teachers or students (peer evaluation). Ratings can be aggregated to form a final grade, which is recorded in the Gradebook.

13. Lesson - The lesson activity module enables a teacher to deliver content and/or practice activities in interesting and flexible ways. A teacher can use the lesson to create a linear set of content pages or instructional activities that offer a variety of paths or options for the learner. In either case, teachers can choose to increase engagement and ensure understanding by including a variety of questions, such as multiple choice, matching and short answer. Depending on the student's choice of answer and how the teacher develops the lesson, students may progress to the next page, be taken back to a previous page or redirected down a different path entirely. Student performance in a lesson may be graded, with the grade recorded in the Gradebook.

14. Questionnaire / Survey – This tool construct surveys using a variety of question types, for the purpose of gathering data from users. It is not linked to the Gradebook – typically it is used for end of course evaluations. You can export the response data in the CSV/Excel format and this can be useful to generate report visuals. This can be useful for diagnostic information and formative evaluation that can be shown to the student cohort to give them an overall sense of progress and the range of learning that is being achieved

15. Quiz / Objective / MCQ style tests – The quiz activity enables a teacher to create quizzes comprising questions of various types, including multiple choice, matching (can be graphical drag and drop), short-answer and numerical interactive multiple-choice, jumbled-sentence, crossword, ordering and gap-fill exercises. The teacher can allow the quiz to be attempted multiple times (or just once), with the questions shuffled or randomly selected from the question bank. A time limit may be set. Each attempt is marked automatically, with the exception of essay free text style questions, and the grade is recorded in the Gradebook. The teacher can choose when and if; hints, feedback and correct answers are shown to students.

16. SCORM Package – A SCORM package is a collection of files, which are packaged according to an agreed technical standard for ‘learning objects’. Commercial e-learning training providers often produce SCORM packages as these as these will run in different online systems. The SCORM activity module enables SCORM packages to be uploaded as a zip file and added to a course. SCORM content produced by commercial and industrial training providers can be a good way of getting ‘industry standard’ learning resources and assessments into your VLE.  Content is usually displayed over several pages, with navigation between the pages. There are various options for displaying content in a pop-up window, with a table of contents, with navigation buttons etc. SCORM activities generally include questions, with grades being recorded in the Gradebook. SCORM activities may be used to present multimedia content and animations and as an assessment tool

17. Survey – Similar to the Questionnaire, the survey activity module provides a number of verified survey instruments that have been found useful in assessing and stimulating learning in online environments. A teacher can use these to gather data from their students that will help them learn about their class and reflect on their own teaching. Note that these survey tools are pre-populated with questions. Teachers who wish to create their own survey should use the feedback activity module.

18. Workshop -Workshops are an activity designed to allow peer assessment and potentially a very powerful one. Students submit work into the assessment. The work is allocated to their peers and they are allocated someone else's work to assess. The students grade the work they are allocated. The lecturer can then review the marks and correct any they feel are too far out. The tool has a handy workshop planner that displays all phases of the activity and lists the tasks for each phase. The current phase is highlighted and task completion is indicated with a tick. The workshop activity module enables the collection, review and peer assessment of students' work. Students can submit any digital content (files), such as word-processed documents or spreadsheets and can also type text directly into a field using the text editor. Submissions are assessed using a multi-criteria assessment form defined by the teacher. The process of peer assessment and understanding the assessment form can be practiced in advance with example submissions provided by the teacher, together with a reference assessment. Students are given the opportunity to assess one or more of their peers' submissions. Submissions and reviewers may be anonymous if required.

Checklist for E-Portfolio e-Assessment Tools

1. Unlike VLEs, which are teacher owned and controlled, the e-portfolio is student owned and controlled, but still usually provided by the college. The student uses the e-Portfolio to collect, organize, present and share digital content they have either created or collected. As well as uploading files they can create web pages and complete journals and blogs detailing their work and reflecting upon it. As we point out in the Tips section above the web formatting tools may be a bit flaky – so check them out first. Students will also create an online profile of themselves that provide their online ‘social media’ identity inside the portfolio system. The teacher uses the e-portfolio system to set up spaces for the students to share their work and to collaborate and use the internal system social media tools. Another important factor to consider and remind your students of at the start is that access to the e-Portfolio usually ends after the student leaves the college – so they need to know how to move their digital materials out of the system if they want to keep it (NB it is best to export in both the formats available – html and LEAP2A). In general, the interest in and use of e-Portfolios is developing rapidly.

2. e-Portfolio systems usually include ‘internal ‘blogging and social media tools and discussion forums, so that students can comment on and discuss each other’s work – inside the system not on the open web. They include the ability for students to develop and keep their own learning plans and assessment exercises (sometimes these will be pre-created by teachers in the form of various ‘templates’). These plans and templates can then be populated with student-generated content (essays, reports, photos, audio, video, weblinks etc.) that provide evidence of the learning involved.

3. Teachers can create groups for students to work in and they can also set up spaces into which students can submit their work into for assessment, these groups will usually be associated with a college course. When under assessment in this way the content can be ‘frozen’ until the assessment is complete then released when the assessment is over. The teacher can provide feedback to each student in the e-Portfolio system. In the Mahara system there is a basic 5-star rating system for grading when users (including teachers) add comments on student work, but this is unlikely to be enough for academic marking needs.

4. For more detailed and complex grading, this can be carried out by using the VLE grading system to mark e-Portfolio content. This is done by setting up an assignment submission in the VLE for the e-portfolio activity and adding the marks and feedback there. The VLE submission can be hidden from students until the marks and feedback are ready to be released. Hiding it in this way will avoid confusion between the two parts of the system. In some institutional systems the VLE and e-Portfolio are more closely integrated and it is possible to set up an assignment in the VLE that is directly linked to the related content in the e-Portfolio

Checklist for Classroom / Lecture Voting Systems

1. These have been around for some time and can be used in a number of ways. They are popular for rapid diagnostic and formative assessment in a classroom or lecture theatre, with the option to have the results projected onto a video screen – usually anonymised. This can be good to quickly assess where a cohort of students are in their progress towards the course learning outcomes. It can also provide a motivational ‘reality check’ for the students to see where they really are in connection with the learning outcomes of the course.

2. With a voting system, each student gets a wireless handset and can feedback views, answers or data to a radio receiver and software on a computer. Simple systems are a bit like the audience poll on television quiz shows. More sophisticated versions allow the user to type free text and numbers. The software records the students’ responses and the stored data can produce and format reports, graphs and marking sheets. The teacher creates a quiz or set of questions using the software provided by the system, or in some cases it can be imported. The teacher runs the quiz in class and the students answer using the handset usually choosing from several options. Recent innovations include systems that use ‘apps’ on tablets and mobile phones. These take a bit of setting up and of course the students need to have these on their own devices or be provided with the devices.

Checklist for Social Media e-Assessment tools

1. When you use social media tools for learning you are stepping out of the closed environment of the college – hence our phrase ‘Leaving the Reservation’. Because most educational use of social media involves using the ‘free’ service options many lecturers and students tend to think of them as ‘natural’ and benign features of the internet environment. But, of course, they are all commercial enterprises and not public services. They make their money by buying and selling information and some of that information is personal – very personal – so it pays to think about the possible statutory and legal implications first. If your college has a policy on this (you should check) then consult it.

2. None of the participants in our project used social media tools (they concentrated on college systems). But we do know of some uses of these tools, for instance the creation of video ‘blogs’ using YouTube to upload and host videos showing the activity and outputs of student work. Other examples are the use of Google Drive to enable students to easily collaborate on co-authoring a document for assessment, the use of Dropbox for students to upload their videos to for lecturers to access, and Slideshare for students to upload their presentation to for lecturers to access.

3. The use of social media tools can provide big benefits in utility, speed, usability and convenience compared to college systems. There are a number of legal considerations to take into account when contemplating using these services.

4. There are a number of important legal and statutory considerations to take into account when contemplating using social media for assessment purposes: Data Protection, Privacy, Inclusion, Discrimination, Defamation, Harassment, and Copyright are some of the more obvious issues. A useful introduction to legal issues has been produced by Jisc in a blog posing entitled ‘Digital skills and values to keep you safe online’ with lots of useful information and links. Jisc also have a handy Social Media for Staff Legal Checklist as well as an online guide to the subject

5. If you are using social media for your assessment you should record this in IV/EV documentation. You should make sure the SQA EV can easily access the evidence produced by the students. You should take steps to make sure that your feedback and marks to your your individual students remains secure and private.