Analyse - Tips and Checklist

1. Analyse Tips

  1. Make sure you and your colleagues are really familiar with the technologies you are planning to use – you need to aim to be as self - sufficient as you can be and only need minimal support from central services (they are under pressure to!). Set yourselves targets of what you need to learn to deliver a particular e-assessment and then do a ‘test run’ using a test student account – so you see what the students will see.

  2. Explain what you want to do to your central support services and ask for their help early on –do not leave it to the last minute. It is wise to plan 6/12 months ahead of going live for a new e-assessment. Discuss your ideas early on with any learning technologists or support staff you have access to. They can help clarify your ideas and let you know how they can help. Look out for any staff development events or drop-in sessions to get ideas and find out what others are doing, remember to look beyond your college as well[1]

  3. Make sure you are familiar with the way your system (the VLE, e-Porfolio etc.) records and manages marks – they can be a bit tricky. Know how to export the marks in different electronic formats so they can be imported into the student records system (to avoid the risks of manual re-entry or the inconvenience of conversion back to paper)

  4. One common problem with recurring e-assessments (like essay submissions or online tests) is that due to the large time intervals between them lecturers often forget how to reset them in the system - using the controls for dates, times and access conditions etc. A good tip is to create (ideally this is a central service job) a help guide that sets out the steps needed in detail and make this available online. Another good tip is that each lecturer and department should work collaboratively to develop a ‘preflight checklist’ of things to do in relation to maintaining their e - assessments at the start of each term – ideally in coordination with any central support service

  5. Are your students ready? Do not assume they have the skills to use the college online systems / equipment needed to access your online learning resources and activities or your e - assessment resources. Ask them and use the UHI skills checklists (see the Downloads section of project website) to assess student digital literacy to use college systems

  6. Remember that the type of digital literacy required of your students will mean knowing how to use college systems that are often quite complex,‘clunky’ and old fashioned relative to what students are used to in social media.To be fair to college systems and educational software generally – they are doing a very different job to the well funded and developed commercial social media products.So, being a whizz on Facebook does not mean being any good at using a college VLE etc. Don’t believe the hype that all youth are automatically tech[2] experts! This article about American students tech skills does quite a good job at busting this particularly pernicious stereotype of young people.

  7. Be aware that often college VLE or e-Portfolio tools will have limited functionality and display differently on mobile devices – find out what yours look like on Android and Apple tablets and phones (involve your local IT / Learning Technology Department). Do this early on.

  8. One of the first useful e-assessments tasks you can do is to set up a simple MCQ diagnostic test in a college VLE to assess if students have to skills to use college software (VLE, e-Portfolio etc.) and what devices they use to access content outside college and how they access the internet. This provides a useful baseline for your planning, it is well worth suggesting this is incorporated into standard induction procedures.

  9. If you are planning to conduct summative e-assessments that require invigilation (also known as proctoring in the USA and elsewhere) you are likely to need access to college facilities (e.g. classrooms with computers). You need to arrange access early on and plan your invigilation arrangements. You must do a test run (it can be short) with your students beforehand to familiarise them with the system they will be using. Some colleges have set up purpose built and equipped e-assessment centres [3]

  10. Make sure you are familiar with the internal quality management system at your college – in the Scottish system this is usually called ‘Internal Verification’. This records and examines any changes to the courses – especially assessment. So make sure you record these changes and get them approved, think about using the design template introduced in the next section of this toolkit.

  11. Prepare for the external quality management procedures that your college is subject to. In the Scottish system this is called ‘External Verification’ and is carried out by subject experts appointed by the SQA. Again, think about using the design template introduced in the next section of this toolkit.

  12. If you are using social media or other commercial non - college services in connection with e -assessment (or indeed learning in general) you will need to consider your own personal and employer legal responsibilities in relation to data protection, privacy, copyright and child protection etc. You will find some useful information about this in the Design section of this guide under the heading entitled ' Checklist for Social Media e-Assessment tools –Leaving the Reservation’

  13. Make sure you develop an understanding of the bigger picture in your college (your context) and how other factors will impact on your work.

  14. When you are thinking about developing an e - assessment it makes sense to target an area that will return real benefits (not some marginal case) – so think in terms of reaching large number of students, or making off - campus submissions possible and reducing existing problems and bottlenecks (such as marking loads and late feedback to students). Start with a formative assessment exercise to iron out problems before moving on to any summative high-stakes assessment.

  15. Training 1:If you are providing training to teachers in the use of the in - house college systems (VLE, e-Portfolio etc.) be aware that sometimes the poor usability of aspects of these systems can cause stress and a lack of confidence and a consequent loss of engagement and motivation (this is true of students also). To counter this, you need to make sure that you are fully competent in your own use of the systems. Provide detailed step - by - step help guides for the teachers to use under their ‘own steam’, the ones provided by UCL for Moodle are excellent, the official Moodle documentation site is also a must. In addition Moodle has its own YouTube Channel and a collection of online training videos that present training in short ‘chunks’ about aspectsof the system.

  16. Training 2:Do not assume basic IT competence when providing training to staff (the same is true of students), do make sure you start with a basic check of the competences needed to undertake the training task in hand. You can then remediate / alter your training to fit. Going slow at the start like this lays the foundation for effective training; access to detailed guides already described helps the teacher become more independent.Manage teacher’s expectations at the start and stress the need to get the basics right and their own responsibilities to become adept with the systems.

[1]The College Development Network, Jisc, ALT and the various user groups all hold events – see the Further Information section