Skip to main content
Print this chapterPrint this chapter

Analyse: Readings

1. Understanding your own context – prompts for analysis

Your context may just involve you as an individual, a department, a faculty or the whole college. These questions are prompts to help you develop your own picture – it will likely change as you continue working in this area .

Your Students

What are their characteristics –in general? Are they ready for some independent learning? Or do they expect to be closely supported? Will this affect your assessment planning? Are they able to use the college systems effectively? Remember students are one of the most under - utilised resources in education. The REAP project contains useful guidance on involving them in assessment practice – such as peer assessment and peer teaching. Remember to think about assessment for learning not just measuring it.

Subject area

Does your subject area have any characteristics that make it more or less likely for you to imagine using e - assessment? Often ideas come after a period of thinking about it – ideally talk to others. For instance at Glasgow the use of e - portfolios for assessment is growing in areas like construction trades. If your subject requires students to write reports and essays then an online submission is a natural progression from paper essays. The use of objective tests / MCQs has wide potential for application – but does require some thought and experimentation and of course considerable up - front investment of your time. Check out the project case studies for examples of solutions people have developed – what worked and what didn’t.

Are there any existing problem areas in your current assessment practice that you would like to use technology to improve, for instance, a high marking load or late feedback to students? The assessment design template that we introduce in the design section of this toolkit can help you record and share your ideas in a simple and consistent way – and you can customise it to suit your own needs.

Teaching Culture

What are the attitudes and values of the lecturers that you work with? As the Jisc guide to Networked Learning observes, the introduction of technology can highlight personal ideas, values and philosophy about teaching and learning in quite unexpected ways (see the illustration from the Jisc guide in section 1 ‘Getting Stated’ about contextual factors). Here’s a real example from a workshop we attended at one of the partner colleges:

“We took a long hard look at ourselves and our teaching. We realised that we had become stale and that we were teaching on the same programme as each other but in isolation – in our own little silos. We were teaching theory first then doing the practical work so the students had no context for the theory. We weren’t happy and neither were our students.

We decided to change the way we worked. Instead of teaching in this disjointed, way we worked together to redesign the curriculum. We moved from teaching by numbers to teaching through projects. This meant changing everything, especially the assessment as we had now merged 4 units together and were doing the assessment for them through the project work. This was much better, the theory was taught in a practical context and could be applied immediately. The students saw the point of the theory and did not have to wait weeks to use the theory they had been taught previously in an abstract manner. This meant getting the unit re-verified. The result? Students are much happier and are getting excellent results and the staff are happier too.”

Technology

Not surprisingly technology is a major factor in the successful use of - assessment. So, as we say elsewhere, your No.1 priority is to find out the technology you college has and learn how to use it. It is especially important to do this early on in the process and not leave it to the last minute. If you are using a technology that only works well in certain web browsers (as is the case with many of the VLEs) you need to know what those web browsers are and if they are supported in your college. You must make no assumptions in these matters and you must find out for yourself and test the technologies regularly. Obviously this is a lot easier if you are working as part of a collaborative group or team and have some technical support.

Central IT services often have policies restricting what technologies that they will allow on college machines and offer support for, like browsers and plugins and versions of programmes such as Microsoft Office, etc.This is why it is wise to start with a pilot exercise that targets formative assessments in order to find out about your local technology and administrative context. Try to find out when upgrades and changes to the IT systems are planned by the college, if there is no policy of communicating this kind of information routinely ask your IT Dept. Make a point of telling your college IT service when your assessments are scheduled and ask them to alert you to any changes during that period. It obviously makes sense to cultivate good relations with your IT department and find someone you can talk to there. Many central IT departments are still coming to terms with e - learning technology as being part of their support remit and actual arrangements on the ground may still be under negotiation. If things do go wrong due to unannounced systems change etc. having a clear electronic ‘paper trail’ of consultation and notification will help make clear where responsibility lies.

As we indicate elsewhere, it is important to find out what your students skills levels are in relation to using college systems (as are those of your teaching colleagues) and take any remedial measures early on.

Learning Technology Support

Leading on from the previous section if you have access to learning technologists you can ask for their support. This can be especially important during setting up and testing an assessment. Aim to make yourself self - sufficient over time.

Administration systems

Your college administrative systems maybe a mixture of different paper and electronic systems and you need to see how this affects your e - assessments over their whole lifecycle. Systems of paper assessments have well - established processes that do not require much engagement on your part. But with e - assessments you are much more likely to need to need to be able to trace the flow of information and be prepared to take steps to intervene. Again this is a good reason for doing pilot exercises to iron these things out.

Quality Systems

In Scotland this will centre on the internal and external verification procedures in relation to SQA qualifications. Your college will have established internal verification (IV) procedures that are there to manage and account for any changes to teaching and assessment. Often this will primarily take the form of paper - based records, although increasingly colleges are moving to using online systems that include simple shared network drive folders or tools like SharePoint, Drupal or even ‘private’ areas within the VLE system that are used solely for administrative functions.

The crucial thing is to record your reasons for changing an assessment and to indicate where and how verifiers can see and examine the evidence of learning produced by your students. We have produced a simple and adaptable design template that should be able to help with this process. Before we leave this area we suggest that it is really worthwhile to agree a naming convention for common items such as module, unit, programme qualification etc. as well as test, mcq assignment, dropboxes etc. Develop a common structure and layout for online learning resources in the VLE and the location of assessments. It is worth thinking about having an agreed glossary for these terms and to get staff to stick to it and publish it in the VLE for the students to refer to as well. It is also worth thinking about having the assessments for a unit in a VLE located consistently in the same place in the online course (it is confusing for lecturers and students alike when they appear all over the place!). These all seem like small things but together then can make yourstudents (and your own) experience much easier.

Institutional Factors Mini Checklist

These factors are much more general and in some cases intangible but can also be the most important, here are some things to consider:

  1. Strategy: is there a clear and agreed strategy for the use of learning technology and e-assessment? Is there an implementation plan? (it is not uncommon to have a strategy but no plan for implementing it). Is progress monitored / audited? Are there resources allocated to support this?

  2. Academic leadership: We shall be picking up this theme in our Collaborative Frameworks section later. Is there a clear ownership of pedagogic and educational matters at the college or is it scattered across several units? Is there a unit “indicate where and how verifiers can see and examine the evidence” that deals with this and is it led be a senior teaching academic?

  3. Morale: What is the staff morale like? This can have a big bearing on the appetite for change and openness to trying new things.

  4. Support: Is there adequate resources to support for staff in e -learning / e -assessment? In terms of IT infrastructure and learning technologists. Is there access to training and development inside and outside the college?

  5. Finances: The state of the college finances will have a major impact on the other factors and on planning, especially on staffing levels for teaching, learning technology support and equipment and IT infrastructure.