1 Learning analytics
There are a number of different methods that we can adopt to evaluate online teaching. Oliver (2000) provides a detailed overview of some of these methods, including:
- online synchronous focus groups (Cousin and Deepwell, 1998)
- web-based questionnaires (Phelps and Reynolds, 1998; Taylor et al., 2000)
- creation of an online feedback discussion area (Taylor et al., 2000).
Oliver goes on to summarise the general difficulties with the above methods, which can be grouped into two main themes: the processes are largely ‘uncontrolled’ and feedback can be unfocused or anonymous; and methods for evaluating this kind of data are still developing, with different approaches leading to different conclusions.
Learning analytics offers an alternative to these methods of gathering feedback and reports from learners. These approaches instead make use of the data left by learners and teachers as they act: their ‘trace data’. These can tell us when learners join courses, when and how they engage with online activities, view pages, borrow resources from the library, set or complete activities or assessments, and so on. Any interaction with a web-based system can be tracked, and this data could be used to better understand what learners and teachers do. The widespread use of virtual learning environments (VLEs) – also known as learning management systems (LMSs) – has meant that educational institutions now deal with increasingly large sets of data. Each day their systems gather more personal data, systems information and academic records.
Learning analytics is a field of innovative research, but it is increasingly something that many educators and institutions make use of through new tools, dashboards and reports, using online data to investigate user activity. It helps to answer questions such as:
- How many people visit the website / online learning materials?
- When do they visit / interact?
- Which links are popular?
- How many people complete the activities?
Answering the questions posed above could involve analysing large data sets from VLEs and other technologies used for learning. Learning analytics can go one step further by providing actionable insights – they take trace data from educational settings and suggest, prompt or initiate actions to improve learning and teaching. You may have heard the term ‘big data’ used in discussions of technology. It is used in a lot of different ways, but essentially means that the dataset is very large and also very complex. Because of this, it may not be possible to use a simple, traditional approach to data processing and analysis. Learning analytics of the behaviours of large numbers of students can easily fall into the category of big data. But equally, you might look at the behaviour of one class of students over a course and find that useful insights can be gained without advanced techniques and tools.
For example, in an online forum discussion associated with a particular online module or course, a VLE could capture a range of forum data, including:
- who accessed the forum
- when they did this
- how long they stayed
- what operating system they were using
- how many words they added.
Any of these data could be used to create analytics. However, only some of these analytics would be useful to teachers. It is not possible to identify which analytics will be most useful without knowing something about how the forum is being used. The presence of a learning design should identify the purpose of the forum in relation to learning outcomes. This makes it easier to decide which analytics to use.
Activity 1 What can we learn from learning analytics?
Professor Bart Rienties of the Institute of Educational Technology at The Open University has played a leading role in research and practice around learning analytics. Here he ison the subject in January 2018. His talk introduces some of the findings from learning analytics research at the OU, and some of the ways in which this provides insights for our teaching.
Watch the video, and as you do so, make notes about what kinds of learning analytics you would like access to regarding your own teaching. When moving your teaching online, are there any of these kinds of learning analytics that you could begin to collect? How might you do this?
For some teachers, working in an organisation may provide them with access to certain data from learning analytics. Conversely, this kind of data may not be routinely collected, or not routinely shared with teachers. This activity should help you to think about what you currently have available, and what you could gather when teaching online. The video shows some of what is possible, but also that there is a lot more potential to use learning analytics than is currently mainstream practice, particularly if we improve our abilities for data collection and analysis.
If you are interested in finding our more about learning analytics, you may like to read Ferguson (2012) and Long and Siemens (2011) as well as Jisc’s Code of Practice for Learning Analytics (Sclater and Bailey, 2015).