Skip to content
Skip to main content

About this free course

Author

Become an OU student

Share this free course

Scholarship of Teaching and Learning in STEM
Scholarship of Teaching and Learning in STEM

Start this free course now. Just create an account and sign in. Enrol and complete the course for a free statement of participation or digital badge if available.

3 Data collection, preparation, analysis and interpretation

The research process involves data collection, preparation, analysis and interpretation, and reporting the findings of the SoTL inquiry.

Based on the research design, the process of data collection will require preparation; for example, preparing interview protocols to guide the interview process, designing an online survey preparing for observations. Depending upon the choice of methods in your research design, you may require audio or video recording equipment, or an online survey tool. Before the data collection, you may decide to have a pilot run of your materials with participants who are representative of your research participants. This will help to refine the materials and procedures of data collection ahead of the actual data collection process.

Once you have gathered all of the data you need, you will have a mass of data: quantitative, qualitative or a combination of both. If you have conducted survey research, you may have a large number of responses to a questionnaire that now need analysing. If you have conducted interviews, the written transcripts may be lengthy. You may have notes made during observations, or perhaps a pile of reflective journals of students associated with case study research as a part of your SoTL inquiry. It will be helpful to prepare this data ahead of data analysis.

Data preparation consists of organising the data you have gathered in such a way that you can readily examine it and then make relevant observations and comparisons. In practice, and irrespective of what sort of data you are dealing with, preparation needs to achieve the following:

  • a.bringing all your data together so that it is readily accessible, which can be challenging if your data is in different formats. You may like to use NVivo [Tip: hold Ctrl and click a link to open it in a new tab. (Hide tip)] for data consolidation and analysis
  • b.creating a plan of what you will examine first and why.

There may be a case to be made for preparing an initial overview of the data as a whole, before scrutinising the data more closely and carefully.

Guided by the research question(s), data analysis is a process by which large amounts of sometimes complex data are distilled to highlight that which is true, essential or important within the data. For example:

  • searching for recurring statements or points made within the data; recurring themes are considered significant because they either suggest that something is widely and commonly experienced or that many people think it is important. Thematic analysis is a popular method for analysing qualitative data that focuses on identifying themes (patterns of meaning) in qualitative data (Braun and Clarke, 2006)
  • identifying features in the data that are given special emphasis by participants.

Having analysed the data, you will then interpret the data. Interpretation is all about answering the ‘So what?’ question, based on the results of the research.

You will find resources at the end of this session, which will help you to find detailed guidance for planning the research design and conducting the research process.

Activity 8 Development and evaluation of a software tool for automated Java specification marking

Timing: Allow approximately 10 minutes

This 5 minute video features Anton Dil of the School of Computing and Communications in the Faculty of STEM at The Open University discussing his eSTEeM-funded SoTL inquiry involving students and tutors on a programming module.

As you watch this video, make notes on the choice of research methods, reflections on the outcomes and the use of the literature. Anton refers to TMA in the video. A TMA is a tutor-marked assignment.

For a better viewing experience, you may like to view the video in full screen mode.

Download this video clip.Video player: Video 6
Copy this transcript to the clipboard
Print this transcript
Show transcript|Hide transcript
Video 6 Development and evaluation of a software tool for automated Java specification marking
Interactive feature not available in single page view (see it in standard view).
To use this interactive functionality a free OU account is required. Sign in or register.
Interactive feature not available in single page view (see it in standard view).

Discussion

In a second-year object-oriented programming module on an undergraduate programme, students submit solutions to programming assignments on paper. The tutors mark the paper versions of the solutions as these are easier to annotate and provide feedback on. Anton’s project involved developing software support for checking students' solutions in order to support tutors in locating errors in students’ code, and for producing feedback for students so that they could work towards a software solution in stages, based on automated feedback.

Anton developed two software tools. His evaluations involved running a survey and conducting interviews to collect tutor feedback. He analysed postings in the discussion forum to understand how students were using the tool and to learn about their experiences. Based on his evaluations, he conducted a literature review to enhance his understanding on technology acceptance and, although the tools are in operation, he is continuing to work on improving the experience for the students and tutors.

Details of Anton’s eSTEeM-funded SoTL project and related resources are available on the project webpage: Development and evaluation of a software tool for automated Java specification marking.

In the next section, you will consider some guidance on presenting the research findings of your SoTL inquiry.