4 Designing a SoTL inquiry
In this section, a framework is presented with seven criteria that can be used for the design and evaluation of a SoTL inquiry.
Glassick et al. (1997) developed an evaluation framework that has six guiding principles or ‘yardsticks of excellence’ or criteria: clear goals, adequate preparation, appropriate methods, outstanding results, effective communication and a reflective critique. As you have seen earlier in the course, sharing the results of a SoTL inquiry is the distinguishing characteristic of SoTL as compared to other forms of reflective academic practice. Therefore, a seventh criteria in the framework called ‘going public’ has been added. Questions against each of the seven criteria have been influenced by Glassick et al. (1997), Glassick (2000) and Huber and Hutchings (2005).
The framework with seven criteria and the corresponding questions is presented in the table below (Table 1). The framework can be used as a checklist for designing a SoTL inquiry. The criteria will enable you to think through why you are engaging in the project, why it is important, what you hope to achieve, what evidence you will collect that would demonstrate that your inquiry has been a success and whether the SoTL inquiry will result in or has resulted in reflections and outcomes that will be/are publicly shared for peer review and for others to build upon.
| Criteria | Questions |
Clear goals |
Are the basic purposes of the SoTL inquiry and the aim clearly stated? Are the stated research questions realistic and achievable? Are important issues in the field (i.e. SoTL practice or discipline) identified? Does the inquiry address the strategic priorities in the funding call? (in case the SoTL inquiry is being developed in response to a funding call) Is the inquiry aligned with the institution’s learning and teaching vision and plan? Does the inquiry address the strategic priorities in the department’s and/or institution’s scholarship strategy? |
Adequate preparation |
Do the materials such as literature review demonstrate understanding of existing scholarship in the field? Does the individual educator/team have the necessary skills for the inquiry? Does the educator/team bring together the resources necessary to move the project forward? |
Appropriate methods |
Are the (research) methods appropriate to the goals? Does the educator/team know how to apply the methods selected? Will the educator/team be able to modify procedures in response to changing circumstances? |
Significant results |
How feasible is it that the inquiry will achieve the expected goals? How will this SoTL inquiry add consequentially to the field? Will this inquiry open additional areas for further exploration? |
Effective communication |
Does the educator/team use a suitable style and effective organisation to present their work? Has the educator/team identified appropriate forums for communicating work to its intended audiences? Does the educator/team present their message with clarity and integrity? |
Reflective critique |
How will the educator/team critically evaluate their own work? Will the educator/team bring an appropriate breadth of evidence to self-critique? Will the educator/team use evaluation to improve the quality of current and future work? |
Going public |
Does the educator/team have a strategy to disseminate the outcomes? Does the dissemination strategy involve peer review? If so, what is the nature of that peer review? Will the outcomes be usable by other educators, disciplines and institutions? How does the inquiry relate to the wider and external context of SoTL? |
You may like to use the framework to self-evaluate your project proposal ahead of bid submission, or you may like to use the framework to self-evaluate your inquiry when it’s in progress, or at the end.
You will learn about planning for impact of a SoTL inquiry, or evaluating a SoTL inquiry for impact in Session 6. You may consider guidance for impact in Session 6 in conjunction with these seven criteria when planning or evaluating a SoTL inquiry.
At The Open University, and in other institutions, educators and other colleagues (e.g. learning designers, developers) associated with teaching can apply for institutional funding for conducting a SoTL inquiry. The bid for funding may include resources such as for a research assistant, transcription of interviews, attendance at a conference, etc. The bid for funding SoTL generally goes through a peer-review process. Once the project is funded, there are peer-review processes to assess if the inquiry is proceeding as planned, and at the end of the inquiry to assess whether/how the SoTL inquiry has met its objectives.
The framework in the table can be used for peer-evaluation of the inquiry by funding bodies, mentors and collaborators.
The seven-criteria framework in Table 1 can, therefore, help in several situations of peer- and self-evaluation of SoTL.