Methods for data production and collection

A range of methods need to be used for producing the data necessary to address some of the questions delineated above. Variously, data: will need to be created, may exist and be sufficient, or may exist but be insufficient.

Quantitative data 

Quantitative data will probably already be available for a range of education-related areas at a national or international level - for example, the data and associated databases generated by International Association for the Evaluation of Educational Achievement  (IEA) and Progress in International Reading Literacy Study (PIRLS) and within national government reports and records. It will also exist at the institutional level where it will have emerged from standard record-keeping procedures.  

The wider the scope of the data base, the greater the danger of it having been established for purposes other than that of evaluating the reach of TPD or considering matters of equity.  Thus data bases need to be interrogated for their efficacy in addressing questions of equity relevant to context.  For example, are relevant data about specific minority or vulnerable groups separated out?  Do they give overall figures or show distribution? How much is relevant for TPD and teaching related activities - eg data which have implications for equity in relation to accessing the curriculum, assessment outcomes, school achievement and/or appropriate pedagogies and practices.

Data bases at the classroom level, on the other hand, are likely to be of more use to individual teachers but are often not used to best effect and the messages about issues of equity not extracted.  For example, such data might yield evidence that some teachers had been able to raise standards of achievement by implementation of a particular teaching technique but that others had not because they had not implemented this technique.  Staff need to have both the ability and the time to interpret data, identify the questions that emerge from them, consider how these questions are best answered and by whom (ie at what level) and what subsequent action needs to be taken.

Qualitative data

These will mostly deal with processes, perceptions and lived experiences around professional development and engaging with learners.  They will emerge in the interpersonal plane and will depend on personal narratives offered in interviews/focus groups/feedback sessions – any space in which participants in professional development can comment on the way in which the initiatives have, or have not, changed their behaviour or beliefs with regard to their pedagogy.  Data collection needs to include the voice of all those involved both in the TPD activity (the teachers, event facilitators), any support staff whose work will be affected by any changes in teacher action (eg classroom assistants), and those who are experiencing changes in teaching approach (eg students, other professional colleagues).

Complementarity of qualitative and quantitative data

While there is some distinction between these two types of data, they can be regarded as complementary rather than producing discrete bases for decision-making.  Any measurement (quantitative data) needs to be interpreted, and this interpretation will be influenced by context, which will render interpretation complex.  Any inference from quantitative data will be affected by psychometric, sociocultural and situated approaches (Ercikan and Roth, 2006).  Qualitative data can problematise quantitative data by interrogating it and seeking evidence about cause of consequence.  It is, thus, critical that a collaborative approach is taken to the evaluation of TPD so that policy-makers, legislators and practitioners can work together both to identify strong practices and ameliorate weaker practices.