4.3 Frameworks that can help ensure rigour

To help researchers maintain quality and transparency, specific frameworks for conducting and reporting mixed methods research have been developed. Two such frameworks are presented and explained below.

The GRAMMS framework

The Good Reporting of a Mixed Methods Study (GRAMMS) framework, developed by O’Cathain, Murphy and Nicholl (2008), provides six criteria for rigorous reporting where researchers describe:

  • the justification for using a mixed methods approach to the research question
  • the design in terms of the purpose, priority and sequence of methods
  • each method in terms of sampling, data collection and analysis
  • where integration has occurred, how it has occurred and who participated in it
  • any limitation of one method associated with the presence of the other method
  • any insights gained from mixing or integrating methods.

The rigorous mixed methods (RMM) framework

Developed by Harrison, Reilly and Creswell (2020), this coding scheme identifies four primary or core elements that must be transparently reported:

  • Data collection: You must detail the specific collection procedures for both the quantitative and qualitative strands, including sampling procedures, instruments used and the specific types of data collected.
  • Data analysis: Procedures for analysing both data strands need to be thoroughly reported, ranging from descriptive or inferential statistical approaches for the quantitative side to qualitative coding and thematic development for the qualitative side.
  • Data integration: You must clearly describe how the two strands are linked, merged or build upon one another depending on your design, including the timing of the strands (concurrent or sequential) and the points across phases at which integration occurs. High rigour in this category involves clearly explaining how integration affects the overall study and often uses joint displays or direct data comparisons.
  • Mixed methods design type: The study must explicitly name the specific mixed methods design being used (such as ‘sequential explanatory’), clarify whether the strands carry equal weight or one is dominant, and ideally include a diagram to illustrate the design.

In addition to these core elements the RMM framework evaluates advanced procedures related to how the study is framed and written:

  • Aims and purpose: A highly rigorous study includes a specific mixed methods research question, a clear rationale for using a mixed approach and a discussion of the unique value that mixing methods brings to the research.
  • Elements of writing: Researchers should use appropriate language and terminology, cite relevant mixed methods literature, and explicitly identify the study as ‘mixed methods’ in the title, abstract and throughout the paper.

The following table summarises what high, medium and low levels of rigour typically look like across the four core RMM elements (drawing on the categories developed by Harrison, Reilly and Creswell, 2020).

ElementHigh rigourMedium rigourLow rigour
Data collectionBoth strands fully described, including sampling logic, instruments and procedures, with rationales linked back to the research questionsOne strand fully described and the other partially described; some procedural detail missingOne or both strands described only superficially; sampling or instruments not justified
Data analysisAnalytic procedures for each strand are clearly reported, including coding approach, statistical tests and how decisions were madeAnalytic procedures partially reported; some decisions visible but rationale thinAnalytical procedures unclear; the reader cannot see how findings were produced from the data
Data integrationIntegration is explicit, specifying timing (concurrent/sequential), the stage(s) at which it occurs and how strands inform each other; joint displays or direct comparisons are usedSome integration described but stages or strategies remain implicitIntegration claimed but not actually demonstrated; strands sit in parallel
Mixed methods design typeDesign is explicitly named (e.g. ‘explanatory sequential’), priority of strands is stated and a visual or notation accompanies itDesign is named but priority or visual representation is missingDesign type is not stated or is stated inconsistently

You can use this summary as a self-check when designing or reporting your own mixed methods study.

4.2 Managing conflicting findings

4.4 Sampling and integration challenges