Studying generalisability

So you think your research has potential do good in the world, but don’t know how widely it can be applied? There are lots of different ways to study generalisability:

  • Systematic reviews: these look at how an outcome varies in the published literature across samples, settings, measures and methods (meta-analyses do this statistically). This can be done without conducting any new studies.
  • Comparative studies: comparing results from different populations using the same (adapted) materials can show where there may be similarities and differences.
  • Big team science: when researchers from around the world conduct the same study and pool their results, they can look at various factors affecting the presence or size of the effect they’re interested in.
    • For example, the first ManyGoats project is examining goat responses to different human attentional states, and will be testing a diverse range of goats in different living conditions.

  

Activity 3:

Allow about 10 minutes

Think about when a study in your field would or wouldn’t generalise, and make a few notes as to why this might be the case.

To use this interactive functionality a free OU account is required. Sign in or register.
Interactive feature not available in single page view (see it in standard view).

When you are ready, press 'reveal' to see our comments.

Discussion

There are lots of reasons why a study may or may not generalise. Imagine a study evaluating a new therapy for depression in a university clinic with primarily urban-based participants. While the therapy showed significant improvement in depressive symptoms over ten weeks among a diverse sample, including college students and middle-aged adults of various ethnicities recruited through local health centres and university channels, its applicability to other populations and settings may be limited. Factors such as regional differences in mental health resources, demographic diversity beyond the studied age groups, and recruitment biases could affect the therapy's effectiveness in rural or suburban areas and among older adults or adolescents.