Glossary

Big team science

A research project in which researchers from around the world conduct the same study and pool their results.

Conceptual replications

A type of replication study which aims to vary some aspect of the original study, in order to better understand the underlying phenomenon.

Constraints on generality

A statement identifying populations sampled in the study and potential limits to the samples and methods, enabling others to assess the extent to which results can be generalised.

Direct replications

A type of replication study which aims to stay as close to the original study as possible.

False positive

An error that occurs when a researcher believes that there is a genuine effect or difference when there is not (e.g. a person has a positive Covid test although they do not have Covid).

False negative

An error that occurs when a researcher believes that there is no effect or difference, when actually there is (e.g. a person has a negative Covid test although they do have Covid).

Generalisability

The extent to which the findings of a study can be generalised to other situations, beyond the specific participants and conditions of the study.

HARK-ing

Researchers are HARK-ing if they write papers as if they had a hypothesis they wanted to test in their study, whereas in reality, they made up the hypothesis after seeing the results.

P-hacking

In quantitative research, exploiting techniques that increase the likelihood of obtaining a statistically significant result.

Post-hoc justifications

Researchers write up justifications for their actions after a study – these justifications were not planned or decided before the study happened.

Reproducibility

A study is reproducible if you are able to get the same results when conducting the same analyses on the same data as the original study.

Replicability

A study is replicable if you are able to conduct the same study again, generate new data, and still get the same results as the original study.

Selective reporting

Researchers are selective reporting if their results are deliberately not fully or accurately reported, in order to suppress negative or undesirable findings.

Systematic review

A structured literature review, which analyses existing research evidence according to a fixed set of criteria, then synthesises what the research evidence shows.