6 Interpreting science news
An important part of being a scientist or a scientifically-informed citizen is being able to interpret scientific information that is represented in the public sphere.
By choosing to study this course, you have expressed an interest in learning about how contemporary science is conducted, and this course aims to help you build the skills and confidence to critically evaluate scientific research. However, it is also useful to have an idea of how to judge the value of science as it is reported to members of the public. The next activity presents some approaches that can be applied to this task.
Activity 6 Using ‘Score and ignore’ to assess news reports
Begin by listening to the following audio clip, taken from a 2013 episode of BBC Radio 4’s Inside Science. Kevin McConway, Emeritus Professor of Applied Statistics at The Open University, discusses how he interprets science as it is reported on radio news bulletins. You do not need to make any notes on this interview, unless you particularly want to.
It's not explored in this short clip, but Kevin has developed a 12-point checklist for evaluating science news on the radio. You will have a chance to read this checklist shortly.
With his co-author, Professor David Spiegelhalter, Kevin expands on the points he makes in the audio feature in a written article ‘’ (McConway and Spiegelhalter, 2012). Take a look at this article – you can use the box below to make your own notes about the content, if you wish, and it would be logical to focus on how he critically assesses the reports on science.
Now you will use McConway and Spiegelhalter’s checklist to evaluate two online news articles that report contemporary scientific research:
- McGrath, M. (2016) ‘Men may have evolved better ‘making up’ skills’, BBC News website.
- Johnston, I. (2016) ‘Nature videos seem to make maximum security prisoners less violent’, Independent.
First, read each story, and score it against the 12 points from the McConway and Spiegelhalter (2012) article.
Now answer the following questions based on your analysis:
- Which article scored highest?
- If you found one article to be particularly low scoring, what made it so?
When you’ve finished your evaluation, read the discussion provided below.
Here are some thoughts on how each article can be assessed through McConway and Spiegelhalter’s checklist. (Don’t worry if your answers differ from these to some degree – evaluation of this sort is a subjective exercise.)
Analysis of McGrath (2016)
- Just observing people? Yes, just looked at recordings.
- Original information unavailable? No, it is mentioned that the paper was published in Current Biology and a link provided.
- Headline exaggerated? No, headline suggests it is only a possibility.
- No independent comment? No, independent comments included.
- ‘Higher risk’? No, risks not really mentioned.
- Unjustified advice? Not really.
- Might be explained by something else? Yes, other explanations are possible, as correlation does not mean causation.
- Public relations puff? Yes, it is a sports and gender story that will attract attention.
- Half the picture? No.
- Relevance unclear? Yes, it is related to business but the data don’t apply to business situations.
- Yet another single study? Yes.
- Small? Yes.
Total score: 7/12
Analysis of Johnston (2016)
- Just observing people? No, there was a control group.
- Original information unavailable? Yes, the research was presented at the American Psychological Association’s annual convention. Therefore you would have to search for the associated publication which was only published much later in September 2017 in Frontiers in Ecology and the Environment: https://doi.org/10.1002/fee.1518
- Headline exaggerated? No, uses the word ‘seems’.
- No independent comment? Yes, only researchers quoted.
- ‘Higher risk’? No, risks not really discussed that would affect the public.
- Unjustified advice? No, advice not relevant to the public in this context.
- Might be explained by something else? Yes, and the authors acknowledge this.
- Public relations puff? Yes, the story is about criminals to attract people’s attention.
- Half the picture? Yes, but hard to say for definite as the data are unavailable.
- Relevance unclear? No, relevance is clear for reducing violence in prison.
- Yet another single study? No, several studies are mentioned.
- Small? Possibly, study size not given.
Total score: 5/12, but note that the information is less clear in this second article and can’t be easily verified, as the article refers to a presentation rather than a peer-reviewed paper.
Now that you’ve practiced some techniques for critically appraising information that is presented to you, the next few sections of the course will explore a scientific area in some closer detail. The topic is plastics in society. You don’t need to worry if this topic is new to you, and you don’t follow all of the science that will be explored. This will be an exercise in gauging your current knowledge of a subject, learning some new information, and examining how the subject relates to and impacts your own life, before using the skills you’ve just developed to evaluate some sources of information about plastics.