Claims about educational technology’s beneficial impact on learning, teaching and society are widespread. They are also frequently contradictory, as mentioned in Week 1.

In late 2023, educational technology-related news reported on the website Science Daily included the following headlines:

• Children’s Brains Shaped by their Time on Tech Devices (ScienceDaily, 2023)
• Chatbot Can Teach Kids Supportive Self-Talk (ScienceDaily, 2023)
• Board Games Boost Math Ability in Kids (ScienceDaily, 2023)
• Combining Maths with Music Leads to Higher Scores (ScienceDaily, 2023)
• Robot Helps Students with Learning Disabilities Stay Focused (ScienceDaily, 2023)
• The Way You Talk to Your Child about Math Matters (ScienceDaily, 2023)
• Helping Adolescents To Feel Competent and Purposeful – Not Just Happy – May Improve Grades (ScienceDaily, 2023)

The headlines each contain an implicit assertion that they are presenting truth about the world. These are known as ‘knowledge claims’.

Knowledge claims like these can have a powerful impact in influencing educational policy, funding decisions, institutional initiatives and individual educators’ practices. For example, in the UK House of Lords in November 2023, Baroness Harding of Winscombe said:

I was really struck by some statistics from an RM Technology research pamphlet, published in June 2023. It did some research on 1,000 secondary school students this summer: 67% of them already used chatbots such as ChatGPT—67%, just six months after it launched—and 48% said that excluding it would really hold them back. However, 38% said they felt guilty about using it (Hansard, 2023).

Social media platforms such as Facebook, BlueSky and blogs allow knowledge claims to spread fast and wide.

However, the potential reach of knowledge claims and related research reports does not prove their accuracy. Just because a lot of people are saying something works, that doesn’t mean it necessarily does, or that it will work for you.

A recent review by the UCL’s Centre for Education Policy found that, of 25 of the most popular maths apps for children aged five, only one had been empirically evaluated for positive impacts on maths outcomes. Half of them did not include features known to support learning, such as feedback loops, and six of the 25 contained no mathematical content at all. If the UCL finding was extrapolated across the half a million apps labelled “education apps” in the app store, 480,000 would not be evaluated, a quarter of a million would provide no learning support and 120,000 would have no educational content at all (Hansard, 2023)

As an online educator you’ll need to navigate often contradictory claims about the online pedagogy and educational technology options available to you, in order to make decisions about your own practice. Reading research reports is an obvious place to start to gain an insight into what works for others, or might work for you. However, not all research reports are equal and while some make claims on the basis of rigorous, valid studies, others rely on flimsy and flawed evidence to support the assertions made.

In the first section of this week you’ll explore ways of identifying and evaluating claims made about online teaching methods and educational technology innovations.

© The Open University