The Extended Project Qualification (EPQ) is an opportunity for you to work independently on a topic that really interests you or that you think is important. It is equivalent to an A-level qualification. These articles are designed to help you if you are enrolled on an EPQ.
See previous article in series: Designing your research question
Before working through this article, you should have settled on your research question.
This article will support you through the next steps in the research cycle (Figure 1): collecting evidence (or data) to help you answer your question and starting your analysis of that evidence. First, let’s look at collecting data.
What is already known about your topic?
The first step in answering a research question is usually to do a ‘literature review’ or ‘research review’.
These articles focus on the ‘research review’ type of EPQ, in which collecting and analysing evidence from what other people have written will be a major part of what you do.
Researchers need to do research reviews for two reasons. First, they want to uncover what is already known. A lot is already known about some topics and they want to be sure that they are researching a novel question. Second, they want to get a balanced view of what is known, rather than jumping in and relying on the first pieces of information they find.
A literature review starts with focused and serious reading, to help you develop your knowledge and understanding of the topic, and begin to gather evidence that will help you answer your question.
Julia, a researcher in AstrobiologyOU, whose research explores the possibilities for habitable environments in the Solar System, demonstrates how she starts a literature review. As you read through, think about the process Julia describes and how you could apply the steps she takes to your EPQ research review.
When you’re starting a literature review, where do you look for evidence?
Any literature review requires a well-defined topic. Let’s assume the topic is something in planetary science!
Find an exciting topic: https://solarsystem.nasa.gov provides a good overview of our Solar System. Let’s pick Mars.
Define a specific question. How did Valles Marineris form?
Learn about Valles Marineris. You can use Wikipedia as a starting point and look up the references given there.
Use Google Scholar to find specific literature (e.g. search words such as *formation of Valles Marineris*). Look up research cited in the papers you read if they are also relevant to your topic. Make sure you always cite your sources.
My colleague Julia is a research fellow in AstrobiologyOU. She uses computer modelling to understand geological processes on planets such as Venus and Mars. For her, the process of starting a literature review is all about starting broad and gradually narrowing down. So, assuming that the topic is something in planetary science, start with something that gives you an overview. In this case, it’s the whole solar system. Then gradually coming in a step, first of all, focusing on one planet: Mars. Coming down to something on that planet; how did Valles Marineris form? And then having decided on the specific topic, learning more about it, Julia suggests using Wikipedia as a starting point, but looking up references given from it, and then using Google Scholar or other research engines to find specific literature, to look up research, and to find papers that are relevant to your topic.
Comments on Julia’s approach
You’ll notice that Julia starts from the ‘big picture’ and gradually focuses down to more specific material. This is very similar to the process you followed in article 1 (Designing your research question) to move from ‘this is a topic I’m interested in’ to ‘this is the question I want to ask’.
You’ll also notice that Julia moves from a quite open, broad source of evidence (Wikipedia) to using more serious sources (Google Scholar). There’s more on where and how to look for evidence in Section 4.
You will know that Wikipedia, despite having information about hundreds of thousands of topics, is not a 100% reliable source. Wikipedia can be a good place to start your research, get your ideas moving and find places where you can look for more information, but you should always cross-check any information you find there against another source.
With so much information available online, how do you begin finding relevant material? Most researchers start by assembling a collection of keywords that relate to the topic and can be used to search for more information. But how do you find those all-important keywords?
In some ways, deciding on your keywords is like the ‘ten words’ activity you used in Article 1 to help you move from topic to question.One way of identifying keywords is to start with the ‘big idea’ and gradually work your way in. There are lots of different ways to do this. As an example, Devyani, a PhD student with AstrobiologyOU, made a short video to show how she creates a mind map (Figure 5) to identify keywords before starting her literature search.
Photo of mind map
But how do you come up with those all-important keywords? Devyani, who is a PhD student in AstrobiologyOU, gave me a few thoughts. As a PhD student, she’s just been through this process herself.
She suggested looking for key concepts and words in the main topic, perhaps splitting your question up and looking at separate elements of it. So for her, space technologies, sustainable development, SDGs, would be good keywords. Perhaps writing down similar words to the keywords that you've identified, using a thesaurus to help you find alternatives.
Once you start searching, using things like Google Scholar or the Web of Science, you can use advanced search options such as ‘and’ and ‘or’ functions to help you link topics that you need to look for.
We’ll come on to different places you can actually do your searches in a couple of minutes.
You might want to try Devyani’s method, or you could try this quick activity from The Open University to get started on finding good keywords.
Going back to the example research question we used in Article 1: ‘How did young people use social media for activism?: comparing the content of Instagram posts on #blacklivesmatter and #FridaysForFuture during 2020’
What keywords might you use to search for relevant material to help answer this question? Perhaps you’d come up with ideas like “social media”, “activism”, “black lives matter”, “Fridays for future”.
As you can see, keywords can sometimes be more than one word!
Searching for evidence.
Having identified your keywords, the next step is to use them to start searching for evidence that will help you answer your research question.
Evidence can come from many sources: books; academic articles (often called ‘papers’); reports from businesses, charities and other organisations; newspaper articles; radio and television; websites; social media … the list could go on!
With so much material out there, it’s helpful to make a plan before you start your search. Think about:
- the keywords you will use
- the most useful places to search
- the time you have available for searching.
You will gather a lot of information, so you should keep good records. Knowing what you’ve looked for, where you looked and what you found there will help you avoid repeating something you’ve already done. You’ll also be able to fully acknowledge the sources you have used in your research (this will be discussed further in Article 3). Make notes of:
- what keywords and combinations of keywords you used (your ‘search terms’)
- where you looked
- what documents you downloaded or read online
- notes you made about what you read.
You could keep a record in a simple table, perhaps on a spreadsheet (you can download an example here).
For any documents you find, you’ll also find it useful to keep records of:
- the name of the author(s)
- when it was published
- the title of the article, report, book or chapter
- where it was published – i.e., the title of the journal, book or website
- a link to the document or its DOI (digital object identifier)
- (for books) the name of the publishing company and where it’s based.
The digital object identifier (DOI): You might not have come across a DOI before. Most academic articles have a DOI – a unique string of numbers and letters that is permanently attached to an article. If you paste a DOI into a browser, it will take you straight to that article. Try this one: https://doi.org/10.14324/RFA.05.2.14
When it comes to writing up your dissertation, these notes can be good evidence in themselves, showing how you carried out your review, so it’s sensible to get your record-keeping method set up before you begin searching.
Where should I search?
It’s very likely that you will carry out your search using online resources. But there are so many to choose from that it can be difficult to know where to start.
You can use the well-known search engines, such as Google, Bing or Yahoo, but you will get more reliable results if you use a specialised search engine such as Google Scholar, which returns links to articles and academic papers produced by researchers in universities and research institutions.
Many of the papers, articles, newspaper pieces and other items that you will find in your searching – but not all – will be open access, which means they are free to view and you can download them.
Many universities have
repositories that store the work of researchers at that university. The Open
University’s repository is called Open Research Online (ORO). Most UK and international universities have something similar. Universities
maintain these repositories because they want to make the work of their
researchers available for others to use.
Repositories are often looked after by the university’s library team, so if you can’t find the repository easily, try looking on the library pages on the university website.
Online repositories usually allow you to search by a topic, or by the name of the author if you are trying to get hold of a particular resource.
If you’re interested in the work of a particular research group (such as AstrobiologyOU) or a specific researcher, you’ll often find them on social media. Most research groups keep copies of work published by their researchers; it’s always worth contacting them. Explain that you want to use the research for your EPQ.
Researchers often have their own websites (here’s an example), which you can find using a search engine. If you can’t get hold of an article that you need for your work, email the researcher. Researchers are often willing to share copies of their work to help other researchers, so explain that you want to use it for your EPQ.
Many researchers keep copies of their papers and articles on networking sites such as ResearchGate, which also allow you to search by topic. However, these records are kept up to date by the researchers themselves, so you should always check against another source if you can.
However, you will have to create an account, or pay a subscription to make full use of sites like this, which might not be practical for you.
If you find the researcher on the database but the paper you want isn’t there, use the database’s facilities to contact the researcher. Again, tell them why you want to read the article.
Universities and research institutions are increasingly bringing their resources together into large collections.
CORE is the world’s largest collection of open access research papers and articles.
ARXIV.org has more than a million articles, mainly in physics, mathematics and engineering.
Journals are the places where researchers publish the articles that discuss their research. There are many thousands of academic journals in existence, covering every topic imaginable.
For example, try typing the phrase "ecology journal" (complete with speech marks) into a search engine. You will find a long list of possible titles is returned.
On the journals’ websites, you can search by topic to find relevant articles. If you are searching for a specific paper, and know the journal it was published in, you can use the journal’s search function to find it. However, many journals are commercial organisations and keep articles behind a ‘paywall’, meaning they are only available to people or institutions that subscribe to them.
Fortunately, more and more journals (for example PLoS – Public Library of Science) are fully open access and many journals make some articles freely available.
Print media: If you’re looking for material published in newspapers or magazines, first try the website of the newspaper or search for their digital archive. If you can’t find it online, try your local public library; they often have access to hard-copy archives or can advise you where to find them.
Book publishers: There are many academic publishers, such as Ubiquity Press, that produce open access books you can download.
Organisations: research funders, such as the Wellcome Trust, make available copies of papers and reports written by their researchers. Other organisations’ websites, such as those from the government or charities, often have copies of project reports or annual reports available.
Public libraries: Finally, don’t forget your local public library. Even if they don't have the exact article or book you’re looking for on the premises, they have trained librarians who can help you find it or suggest alternative routes to get the information you need.
Refining your search.
Even if you use very specific keywords, an online search might return hundreds, if not thousands of results. How can you cut hundreds of results down to a sensible level?
Focusing and refining the keywords you use for your search will help you be realistic, making the best possible use of the limited time you have available for the EPQ, and cutting down your results to an amount that you can realistically deal with.
Here are Michael’s key tactics for refining your searches:
- Linking several search terms to narrow down your search.
- Starting broad and then refining – he used a personal example where he started by looking for ‘bacteria associated with plants’ and narrowed down to ‘bacteria associated with peas’.
- Keeping track of your topics and sub-topics with a list, spreadsheet or mind map.
- Remembering you don’t have to read every single resource you find.
My colleague Michael, who’s a microbiologist, is particularly interested in looking at life in extreme environments on Earth that might be similar to places such as Mars.
He had some useful ideas on how to start refining your searches. He suggests linking search terms, so when searching online you can use symbols such as the plus sign and quotation marks that are incredibly helpful. Typing in ‘this’ plus ‘that’ will return results that have both of those elements. Conversely, typing ‘this’ minus ‘that’ will mean you only get results that include the phrase ‘this’. You can use quote marks to make phrases stick together, so quote marks "this stuff" and plus quote marks "that thing", will enable you to find papers that refer to both of those topics.
Michael also suggested it’s good to start broad and then refine searches. For example, for his PhD, he had to write several chapters for his thesis about bacteria that are associated with plants. The first searches were in the obvious place ‘plant-associated bacteria’, which he then refined to think about specific types of plant, such as ‘pea-associated bacteria’ or ‘legume-associated bacteria’. Those initial searches and papers allowed him to identify other useful terms that then meant he could be even more specific with his searches. For example, searching for ‘nitrogen’ plus ‘plant-associated bacteria’ or ‘legumes’ minus ‘disease’.
When searching, he also suggested it’s really helpful to keep track of topics and subtopics with a list, spreadsheet or a mind map. Whatever works for you. He found this helped him keep his notes organised and meant he didn’t have to repeat searches, because he had a record of what he had looked for.
But the most important point, he felt, was not to feel overwhelmed when doing a literature review. It’s possible to turn up hundreds of papers on any topic, but you don’t have to read all of them. As a start, he suggested that you look for review papers. These are papers in which the authors have pulled together material from several other sources. They give a really good introduction and overview of a topic, and can give you some clues on where to look next.
Practising searching using Google Scholar.
Google Scholar is a freely-available search engine that only returns links to scholarly literature, so it’s a good place to practise your searching techniques.
Imagine that you have decided to research this question:
What are the health benefits of people spending time in nature?
What are the keywords you could use in your search? You might think of:
- nature therapy
- green health
- forest bathing
Experiment with entering the search terms into Google Scholar. For example, type the words ‘green health’ into the search box and press enter.
Using a simple search term like this can generate a long list of articles. In this example (Figure 10) it was somewhere around five million, which is far too many for you to review meaningfully!
Combining search terms and using punctuation to keep two or more words together can help you focus your search and return fewer results to look through. Try typing “forest bathing” + health + ”open access” into the search bar:
Quote marks (“…”) keep words together. With quote marks (“forest bathing”) the search will return articles about forest bathing. Without them (forest bathing), you’d get articles on woodland and swimming pools (among other things!).
Using the + symbol combines “forest bathing” AND health. Adding + “open access” means you will only see results where you have free access to the full article.
This cuts down the number of results to around 1600 (Figure 12):
You can then click on the search results to access the articles. Clicking the top link brings up an article about forest bathing (at time of writing – the top result may well have changed since then).
To refine your search even further, Google Scholar has filters you can use to tweak your search. For example, if you are only interested in very recent material, you could filter so that only material published after 2021 is shown. This is useful if you want to access material published in a particular time range.
Most search engines or search facilities on repositories, collections or websites have an advanced search function that allows you to refine your search to cut down the number of results to something more useful. You can search for material published before or since a specific date, include or exclude specific words, or look for articles by a specific author.
To access advanced search
on Google Scholar, click on the menu hamburger in the top left.
What is the right number of resources to include in your EPQ research?
It’s difficult to give an exact number of resources you should aim to use in your EPQ research. You could keep going for ever – new articles come from researchers in a constant stream! A rule of thumb is to stop reading when you sense you are no longer finding new ideas.
Even a refined search is likely to throw up lots of material from a range of sources. Wherever your material comes from, you should always scrutinise it carefully. But how do you decide what you can trust (therefore making it useful), and what you can’t?
Assessing a source’s credibility is a good place to start. Making sure you draw your evidence from credible, believable, trustworthy sources is very important for your research.
Quiz: judging credibility.
Comments on credibility of sources
In order of least to most credible, below we explain why each source is credible or not:
- Tabloid newspaper article: this kind of reporting puts a priority on the sensational and doesn't always give a full and balanced story.
- Podcast: podcasts often present the podcaster’s personal opinion, and it’s not always obvious whether that’s based on research.
- Popular science books: authors usually draw material from a range of sources, and sometimes interview the researchers to get a first-hand view.
- New Scientist article: this magazine has a good reputation for serious science journalism, and the articles usually have links to the original research so the reader can investigate further for themselves.
- Original research paper: research papers usually give the reader the evidence that the researchers gathered, so the reader can review it for themselves.
Judging credibility – Thomas’ thoughts.
Thomas, who is a lecturer in space governance, discusses the credibility of materials. As you watch his video, listen out for the ways in which he judges credibility.
‘How do you judge the
credibility of a source? That is, what questions do you ask yourself when you
read or review a source?’
So this is one of the more important questions for any researcher, particularly one in the humanities such as myself. It’s also one of the hardest, especially when you’re just starting out because the honest answer is: it’s experience. I’ve learned enough about my field that I can differentiate between ‘I don’t agree with this’ and ‘this is nonsense’ but obviously that wasn’t always the case. So in the early days, in high school and undergraduate, there are a few things you can look at to get an understanding of what makes a good source. First port of call is always what your teachers and lecturers recommend, they’ve got that experience that you don’t have, so they’ll be pushing you in the right direction. Pay attention to how those sources are written, that’ll give you clues as to what makes a good source.
But how to find them on your own? Well again, when just starting out, it’s best to be conservative, to err on the side of caution, so there are a few things you can look to. First is publisher. Academic presses (like Oxford University Press) specialise in publishing scholarly work, so that acts as a form of quality filter. Author is another one. Who is this person, why are they qualified to write an article or a book on this topic? There are other indicators you can use. For example: are there footnotes, references, a bibliography? What sort of sources do they use? How up-to-date are they?
Then there’s the work itself. Is it well-structured and thought out? Do they actually make an argument? Do they explain their reasoning to you, or do they just declare things to be true? ("Well of course it’s true, I said it" – it happens more often than you would think it does!)
Finally, you need to read widely and broadly. You need to read authors you agree with, and authors that you disagree with, and then work out what you think. Gradually you’ll be able to work out what constitutes a good source without being dependent upon some of these indicators – which is good, they’re not ironclad rules; some excellent works of history have been written by people without history degrees. And more importantly, you’ll be able to discern the difference between ‘I don’t agree with this’ and ‘this is nonsense’.
Thomas’ main points
Thomas’ main points were:
- Ask people you trust, such as teachers, what they recommend.
- Look at the quality of the writing – good sources are well-written and well-structured, with evidence to back up the arguments they present.
- Go to reputable sources such as academic publishers.
- Look at the writer’s qualifications on the topic.
- Look at the sources the writer has used.
- Read lots for yourself and build your ability to judge gradually.
Judging credibility – Charlotte's thoughts.
Charlotte, who is a researcher in geochemistry, also has some thoughts about how she judges credibility.
If the source is in her field (in other words, she knows something about it), she looks in academic journals where she knows the work has been reviewed by experts (or ‘peer-reviewed’). She checks that:
- the data support the conclusion the authors have come to
- the methods are appropriate and up-to-date
- they haven’t cherry-picked the best data and ignored others
- the authors don’t have any financial interest in coming to a particular conclusion.
If it’s an area she’s less familiar with, she starts with newspapers, online news and experts. She looks for:
- links to the original research
- what qualifies the writer to be an expert in that area
- whether other experts agree with the author
- any hint of conspiracy theories
- whether the authors have any financial interest in a particular conclusion.
Another colleague, Charlotte, who’s interested in extremophiles – life that lives in very extreme environments on Earth – said that for her, essentially the rule of thumb that she uses is, one: peer-reviewed research. This is a term that means that the paper, before it’s published, has been looked at by two or three colleagues who are knowledgeable in the area. They offer comments and the original authors are then able to improve what they’ve written.
Peer-reviewed research is typically what’s published in academic journals, and that’s the most credible source. Everything else; social media, newspapers, online news forums, is less credible. Charlotte says that when she is reading or researching in the field that she’s an expert in – geochemistry – she always develops her ideas using peer-reviewed academic articles, asking herself: do their data support their conclusions? Are they using the most appropriate and up-to-date methods?
Are there signs that they’re cherry-picking the best data, or ignoring data that doesn’t quite fit what they want it to be? And, super important: do the researchers have financial interests in the conclusions? You might not necessarily trust someone who was working for a toothpaste firm to give you the absolute disinterested best data on tooth decay.
However, Charlotte said that when she’s reading about a topic that’s outside her field of expertise, perhaps in something in health and medicine, she finds it much harder to read and understand the peer-reviewed literature, as sadly it will often use specialist language or jargon. Therefore, in such situations, she often uses a larger range of sources, which might well include newspapers, online news, and things written by people who have been deemed experts.
In that case, some of the questions she asks herself are things like: does the expert or the news article refer back to the original research? Is the news article or expert giving an opinion that’s backed up by a range of research? If it’s someone giving their opinion, what qualifies them to give that opinion? Is there anything about conspiracy theories? Those red flags that Thomas talked about. And again, do the authors seem to have a financial interest in the opinion that they’re pulling out? If the sources she looks at refer to specific academic articles, it makes her think that these sources may be more credible than those that don’t. And if the experts on news articles are spreading an idea that feels like conspiracy, has been widely debunked, or for which there is very limited evidence, then she doesn’t count that as a credible source.
What are the similarities and differences between how Charlotte and Thomas judge credibility? Charlotte is a science researcher, whereas Thomas is a law researcher – do you think this affects their processes?
Thomas and Charlotte both:
look at where the article has come from – a reputable source or one where work is reviewed by experts
consider the writer’s qualifications to be an expert on a subject
review whether other experts or sources agree with the writer
- check that the arguments or conclusions are supported by data.
Charlotte will look for links to the original research and any evidence of financial interests.
Thomas will look at the quality of the writing and the structure of the article.
Whether you use PROMPT, RAVEN or another method of assessing credibility, this will help you determine that you have the best sources possible for your project. Then it is time to get started reading for the literature review.
A good research review is more than just a list of ‘she says
this ... they say that … he says the other’. It’s an opportunity to test and
show the strength of different arguments and the contribution they have made to
The key is to critically read the material you have found.
The aim of critical reading is to assess the strength of the evidence and the argument. It is just as useful to conclude that a study, or an article, presents very strong evidence and a well-reasoned argument as it is to identify the studies or articles that are weak.
As you read, it is useful to keep asking yourself questions such as:
Why am I reading this? – Because it’s interesting? Useful? Has good information for me?
Do I trust the source? – What evidence do I have that the source is trustworthy?
What claims are the authors making? – Have they included their conclusions?
Do I think those claims are trustworthy? – Have the authors given me the evidence they are basing their claims on so I can judge for myself?
Imagine you were researching the question: ‘what is the influence of advertising on people’s
consumption of junk food?’
Remembering that you will find material from a range of sources. Here we’ll use an article from the Guardian newspaper’s website.
What information in the article do you think would be relevant or useful to help you answer this question? Use the questions in the ‘Reading critically’ tab to help you reflect.
What did you think about as you read the article?
You might have picked up points such as:
Why am I reading this?
- It is about the effect of advertising on people buying junk food, so it is relevant to the question.
Do I trust the source?
- It comes from the Guardian, a well-known UK newspaper.
- The writer is named, so you could check on other things they have written – for example, if they specialise in writing articles about food or have a background in food science.
What claims are the authors making?
- That a ban on junk food advertising has led to a reduction in purchasing of unhealthy food.
Do I think those claims are trustworthy?
- The article mentions that the research was done by researchers at a university – you could look for them on the university website.
- One of the researchers is named and there is a quote from them.
At this stage, no one expects you to come up with all these possibilities! But keep those critical questions at the front of your mind as you search for and examine the evidence. Remember to ask:
- Why am I reading this?
- Do I trust the source?
- What claims are the authors making?
- Do I think those claims are trustworthy?
For more ideas on how to judge the credibility of the material you find and think critically about the contents, watch this short video on critical thinking, produced by the BBC and The Open University. It discusses five key strategies you can use to sharpen your critical thinking.
In this article, we have looked at the second part of the research cycle – how to find the evidence that will help you answer your research question, and how you can start to read it critically and analyse its relevance to your question. When you have finished searching, reading and analysing, you are ready to move on to the next step: writing your dissertation.