6.2 Evaluating information using the ‘PROMPT’ criteria

Hello and welcome. This slidecast will give you some useful tips on evaluating information using the ‘PROMPT’ Criteria. ‘PROMPT’ is a framework for evaluation developed by the Open University. It’s a useful way to systematically assess the credibility and potential value of any information or resource that you come across online.

The mnemonic stands for six criteria. These are: Provenance, Relevance, Objectivity, Method, Presentation and Timeliness. Let’s take a closer look at each of these criteria in turn.

The provenance of information – that is, who produced it and where it came from – may provide a useful clue to its reliability. It represents the ‘credentials’ that support its status and perceived value. It’s therefore very important to be able to identify the author, sponsoring body or source of your information. You can do this by searching online for more information. For example, you could find out where an individual works, and whether they’ve written any other material in the field. You should always treat anonymous information with caution.

Any individual can publish on the web or post to a discussion. Content therefore, has to be judged on its own merit and with reference to the author’s credentials. It’s also useful to consider the sponsoring organisation and method of publication.

You need to establish the author’s status and expertise. For example, ‘Are they acknowledged experts in the subject area?’ You could check how many papers they’ve published on the topic. Have they been frequently cited by other authors in the field? Are they respected and reliable sources? Are their views controversial?

You can ask questions about sponsoring organisations. What type of organisation is it? A commercial company, voluntary organisation, statutory body, research organisation and so on? How well established is the organisation? Does the organisation have any vested interests in the subject area?

Consider the method of publication. Is the source of information well-regarded within the academic community? Is the information peer-reviewed? What do you know of the editor and the editorial board, and how their editorial policy influences what will be published?

The provenance of a piece of information does not always offer a direct clue to its quality. There is something called the ‘stable theory’, which suggests that academic work is often valued highly just because it emanates from a prestigious group, or is published in a prestigious journal. Published information should be judged on its own merits. However, provenance offers an indirect clue to the reliability of information – a safety net that gives you the opportunity to check things out. Provenance can also affect other people’s confidence in the sources you are citing.

Relevance is an important aspect of information quality. Relevance is not a property of the information itself, but rather of its relationship to the specific need that you’ve identified in your work. A particular source may be well researched and written, but if it’s not relevant to the question you are asking or the scope of your work, it’ll be of little value.

Here are some tips for determining relevance of information. Make sure that you are clear about your requirements. Think about why you need the information. Try to avoid having to read everything in full on first pass. If you’re evaluating a large body of material, learn to skim read and scan information to get a quick indication of what it’s about. Look out for the title, headings and subheadings, abstracts or summaries, keywords or descriptors that are relevant to your topic area. Consider whether the information has the right emphasis, and is at the right level for your requirements. It might for instance be too simple for Master’s level. Consider it in the wider context. Does it provide a unique insight into an aspect of your subject? Does the work confirm or refute the findings of others? At Master’s level you’re expected to gain an in-depth knowledge of your subject area, so you should be aware of relevant opposing discourse.

In an ideal world, ‘objective’ or ‘balanced’ information would present all the evidence and all the arguments, and leave you to weigh this up and draw conclusions. In the real world however, we recognise that all information is presented from a position of interest, although this may not necessarily be intentional. Objectivity therefore, may be an unachievable ideal. This means that the onus is on you, as the reader, to develop a critical awareness of the positions represented in what you read, and to take account of this when you interpret the information. It’s also important to recognise that your own belief systems and opinions will influence your ability to be dispassionate, and to evaluate information objectively.

In some cases, authors may be expressing a particular viewpoint. This is perfectly valid as long as they are explicit about the stance they are taking. Hidden bias or errors of omission, whether deliberate or not, can be misleading. You need to identify the views, opinions and judgements being expressed in information. Here are a few tips to help you in your evaluation. Consider perspectives. Do the authors clearly state the viewpoint they are taking? Academic articles will often present unsubstantiated theories for debate. Look out for opinion presented as if it were fact. Look out also for language that is either emotionally charged or vague. Consider sponsorship and vested interests. These may be political, personal or commercial. Academic research for example may be sponsored by industry or government. This does not necessarily make the research less objective, but it may make its interpretation selective. Make sure that all potential vested interests are clearly identified, and any conflicts of interest have been declared.

When preparing for your postgraduate assignments (for example an essay or a literature review), there is a particular onus on you to recognise any selective interpretations of information. You will need to comment on any significant omission or biases that you may encounter in other people’s findings. At postgraduate level, you will also need to be reflective and aware that you may have your own bias, which filters information that you find.

Let’s briefly consider method. This is an aspect that refers to the information produced as a result of using particular methods. This criterion is particularly important when evaluating research, data, questionnaires and for historical information. In these instances, the method is also an important indicator of reliability. Don’t assume that because a research report has been accepted for publication, it is error-free.

With your knowledge of the methods used in your subject area think about whether it’s clear how the work was carried out. Were the methods appropriate? Ask basic questions about methodology (for example the size and nature of samples or participants, questionnaire design and so on). Are the results consistent with the methods stated? Are the methods suitable for your needs?

The way in which information is presented has a profound effect on the way we perceive it. There are many aspects of presentation, any of which, if badly applied, can create a barrier between the message and the audience.

Consider for example, the choice of colour, font size and type, the use of diagrams and images, logical structure, flow and layout, use of language and writing style. Be aware that poor presentation and inappropriate or confusing use of language will hinder your ability to critically evaluate the academic content. You may find it difficult to understand or to follow arguments if the information is poorly structured, has a confused layout, and is badly written with poor language use. Having noted this however, you should try not to let poor presentation stop you using what might otherwise be good quality, relevant information, if that’s the case.

Knowing when information was produced or published can be an important aspect to help you judge its value for a specific purpose, such as a particular assignment or report that you are putting together. This is not quite as simple as saying that ‘good’ information has to be up-to-date. It depends on your particular needs. If you’re looking for the latest research, information about current technology, or key policy changes, then the publication date would be considered to be very important. In some instances, the relevance of material may be more important than its age.

When evaluating timeliness you should ask whether it’s clear when the information was produced. Does the date of the information meet your specific requirements, or would it be considered ‘obsolete’? Has it been superseded?

This brings us to the end of this presentation.

Thanks for listening to this slidecast. We hope you found it useful.