Skip to content
Health, Sports & Psychology

Evaluating alternative therapies

Updated Monday 17th March 2008

Toby Murcott explains the difficulties in evaluating alternative therapies - problems conventional medicine are familair with.

The two most important questions asked of any treatment are "does it work?" and "is it safe?". It doesn't matter what type of intervention - mainstream, alternative or complementary - both the practitioner giving the treatment and those receiving it need to have some type of answer to these basic issues.

There is a third question, "how does it work?", which is both interesting and important. It is not, though, directly relevant to a discussion of effectiveness or safety of a treatment. We do not fully understand how many different drugs work but they are used safely and effectively day in day out. You don't need to know the physics of the Sun's inner furnace to feel its heat on your back.

The gold standard of medical research is the double blind randomised controlled trial (RCT). Take a group of people who have the condition you want to treat. Divide them into two groups at random. Give one group the treatment and the other group either a known treatment or a dummy treatment (a placebo). This is the control or comparison.

To complete the picture neither the person administering the treatment nor the patient receiving it know whether they are getting the treatment or the control. This is the double blind.

To get meaningful results the number of patients in the trial needs to be large, the more the better. A study of 15 or so individuals can give hints but not much more; whereas a trial of 10,000 patients would be expected to produce a good indication of how effective a treatment really is. It would not, though, be comprehensive. It could easily miss an adverse reaction that only affects one in fifty thousand.

The double blind RCT works very well for pharmaceuticals. It's easy to hide whether patients are given the treatment or the control. The same is true for any herbal medicines or anything that is given as a pill or potion. However, things start to get trickier for other procedures. Take physiotherapy. It's relatively easy to prevent the patient from knowing which treatment they are receiving. It's impossible to ask the therapist to work with a patient and not know whether they are doing it for real or just pretending. So its extremely difficult to make any physical therapy truly double blind, single blind is likely to be the best available.

Then there is the matter of the relationship with the therapist. If you like the person who is administering the treatment the chances are you'll follow their instructions more closely than if you hate them. And then there's the fact that physical treatments are often tailored to the individual so a study is not necessarily comparing like with like.

The final major hurdle is where a treatment is made up of more than one element, a so called complex interaction. Physiotherapy is often a complex interaction, the patient might receive treatment in clinic and have to do exercises at home. It could be that these two together are much more effective than each on their own. But suddenly you have to do three separate studies to work this out properly. One of clinic based treatment, one of home exercises and one of both together. This increases the complexity, and the cost, significantly.

All of these problems are well known to medical researchers and there are ways of getting around them, at least in part. These are, however, often writ large when applying the science of clinical trials to complementary and alternative therapies.

Many are physical therapies that cannot be classically double blinded. Massage, reiki, Bowen therapy, aromatherapy or reflexology, for example. The relationship between the therapist and patient is often central to the treatment. They are often tailored to the individual patient. And many are complex interactions with more than one thing going on simultaneously.

The debate around complementary and alternative medicines often centres around the quality of evidence. Anecdotal evidence - individual success stories - can be compelling but carry no scientific weight. Applying classic clinical trial techniques is often difficult and the results are often open to different interpretations.

There is a gulf between the quality of evidence for most complementary therapies and many conventional treatments. That gulf is not always as large, nor as well defined, as it might appear. The problems of researching complementary treatments are many and varied, but they are shared with some mainstream therapies. Clinical research is a powerful tool, but it cannot provide certainty. When it provides strong evidence it is generally accepted. While it is less clear or absent there will be debate and, often, controversy.

This website is provided for general information only. You should not treat it as a substitute for the medical advice of your own doctor or any other health care professional.

The BBC and the Open University are not responsible or liable for any diagnosis made by a user based on the content of the OpenLearn website.

The BBC and the Open University are not liable for the contents of any external internet sites listed, nor do they endorse any commercial product or service mentioned or advised on any of the sites. Always consult your own GP if you are in any way concerned about your health.



For further information, take a look at our frequently asked questions which may give you the support you need.

Have a question?