How to pick the best studies

In this section
Key points to remember about looking at studies
What are the different types of studies?
How can I find reliable studies?
How can I use research to make decisions?

There are more than 30,000 medical journals and hundreds of thousands of research articles published every year. How can anyone keep up with them? And more importantly how can you tell if the research is good or not?

Each research study is only one piece of a jigsaw that may take years to finish. Rarely does any one piece of research answer either a doctor's, or your, question.

Best Health is based on Clinical Evidence, a publication doctors by experts from all over the world. These experts look at all the research evidence for a treatment. They weigh it up carefully and decide which treatments work and which don't work. They include research studies that have been published and also those that haven't been published.

Clinical Evidence is published by the British Medical Journal, which has been publishing information for doctors for 160 years.

As new studies are published, Clinical Evidence is updated. And the results of these new studies are evaluated regularly.

Best Health provides you with the same high standard of information as you find in Clinical Evidence. So you can see the information that your doctor uses to make decisions about your treatment.

Key points to remember about looking at studies

• Sometimes doctors introduce new treatments too quickly, before they have been shown to work.
• And sometimes they are too slow to use new treatments that have been shown to work.
• Doctors and other health professionals are not always as good as we might think at deciding what is good research.
• You need to be sure that the advice you get on health or treatments comes from high-quality studies whenever possible.
• Two of the best sorts of research are called systematic reviews and randomised controlled trials.

When you read the articles on Best Health, you'll see numbers at the end of some sentences and at the bottom of some pages. These numbers are reference numbers. They tell you where you can find the research evidence that backs up what we're saying.

Some of the titles of the research articles might sound complicated. That's because they're written for doctors. But a librarian should be able to help you find the reference.

The information in this section will help you answer the following questions:

• What are the different types of studies?
• How can I find reliable studies?
• How can I use research to make decisions?

What are the different types of studies?

There are different types of studies, but they don't all give you reliable information that can help in making decisions. There are three main types of studies that can help you decide whether a treatment works:

• Randomised controlled trials
• Systematic reviews
• Observational studies.

It's not easy to explain them in one sentence, so we've used examples to show what they are.

Randomised controlled trials

This is a type of research study in which patients are randomly put into two groups. One group gets the treatment being tested. The other group gets an inactive (or dummy) treatment. The decision to put someone into one group or another is made like tossing a coin: heads they go into one group, tails they go into the other. Randomised controlled trials are sometimes called RCTs for short.

When the treatment being tested is a drug, this dummy treatment is sometimes a pill without any active ingredients. This type of treatment is called a placebo. The placebo shouldn't have any effect.

Sometimes one treatment is compared with another. Neither the researchers nor the people having treatment know what each group has been given until the end of the trial. They shouldn't find out until all the results have been studied.

Randomly putting people into the different groups reduces the risk that anyone, including the researchers, will respond better (or worse) to a new treatment simply because they expect that they should respond a certain way. This problem is known as bias.

Randomised controlled trials are used to answer questions. An example of a question would be: Should women who have been through the menopause take hormone replacement therapy to reduce the risk of having a heart attack?

The only way to see whether hormone replacement therapy (also called HRT) makes a difference is to randomly pick which women get the drug and which do not. This means that each woman in the study is randomly put into either the HRT group or into what researchers call the control group. All the women in the control group take a dummy pill. Dividing the women into groups is a way of making sure that the groups have a similar mix of women. The group of women taking the HRT should be similar to the group of women who are taking the dummy treatment.

After the women have been divided into groups, the researchers keep in contact with them to see how many women in each group have a heart attack.

It is important that neither the women nor the researchers know who is having an active treatment and who is having a dummy treatment. Making sure that no one knows what group a woman is in is called blinding. Blinding is a way of reducing the effect that people's expectations have on the results of a study. It is important because if people or their doctors expect a treatment to have a certain effect, that effect is more likely to be felt or noted.

Studies that are not randomised (with people put into different groups randomly), controlled (with one group given a placebo or comparison treatment) and blinded (where no one knows what group someone is in) tend to exaggerate how good a treatment is.

Where HRT is concerned, it turns out that there are no research studies that are randomised, blinded and placebo-controlled about women who do not have heart disease.

There is a study of women who had already had coronary artery disease.¹ It did not find that HRT reduced the risk of having a heart attack. For women who do not have heart disease, there is no good-quality evidence to show that using HRT for a long time will reduce their risk of dying from heart disease or having a heart attack. There is evidence from other studies that HRT may increase the risk of developing breast cancer.

Systematic reviews of randomised trials

Sometimes a single randomised controlled trial doesn't really tell us enough about a treatment. The best answer is sometimes found by combining the results of many randomised controlled trials.

A systematic review is a type of study that looks at the results from all of the good-quality randomised controlled trials on a topic. It adds together the results of these individual studies into one summary. This gives an estimate of a treatment's risks and benefits. Sometimes these reviews include a statistical analysis, called a meta-analysis, that adds up the results of several studies.

Here is an example of why systematic reviews are useful.

• Each year, tens of thousands of deaths are prevented all over the world because people who have had a heart attack start taking aspirin. But one randomised controlled trial suggested this treatment was not helpful, even though other good studies said it was.²
• Even when randomised controlled trials are done well, they can sometimes come up with different results. This can happen because the researchers may not have taken account of things that could affect the results. (These things are called biases.) And sometimes the results are different just by chance.
• A meta-analysis was done to look at the results of all the studies of whether aspirin was helpful. It found that aspirin does reduce the risk of a person who has had a heart-attack dying, having another heart attack or having a stroke.³

An international organisation called the Cochrane Collaboration, named after a well-known researcher, does some of the best systematic reviews.

Systematic reviews are not foolproof. Their quality is only as good as the quality of the studies that they include. If the studies are not very good, then the review will not be very good either.

Observational studies

Large randomised controlled trials are expensive and take time. And sometimes it would not be ethical to do a study in which some people were randomly given a treatment. For example, it wouldn't be right to give oxygen to some children having an asthma attack and not give it to others. In cases like this, an observational study may be the best choice.

In an observational study, researchers watch groups of people instead of doing something to them. While these studies are cheaper and faster than randomised controlled trials, their results are not so reliable. They can't prove that something caused something else.

For example, one observational study spotted what seemed to be a benefit from the antioxidant known as beta-carotene, which is found in carrots.⁴ Researchers noticed that people with higher levels of beta-carotene in their blood were less likely to have cancer and heart disease. So some doctors thought that beta-carotene might be responsible for keeping them from getting these diseases.

To find out if beta-carotene really did prevent cancer and heart disease, doctors needed to do a randomised controlled trial. The observational study wasn't good enough to answer the question: "Does beta-carotene help prevent cancer and heart disease?"

When the better study was done (people were randomly given either beta-carotene or placebo pills) beta-carotene didn't work. In fact, it looked like beta-carotene might increase the risk of getting cancer.⁵

Laboratory studies

Laboratory studies are another type of study. Newspapers often have stories of studies showing how a drug cured cancer in mice. But just because a treatment works for animals in laboratory experiments, doesn't mean it will work for humans. In fact, most drugs that have been shown to cure cancer in mice do not work for people.

What if there are no studies?

Doctors can't always base their treatment decisions on the results of studies. Sometimes the research hasn't been done because doctors are used to treating a condition in a way that seems to work.

For example, if you break your wrist, it can either be put in a plaster cast or you can have surgery. If you have surgery, this usually involves putting in pins or other devices to hold your bones in place while they heal. But there isn't enough evidence from studies to tell when surgery is a better choice.

Many things doctors do are not supported by good-quality evidence. This is usually because the studies have not been done. A treatment may work even if there is no evidence for it, but no one can say for sure.

How can I find reliable studies?

All the treatments described in Best Health include links to the original reviews of research articles in Clinical Evidence.

You can find research studies in medical journals. They are written for doctors, and some of them may be difficult to understand. A medical dictionary can help.

The main medical journals that your doctor looks at are:

• BMJ (http://www.bmj.com)
• The Lancet (http://www.thelancet.com)
• JAMA (http://jama.ama-assn.org)
• The New England Journal of Medicine (http://www.nejm.org).

Some of the reviews done by the Cochrane Collaboration (http://www.cochraneconsumer.com) are written for patients. And there is a section on how to make sense of research.

How can I use research to make decisions?

You can use our information to help you make decisions. If you are deciding whether or not to have surgery on your knee for osteoarthritis, for example, our information can help you discuss things with your doctor. You will need to take what you would prefer into account when you make decisions about your treatment. You may want to avoid an operation or at least try physiotherapy for example. Or your pain may be stopping you from being independent. If this is the case, you may want to have surgery so you can get around more easily.

Often there is more than one way to treat a condition, and the evidence may not be able to tell you which treatment is best. The evidence certainly can't tell you which treatment is best for you.

What Best Health can do is make it easier to talk to your doctor. If you use our information, what you read will be based on the same research papers that your doctor has read.