How do we find credible research?

So, how can we trust the interpretations of a study knowing that scientists sometimes distort the results to favor their clients? And what about the study's participants? How can they be sure the experimental treatments they receive are chosen on merit and not because of what a researcher stands to gain? With the amount of time and planning that goes into designing a study to stand up to scientific standards, it's absurd to ignore the human element. The more background on the people, the institute, or organization that performed or funded the research, the better we can assess the reliability of the data.

Here are some guidelines we at Smart Publications use when assessing studies:
Who funded the study?
It's easy to automatically disregard a study that has been funded by a company that wants to promote its products, but we realize that a company isn't going to spend a lot of money on research that makes them look bad. The research, however, becomes more valid if the results appear in a peer-reviewed journal because that means the methods and conclusions will have been critically examined by other experts in the field. Of course, we're skeptical if a particular scientist's work is always favorable to the organization that sponsored the study.

• What kind of study is it?

1) Epidemiological studies are observational studies, best at identifying powerful associations, like the one between smoking and lung cancer. They can't establish cause and effect-such as Alzheimer's disease is caused by cooking in aluminum pots-but rather can only suggest a relationship between two factors or events. When the relationship between factors is weaker, like the link between smoking and breast cancer, epidemiological studies are less reliable and often produce contradictory results. Also, the larger the study and the longer it has been carried out-such as the 45-year-old Framingham Heart Study and the Nurses Health Study-the more certain you can be of the findings, because researchers can better account for factors that could lead to a bogus conclusion.

2) Clinical trials use people as subjects and randomly assign people to two treatment groups. In a blind or single-blind trial the subjects don't know whether they are receiving the treatment or a placebo until the study is completed. Placebos are treatments that appear identical to the real treatment, but contain none of the active ingredients being studied. Using placebos helps eliminate results that may be due to the subject's expectation that something is supposed to happen. In a double-blind trial neither the subjects nor the researchers are aware of the treatment being used. This type of study provides the greatest precision because it removes any possibility of experimental bias.

3) Laboratory experiments are done in vitro on cells, or tissue samples in culture, or in vivo on laboratory animals -- usually rodents. They can be tightly controlled, in that the scientists can make sure the compared groups are genetically identical and the conditions to which they are exposed (except for the factor being studied) are identical. But, as Jane Brody points out in her article, ("Personal Health: A Study Guide to Scientific Studies" New York Times, Aug. 11, 1998) "It is a long way from the test tube or laboratory mouse to man or woman. The fact that all people do not have the same genes, hormones, metabolism or life circumstances may greatly modify the effects. Or human biology may be such that whatever happens in a test tube or lower animal would not apply to us." ² 

• Has the study been peer-reviewed?

After receiving funding and completing the research, researchers write a manuscript and submit it for publication. The journal sends out the manuscript for peer review, a process in which several scientists with expertise in the area assess its worthiness and importance. The manuscript is not accepted for publication until the reviewers are satisfied that the methods and conclusions are valid and worthwhile.

Conclusion:

Finally, in looking at scientific studies, we realize that a single study does not mean a substance or nutrient is either beneficial or hazardous. As Jane Brody writes, "No matter how reputable the scientist or the institution where the research was done or how compelling the findings appear to be, they must be confirmed by one or more independent studies before scientists will accept them as gospel."

Before the scientific community and public accept the benefits and advantages of a particular substance or supplement, it's necessary for it to be studied repeatedly from various angles, by more than one research team. Science is often a guessing game, but with ethical researchers insuring that scientific measures are put in place, it is possible to get valid results that have broad and significant implications for the health and welfare of mankind. 

Page 2 of 2 pages  < 1 2

References