Subject: RefReports - Research Bias - Ethics, Statistics - Most Research Findings Wrong
Caution - This study finding no benefit for dietary supplements is a survey study not a randomized control study with statistical controls and is not double blind. The summary article says "Zhang’s team took advantage of survey data from tens of thousands of U.S. adults". This does not mean that the study results are wrong but that it may lack much power.
As one example, it mentions "People who reported taking more than 1,000 milligrams of calcium per day were more likely to die of cancer." There have been studies on calcium. My endocronologist dropped a study on me that showed that the way to avoid these calcium cancer problems is to take time release calcium and mentioned that yogurt and other foods are effectively time release. Our son separately found research that shows that the human body can only absorb about 500 mg at a time which meant that the large doses we were taking were being wasted regardless of whether they caused cancer.
Note also that one needs to read
"Why Most Published Research Findings Are False"
by John P. A. Ioannidis. Research and research bias are much much more complicated then they appear on the surface. Follow the money is only partly true, there is the desire to do good and there is the problem that sometimes researchers already know the correct answer. And of course there are all sorts of hidden biases that the researcher may not even know that they have.
The summary also mentioned "So, it appears that people who take dietary supplements are likely to live a longer and healthier life for reasons that are unrelated to their supplement use." However, they made a number of assumptions about cause and effect to draw this conclusion. They did not have control group of identically educated and exercising people taking supplements and taking placebo for ten years. Every statin study (before the EU changed its rules to require more openness in "double blind" controlled studies) showed 50% reduction in mortality for taking statins. None since then have. None of the early statin studies showed strong links between statins and diabetes. Now, this VA study does: "
https://www.research.va.gov/currents/spring2015/spring2015-19.cfm "
Also,
Analyzing tests and studies is always a problem. You have to look to see if the study design was right, you have to determine if the study subjects/locations were correctly selected according to the study design, you have to determine if the statistics used to analyze the study were correctly used and if the math in them was accurate.
Above all, you need to avoid confirmation bias such as happened with results of the 55 mph speed limit on 65 mph interstate in the USA. It was supposed to have caused declines in fatalities during the Carter administration. Accident fatalities on roads with 45 mph and 55 mph before the 55 mph speed limit reduction turned out later to have the same percentage accident fatality reduction per vehicle mile traveled as the high speed roads. Also, numerous "eating eggs causes heart disease" studies were done before the egg farmers study. The study sponsored by the egg farmers showed that egg farmers that ate 6 or more eggs a day had the same rate of heart disease as the non egg farmers. This difference was probably a confirmation bias problem as all the previous researchers knew that eggs had cholesterol and that cholesterol caused heart disease. A land grant university set out to prove the egg farmers study wrong and when it could not, examined the other studies and found that they all had statistics problems in them.
Also, numerous studies showed that ulcers were caused by too much tension at work or at homea. Then that nutcake Australian doctor drank the bacteria he believed caused ulcers at a medical conference, got an ulcer and cured it with antibiotics. They finally published his paper. My father was cured of his twenty year long ulcer the month after publication of the study. His doctor told him, well, the guy may be a nutcake or a quack but the antibiotics can't hurt you that much.
Hidden assumptions/study design factors, of which the study designer is unaware, can radically influence study outcomes An example was when two independent studies on whether coffee drinking influenced heart disease both involved ten thousand patients for several years and came up with diametrically opposite results. Both study’s results were incontestable, with a power of around .001 of being wrong. However, one study used exclusively percolated coffee and the other used exclusively filtered coffee and neither study design team realized that they had an extra variable in their study.
“Check the Numbers-The Case for Due Diligence in Policy Formation” by B.D. McCullough, “The Flaw of Averages” by Jeff Danziger, and “Studying a Study and Testing a Test: How to Read the Medical Evidence” (Core Handbook Series in Pediatrics) by Richard K. Riegelman.
So, be careful how you analyze and trust survey studies. And remember that large double blind controlled studies over 10 or more years are inherently very very expensive which is probably why you do not see very many being successfully done.
Walk in Peace,
ScribeWithALancet,
David
Sorry, my preretirement job was in research, we had a saying that if 90% of your research did not fail, you were not doing real research. Substitute the percentage of your own choice.