Friday, February 23, 2007

Much medical research a waste of money

due to flaws in the design and carrying out of the studies. The Bayesian Heresy pointed out a story in The Economist which documents the poor quality of medical studies and includes this mind-boggling paragraph:

"Unfortunately, many researchers looking for risk factors for diseases are not aware that they need to modify their statistics when they test multiple hypotheses. The consequence of that mistake, as John Ioannidis of the University of Ioannina School of Medicine, in Greece, explained to the meeting, is that a lot of observational health studies—those that go trawling through databases, rather than relying on controlled experiments—cannot be reproduced by other researchers. Previous work by Dr Ioannidis, on six highly cited observational studies, showed that conclusions from five of them were later refuted. In the new work he presented to the meeting, he looked systematically at the causes of bias in such research and confirmed that the results of observational studies are likely to be completely correct only 20% of the time. If such a study tests many hypotheses, the likelihood its conclusions are correct may drop as low as one in 1,000—and studies that appear to find larger effects are likely, in fact, simply to have more bias."

Observational studies result in valid findings only one-fifth of the time!! And the public is basing its health policy decision making on these studies, many of which get a lot of publicity in the main stream media!!

No comments: