Statistics

Both in science, and legal proceedings, statitsics are used as part of the argument. But methamatics has shown to be more difficult and prone to misunderstandings. For an update on the ins and outs Steven Novella discusses the pitfalls in a review of an article by Tom Siegfried.

Update: Some additional points regarding randomized controlled clinical trials (RCT) by Andrew Gelman.

Update II: More from Open Mind explaining:

Statistics works, it does what it’s supposed to do. But it is susceptible to misinterpretation, to false results purely due to randomness, to bias, and of course to error.

Specifically:

One of the dangers inherent in statistical results is over-reliance on the “p-value.” The p-value is the probability of getting the observed result just by random accident, even when there’s no significant effect and the “null hypothesis”

All told, there are many ways for the statistical analysis of experiments to give incorrect results. This may be especially true in medical research, for which the financial incentive is high, the prior probability is often very low, and the sample size may be severely limited by circumstances beyond anyone’s control. But that hardly means that the foundation of statistical analysis is flimsy; that’s just sensationalism. Nor do we need to adhere to an impractically high standard of statistical significance — as desirable as it is, effects are often small and gathering more data (as in clinical trials) can be very expensive and time-consuming, while delays in availability of new treatments can be devastating for a patient with serious disease and few treatment options.

In addition to this explanation of statistics David Gorski and Kimball Atwood use the case of how the aetiology of peptic ulcer disease (PUD) was elucidated to illustrate how plausibility influences medical research. It might be argued that decades earlier than Barry Marshall and Robin Warren, John Lykoudis may have stumbled upon the same interpretation that bacteriae are involved. However, he did so based upon anecdotal evidence and without sufficient evidence to warrant dismissing the then current view that acid was the culprit. Gorski concludes:

I find Lykoudis’s story to be a cautionary tale. Whether he was correct and thus the true “Galileo” of H. pylori, rather than Warren and Marshall or whether he was just another crank, his story demonstrates that we scientists should be very careful to guard against excessive smugness. As has been repeated by many skeptics in many variants over the years, it is not sufficient to claim the mantle of Galileo as a persecuted martyr for science. You must also be right. Even though it is not clear whether, taken in the context of the time, Lykoudis was a crank or a misunderstood physician who was ahead of his time, Warren and Marshall’s vindication of his ideas that PUD is bacterial in etiology reminds us that not all who claim the mantle of Galileo are necessarily cranks. The vast majority usually are, but on very rare occasions we do see a real Galileo.

Another take on statistics can be found at Effect Measure. For the interested reader, and as a note to myself, some general information here, here, here, here.