Modern science is a wondrous and powerful force. It keeps us warm all winter and cool all summer. It allows us to effortlessly leap entire oceans and continents, transplant living organs, feed seven billion people and probe the distant edges of the universe.
Yet for all of its potential to do good, there are instances where science stumbles. Almost every year, studies appear in the public sphere that are fatally flawed; tragic cases of “bad science” that twist the truth, miss the point or, in some cases, outright lie.
One of the most egregious miscarriages of science in recent memory was published in 1998 by Britton Andrew Wakefield in “The Lancet.” The former surgeon’s paper proclaimed a link between the measles, mumps and rubella vaccine; regressive autism; and chronic intestinal inflammation. The paper was officially retracted in 2010 at the conclusion of Wakefield’s twelve-year marathon hearing before the General Medical Council.
Wakefield’s paper looked at twelve children who suffered from autism and intestinal inflammation, each having received the MMR vaccination. The paper purported to show that previously non-autistic children were developing autism and an associated bowel disorder very shortly after their shots. The paper claimed an average separation of only 6.3 days between the events.
The British Medical Journal published a feature in 2011 exploring the multifarious flaws in Wakefield’s practices and experimental design. Wakefield’s paper was rife with manipulations and misrepresentations – some of the most damning revelations were summarized by the 2011 BMJ feature:
Three of nine children reported with regressive autism did not have autism diagnosed at all. Only one child clearly had regressive autism.
Despite the paper claiming that all 12 children were “previously normal,” five had documented pre-existing developmental concerns.
Some children were reported to have experienced first behavioral symptoms within days of MMR, but the records documented these as starting some months after vaccination.
Patients were recruited through anti-MMR campaigners, and the study was commissioned and funded for planned litigation.
Wakefield’s paper wasn’t the only thing retracted once the truth came to light. The former surgeon was stripped of his clinical and academic credentials and promptly booted from the scientific community.
Unfortunately, the damage was by-and-large already done.
“In 2008, for the first time in 14 years, measles was declared endemic in England and Wales,” reads a second 2011 BMJ editorial about Wakefield’s paper. “Hundreds of thousands of children in the UK are currently unprotected as a result of the scare, and the battle to restore parents’ trust in the vaccine is ongoing.”
It is uncomfortable to consider the scientific community as being vulnerable to deception and distortion, but there are ways the scientific layman can arm himself against bad science.
A November 2013 article in “Nature” offered tips for spotting studies that fail to measure up.
Among them is the advice “bigger is usually better for sample size.” The sample size of a study is the number of cases being looked at. In Wakefield’s study, there were only twelve cases examined. Drawing broad conclusions about millions of people based on a study of twelve individuals is something that ought to raise some eyebrows. Good studies will base their conclusions on hundreds, if not thousands, of cases.
A second point is that “controls are important.” Control groups are treated precisely like the experimental groups, except the item being investigated is left out. In Wakefield’s case, the control group would have been twelve children who had developed regressive autism and bowel disorders without ever getting the MMR vaccine. A control group was conspicuously absent from Wakefield’s paper – one of the first red flags noticed by investigators. Well-structured research papers will make explicit mention of control groups and their impact on the conclusions reached.
A final admonishment offered by Nature is “seek replication.” When researching the strength of a scientific paper, it is wise to see if there have been any follow-up studies. If there have, they can offer insight into how replicable the initial results were. Generally, there will be a trend toward either confirmation or refutation of the initial paper that can denote the quality of the research.
Fortunately for the public, the scientific community is rigorously self-policing. The process of peer-review combined with the practice of replicating studies ensures that fallacious conclusions do not stand the test of time. The world need not depend solely on professional scientists, however. Scandals like the Wakefield paper serve as a clarion call for education. A public that understands good scientific procedure can better appreciate the implications of proper science, and more readily discount crackpot theories. Bad science is out there – discerning minds and educated societies represent the best defense against its depredations.
