Monday, July 26, 2010

Squeezing the scientific lemon

James Lind, Scottish naval surgeon, in 1747 conducted the first known clinical trial, to test the curative effects of lemon juice on scurvy. Scurvy was the scourge of the navy, causing more deaths than actual combat, or so Lind was said to have said. Although the curative effects of citrus had apparently been known for centuries, Lind was the first to do a controlled test of its efficacy. As described in Wikipedia,
In his experiment he divided twelve scorbutic sailors into six groups. They all received the same diet, and in addition group one was given a quart of cider daily, group two twenty-five drops of elixir of vitriol (sulfuric acid), group three six spoonfuls of vinegar, group four half a pint of seawater, group five received two oranges and one lemon and the last group a spicy paste plus a drink of barley water. The treatment of group five stopped after six days when they ran out of fruit, but by that time one sailor was fit for duty and the other had almost recovered. Apart from that, only group one also showed some effect of its treatment.
BBC Radio's World Service program, Science in Actionhas a running series called "Moments of Genius", and last week's (July 23) contributor chose Lind's invention of the clinical trial as his important moment. Raymond Tallis, philosopher and retired physician, said that not only did Lind's trial mark the beginning of the science of vitamins, but clinical trials are at the very heart of so-called "evidence-based medicine" (so-called when medical decisions are based on evidence collected with the scientific method; some advocates want doctors to use particular kinds of evidence in particular ways).

As Tallis puts it, we all are tempted to deceive ourselves and others, believing in what we think works even without any evidence, so that the advent of clinical trials was essential to modern medicine. For example, the theory of the four humors, that was formally accepted at least as far back as Galen, certainly seemed to be valid according to informal impressions of doctors -- or they talked themselves into it because it was considered to be the proper standard and/or they had no better ideas. Some patients did, or course, recover after being bled, but the evidence supporting this cure was not systematic or controlled by modern standards.

Clinical trials are controlled studies of the effects of a single treatment or other kinds of intervention. So, in the case of bleeding, for example, a group of people with colds would be bleed and the time to recovery would be compared with people with colds who weren't bled.  If they recovered significantly faster (significance defined statistically), than bleeding would be said to be effective.  If not, bleeding would be considered a non-treatment -- and insurance companies wouldn't cover it.  As Tallis said, "Humility is built into the very idea of a clinical trial," and is the opposite of the 'argument from authority' that is another -- very tempting -- way we have of convincing others of the efficacy of our ideas. For centuries, Galen was authority.

Though, he admits that we've still got the High Street chemist selling homeopathic medicines (see, for example, Holly's story of her adventure with homeopathy in an earlier post), evidence-based medicine has made a big difference.

Like Mendel and his peas, Lind was lucky in his choice of treatment. The effect he hypothesized -- recovery from scurvy -- was large and easily demonstrated, and led to a massive difference in the health of sailors who were on the oceans for months at a time. And certainly clinical trials are indeed fundamental to modern medicine, but as we've blogged about many times, when effects are small, or the outcomes are the accumulation of many small factors, or are complex and can be arrived at in numerous ways, experimental methods like clinical trials -- or, say, genomewide association studies (GWAS) -- aren't necessarily so straightforward. Yes, they are better than argument from authority, just on principle, but they don't necessarily lead to medicine based on evidence.

The problem is simple and it is the same one that applies to GWAS and even to evolutionary explanations: it is the idea that only one relevant factor -- be it a drug under test, or a gene, or a purportedly favored trait -- is varying between those given the drug (or cases, or organisms bearing a particular genetic variant thought to be favored by selection) and those not (or controls, or organisms without the genetic variant).

The number of relevant factors dictates the way that these kinds of statistical associations, between supposed cause and its effect, are found. More variables means larger studies that have to identify, and take into account, those variables. Thus, clinical trials are important, but not unambiguous. And here we don't refer to conflicts of interest or subtle biases in diagnosis, case and control choice, and so on, even though we know those only-too-human foibles can enter into such studies.

Even the idea of natural selection is the idea that exposure to a genetic variant, the logical equivalent to a test drug, is good for the individual. Thus evolution is viewed as a kind of clinical trial where those who have the beneficial 'drug' do well, and the others expire.

There are problems with clinical trials, and there have been recent scandals about cover-ups of relevant data, and so on. Journals have seemed complicit, investigators sometimes sponsored by a company that wants to market a drug, and so on.

But even if scandals are avoided, there are other issues that again have to do with the other, unmeasured but relevant variables. Genomic background of cases vs controls, and lifestyle exposures are among them. Lifestyle exposures of people tested in a study today may be relevant to a drug's effectiveness. But those exposures may change so that, in a decade or two, the same drug will have different effects. Likewise, as is well known, racial background can be highly relevant, and sometimes notoriously hard to control in a clinical trial study.

Nonetheless, limes and lemons do prevent scurvy, and Lind is the insightful person whose insight about how to gather persuasive evidence saved countless sailors, and many others.

No comments: