Too much of nothing

Is more placebo better?

A friend of mine pointed me to the above TED talk, by Ben Goldacre. It's a entertaining presentation with lots of interesting content, although Goldacre's discussion of the placebo effect—"one of the most fascinating things in the whole of medicine" (6:32)—is a little weak. At 6:47, he says:

We know for example that two sugar pills a day are a more effective treatment for getting rid of gastric ulcers than one sugar pill a day. Two sugar pills a day beats one pill a day. And that's an outrageous and ridiculous finding, but it's true.

Notice that the claim is not about pain, but about actually healing the ulcers.

The source of this claim is apparently a 1999 study by de Craen and co-authors titled "Placebo effect in the treatment of duodenal ulcer" [free full text/pdf]. It's a systematic review based on 79 randomized trials comparing various drugs to placebo, taken either four times a day or twice a day depending on the study. (Note that Goldacre refers to twice a day versus once a day; I'm uncertain of the reason for the difference.) From each trial, the authors extracted the results in the placebo group only, obtaining the following results:

The pooled 4 week healing rate of the 51 trials with a four times a day regimen was 44.2% (805 of 1821 patients) compared with 36.2% (545 of 1504 patients) in the 28 trials with a twice a day regimen

This 8% difference was statistically significant, and remained so even when several different statistical models were used.

However, the authors are up-front about a key limitation of the study: "We realize that the comparison was based on nonrandomized data." Even though the data were obtained from randomized trials, none of the trials individually compared a four-times-a-day placebo regimen to a twice-a-day placebo regimen, so the analysis is a nonrandomized comparison. What if there were important differences between the patients, the study procedures, or the overall medical care provided in the four-times-a-day trials and the two-times-a-day trials? The authors discuss various attempts to adjust for gender, age, smoking, and type of comparator drug, but report that this made little difference. But they acknowledge that:

Although we adjusted for a number of possible confounders, we can not rule out that in this nonrandomized comparison the observed difference was caused by some unrecognized confounding factor or factors.

The strength of a randomized comparison is that important differences between groups are unlikely—even when it comes to unrecognized factors. Although the authors go on to consider other possible biases, their bottom line is:

... we speculate that the difference between regimens was induced by the difference in frequency of placebo administration.

These results of this study are intriguing, but they're hardly definitive.