Evidence-based ambiguity

In the last 15 years, evidence-based medicine has taken the world by storm. According to a famous definition, evidence-based medicine is

"the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients."

In this spirit, the evidence supporting many time-honoured practices in medicine has been examined, and in a number of cases found wanting. (For an informative and entertaining look at this, see this slide presentation by former British Medical Journal editor Richard Smith.) The exalted status that expert opinion once enjoyed is waning. Today the cry is "Show me the evidence!" (cf. Jerry Maguire.)

If you'll pardon the pun, the success of evidence-based medicine has been infectious. The prefix "evidence-based" is popping up not just in connection with healthcare: today there is evidence-based education, evidence-based software-engineering, evidence-based librarianship, and the list goes on. The "evidence-based movement" seems victorious.

But there are stirrings of discontent. From the beginning, evidence-based medicine has had its critics. (See this editorial for a balanced account of their objections.) A key issue relates to the ambiguity in the word "evidence". If it means empirical evidence, it would seem that clinical experience, pathophysiological theory, patient values, and expert opinion have no role to play. Alternatively, evidence can be defined broadly: "evidence is anything that establishes a fact or gives reason for believing something" (The Oxford American Dictionary, via this report). But this "colloquial" definition opens the doors so wide as to be useless here. For example, a religious argument might be compelling for the believer, but surely would not constitute "evidence" for the present purposes. For some other interesting perspectives on the definition of evidence in evidence-based medicine, see this essay by Amanda Fullan (an undergraduate student at the time).

In recent years the evidence-based movement has expanded to areas such as public health and policy. In a 2004 essay titled What is Evidence and What is the Problem?, the Acting Executive Director of the American Psychological Association writes

"These days, you can hear the terms “good science”, “evidence”, and “data” a lot in Washington. One of the catch phrases around policy-making circles is “evidence-based”, applied to a host of contents including education, policy, practice, medicine, even architecture. You would think that this would make us all quite happy – at least those who advocate that decisions about policy, social interventions, and future directions be based on data. But, ironically, the new emphasis on evidence-based this and that has been simultaneously welcomed and greeted with raised anxiety levels and red flags of concern."

And the ambiguity of the word "evidence" is even more problematic in this context:

"It is clear that discussions of definitions of evidence, distinctions among kinds of evidence (including scientific data, expert judgment, observation, and theory), and consensus on when to use what, will occupy us for some time."

The Canadian Health Services Research Foundation (CHSRF) has recently grappled with these issues, issuing a report, and holding a workshop. One of the "key messages" from the workshop was that

"Although the literature shows that decision makers work with a colloquial understanding of evidence (often alongside a scientific understanding), some participants felt strongly that the information classified as colloquial evidence should not be called evidence. They acknowledged the importance of this information but suggested finding a substitute term, such as “colloquial knowledge” or “colloquial factors.”"

Finally, the CHSRF adopted the following (rather extended) definition:

"Evidence is information that comes closest to the facts of a matter. The form it takes depends on context. The findings of high-quality, methodologically appropriate research are the most accurate evidence. Because research is often incomplete and sometimes contradictory or unavailable, other kinds of information are necessary supplements to or stand-ins for research. The evidence base for a decision is the multiple forms of evidence combined to balance rigour with expedience—while privileging the former over the latter."

Hmmm ... not entirely convincing, but I see what they're getting at. But where did they get that stuff about coming "closest to the facts of the matter"? I'd say it's either begging the question or using a circular argument.

Epilogue: In their latest newsletter, the CHSRF announce that they've decided to abandon the term "evidence-based":

"Following feedback and discussions at the “Weighing Up the Evidence” workshop in September 2005, the mission of the foundation has been changed to better reflect the emerging realization that research is justifiably only one, albeit very important, input to decision-making."

The new term? Evidence-informed.