Saturday, December 14, 2013

The 6 Worst Words in Evidence-Based Medicine

Language MattersWriting in the November 2013 edition of the Journal of the American Medical Association, R. Scott Braithwaite, MD, MS, from New York University School of Medicine, comments on a deceptive 6-word phrase often used in evidence-based medicine (EBM) that frequently leads to dangerously false inferences for clinical decision making [Braithwaite 2013]. We further contend that, applied to the interpretation and application of pain research — such as relating to the use of opioids analgesics for chronic pain — those 6 words also can encourage poor quality pain management and inexcusable patient suffering.

What are the offending 6 words? They are quite simply, There is no evidence to suggest…. Braithwaite proposes that this phrase should be banished from the lexicon of EBM and, while this could be important, we also recognize that it could foster uncertainty and doubt that could be discomforting to many professionals and patients in the pain field.

Untested Hypotheses

Braithwaite provides the following statements as examples of nefarious usage of the phrase:

  • “There is no evidence to suggest that hospitalizing compared with not hospitalizing patients with acute shortness of breath reduces mortality.”

  • “There is no evidence to suggest that ambulances compared to taxis to transport people with acute GI bleeds reduces prehospital deaths.”

  • “There is no evidence to suggest that looking both ways before crossing a street compared to not looking both ways reduces pedestrian fatalities.”

As Braithwaite maintains, all of the statements are absurd as a basis for decision making, yet each statement is technically correct since its underlying hypothesis has not been suitably tested to establish contradictory evidence. This presumes a definition of “evidence” that requires formal hypothesis testing in an adequately powered (eg, large sized) and well-designed (eg, randomized, controlled) research study.

Taking this further, based on a prior review article [Smith and Pell 2003], we would add the following statement: There is no evidence to suggest that jumping from an airplane in flight without a parachute as compared with using a parachute is fatal. As the review authors note (somewhat satirically), while there have been anecdotal accounts of persons without parachutes surviving falls from airplanes, it is extremely difficult to recruit subjects for good-quality randomized controlled trials comparing parachute use with no parachute in such circumstances; so, the statement is technically correct, but unproven and misleading.

Based on his observations, Braithwaite proposes that “there is no evidence to suggest” has become a mantra for EBM practitioners in a wide variety of settings. And, he says, rarely is the statement followed by the clarifying aphorism “absence of evidence is not evidence of absence” [also see Altman and Bland 1995] or discussions of more inclusive definitions of “evidence” for affirming the hypotheses in question.

Seeking Clarity & Precision

Braithwaite proposes that, when an intervention potentially may incur significant harm or require large commitments of resources, deciding not to intervene when “there is no evidence to suggest” the favorability of the intervention can be prudent. “However, deciding to intervene when ‘there is no evidence to suggest’ also may make sense,” he writes, “particularly if the intervention does not involve harm or large resource commitments, and especially if benefit is suggested by subjective experience (eg, the qualitative analogue of the Bayesian prior probability).”

SideNote50wBayesian theory applied to medical research is regaining popularity — albeit, it is difficult for most people to understand —  and it provides a mathematical framework for inference or reasoning using probability estimates. The approach can be particularly helpful in judging the relative validity of hypotheses in the face of sparse or uncertain data. While actual calculations can be complex, to evaluate the probability of a hypothesis being “true” an investigator specifies a prior probability — based on current observation/experience, past research, or scientific principles — which is then updated in the light of new, relevant research data to provide a posterior probability (ie, outcome result). An important feature of a Bayesian approach is that it takes into account what already is known or can be estimated, either quantitatively or qualitatively, about the likelihood of research outcomes being valid; if there is absolutely no (zero) prior probability to support a hypothesis, then research outcomes — whether favorable or unfavorable — are usually unlikely to be valid and reliable. In many respects, this might be viewed as a statistical application of the “Bradford Hill Criteria” for establishing cause-effect relationships [as discussed in Part 11 (here) of our series on “Making Sense of Pain Research].

Braithwaite further maintains that a fundamental problem with the phrase “there is no evidence to suggest” is that it is “ambiguous while seeming precise.” What does “there is no evidence to suggest” really mean when used to argue against some intervention?

Does it mean that the intervention has been proven to have no benefit? That some evidence does exist, but it is inconclusive or insufficient? That outcomes are somewhat equivocal, with risks exceeding benefits for some patients but not others? Each has a subtly different meaning affecting decision making; whereas, simply stating “there is no evidence to suggest” circumvents the experience or clinical intuition of healthcare providers. Furthermore, as Braithwaite notes, many decisions are particularly sensitive to patient preferences and, when the favorability of an intervention is unclear, “there is no evidence to suggest” may “inhibit shared decision making and may even be corrosive to patient-centered care.”

According to Braithwaite, most practitioners make patient-centered decisions every day without high-quality (eg, randomized controlled trial) evidence as a guide, and those decisions are not always wrong. Furthermore, principles of EBM make it clear that an evidence-based approach was never intended to entirely exclude information derived from clinical experience and intuition — which amounts to a qualitative prior probability in a Bayesian sense.

He recommends that practitioners and researchers make concerted efforts to banish “there is no evidence to suggest” from their professional vocabularies. Instead, they could substitute one of the following 4 phrases, each of which has clearer implications for decision making:

  1. “Scientific evidence is inconclusive, and we don’t know what is best” (corresponding to an uninformative or ambiguous Bayesian prior probability).

  2. “Scientific evidence is inconclusive, but my experience or other knowledge suggests ‘X’” (corresponding to an informative, qualitative Bayesian prior probability supporting ‘X’).

  3. “This has been proven to have no benefit” (if valid evidence indeed exists to confirm this).

  4. “This is a close call, with risks exceeding benefits for some patients but not for others.”

Braithwaite asserts that each of the 4 statements would lead to distinct inferences for decision making and could improve clarity of communication with patients. Finally, he says, “Informed implementation of EBM requires clearly communicating the status of available evidence, rather than ducking behind the shield of 6 dangerous words.”

False Arguments Over Opioids for Chronic Pain

For quite some time, a very outspoken and opinionated group of healthcare professionals in the United States has been arguing against the long-term use of opioids for chronic noncancer pain, based essentially on the premise “there is no evidence to suggest that the benefits of this therapy outweigh its potential risks.” In fact, going further — by relying on similar logic and bolstered by low-quality, invalid, or misinterpreted evidence — they assert that overwhelming risks negate any benefits. The group also went so far as to petition the FDA to make the labeling of all extended-release (ER) and long-acting (LA) prescription opioids more restrictive [first discussed in an UPDATE here]. Even though the petition’s demands were largely rejected by the FDA in updated product-labeling [see UPDATE here], opioid opponents have persisted in their campaign.

Indeed, it is acknowledged that there is virtually no clinical research evidence of good quality examining the efficacy and safety of opioid analgesics prescribed long-term for chronic pain. And, in their labeling-change mandates, the FDA also requires manufacturers to conduct longer duration trials of ER/LA-opioids, including evaluations of serious risks, such as misuse, abuse, addiction, overdose, and death, as well as the risks of developing increasing sensitivity to pain (hyperalgesia).

Meanwhile, the opioid opponents have been using the current lack of evidence as evidence itself to support what might be called argumenta ad ignoratum, or “appeals to ignorance,” as discussed in Part 12 of our “Making Sense of Pain Research” series [here]. In the absence of any high-quality research evidence to the contrary, the opponents have used their own interpretations of data on opioid-related abuse, addiction, deaths, and other risks to arrive at an artificial Bayesian prior probability of harm — and have successfully foisted fallacious inferences on the public.

Additionally, they are most likely driven by a personal set of prior probabilities — coming from likeminded peers or individual experiences with select patients — that help guide the calculus of their conclusions. Essentially, they have fabricated their own rendition of Braithwaite’s second statement above to claim, “Scientific evidence is inconclusive, but my experience or other knowledge suggests that opioids are ineffective and unsafe in the treatment of chronic noncancer pain.”

However, using similar evidence deficits and prior probabilities informed merely by empiricism (eg, anecdotal observations), there are other important arguments about opioids for chronic pain that can be stated:

  • There is no evidence to suggest that opioid-induced hyperalgesia is a frequent clinical occurrence in human subjects administered opioids long-term for any type of pain, or which patients might be most affected.

  • There is no evidence to suggest that there is an inordinately high incidence rate of de novo, iatrogenic addiction among patients with chronic pain prescribed long-term opioid analgesics.

  • There is no evidence to suggest that significant numbers of patients with chronic pain do not or cannot benefit from opioid analgesia.

Other, similar, arguments could be expressed that cast doubts on concerns about the efficacy and safety of opioids for chronic pain. But, in all cases, such doubts are motivated by uncertainty — or, an “ambiguous Bayesian prior probability” — and a most objective and unbiased premise could be a variation of Braithwaite’s first statement above; “Until there is good-quality evidence available we cannot reach definitive conclusions.” Meanwhile, using a lack of evidence to argue for or against opioids for chronic pain becomes a cruel game of sorts in which nothing is scientifically established and patients who presently do or prospectively could benefit from such therapy are the losers.

Ubi Dubium, Ibi Intellectum

If we accept Braithwaite’s proposal to eschew the use of “there is no evidence to suggest” as a valid argument against a therapy or intervention, it also raises nagging doubts about the legitimacy of rejecting certain questionable modalities for pain management because they have little if any high-quality evidentiary support. A number of complementary and alternative medicine (CAM) modalities immediately come to mind: eg, homeopathy, reflexology, energy-field therapies (eg, Reiki, etc.), biomagnetic therapy, some variations of acupuncture, and others.

In most cases, high-quality clinical trials are absent and we are left with observational or anecdotal evidence at best. With certain approaches (eg, homeopathy, Reiki, and others), there is no presently-known biological rationale or plausibility to serve as a prior probability of efficacy. Still, there are ample examples of patients with pain being helped by each of the treatments — an informative prior probability — even if the outcomes are primarily due to placebo effects. So, should those CAM approaches be rejected outright as worthless on the basis of “there is no evidence to suggest that they are clinically effective for pain”?

Indeed, many critics have made strong, rational arguments for unequivocally rejecting most CAM approaches on the basis of absent or inadequate supportive evidence and/or the lack of biological plausibility [eg, see Science Based Medicine blog]. Despite those contentions, and in view of Braithwaite’s perspective, it would appear that less absolutist and more definitive statements are needed. And, these must not rely primarily on the absence of evidence as evidence against CAM approaches and any prior probabilities must be taken into account, including those based on limited observational or anecdotal data.

Pain-Topics UpdatesIn many cases, prior probability or plausibility may be so low that the respective CAM approach is still deemed ineffective. But, in other instances, this could encourage “suspended disbelief” until further investigation via high-quality research is possible. Many practitioners and patients may be discomforted or irritated by the degree of uncertainty and doubt this tolerates. And, a dilemma may be that, as with the parachute example above, there may never be definitive research to make strictly evidence-based pain management decisions. However, a fundamental theme of these UPDATES, as well as our “Making Sense of Pain Research” educational series has been Ubi Dubium, Ibi Intellectum, or “Where There Is Doubt, There Can Be Understanding” [click to download series PDF].

> Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ. 1995;311(7003):485 [access here].
> Braithwaite RS. EBM’s Six Dangerous Words. JAMA. 2013;310(20):2149-2150 [
access by subscription here].
> Smith GC, Pell JP. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ. 2003;327(7429):1459-1461 [abstract here].

eNotifications Don’t Miss Out… Stay Up-to-Date on Pain-Topics UPDATES
Register [here] to receive e-mailed notifications of new postings.
Or, follow us on Twitter [here] or on Facebook [here].


Myron Shank, M.D., Ph.D. said...

Thank you, Dr. Leavitt, for what may be the clearest non-mathematical explanation of the Bayesian decision-making that I have seen.

Your arguments-by-examples are excellent, but I think that it would be even clearer, if you also made the hidden logical syllogism explicit: "(1) There is no evidence for the effectiveness of long-term opioids for chronic pain. (2) The only possible way that there can be no evidence for the effectiveness of long-term opioids for pain is for long-term opioids to be ineffective for pain. (3) Therefore, long-term opioids are ineffective for pain."

In this format, it is immediately apparent that the implicit (but suppressed) premise that "The only way that there can be no evidence for the effectiveness of long-term opioids for pain is for long-term opioids to be ineffective for pain" is false, so long as there is any other possible reason why the evidence might not exist. Since it is easy to imagine other reasons why the evidence might not exist, the hidden premise is false, the syllogism fails, and the conclusion remains unproven.

Not as obviously, even the first premise is false, since there is an abundance of evidence that long-term opioids are effective for chronic pain, even if it does not (and never will) come from large, high-quality, randomized, double-blind, placebo-controlled studies that are independently sponsored. Even if it were ethically and logistically possible to conduct such an experiment (which it clearly is not), no matter how favorable the results were for opioids, nay-sayers would still say that there was "no evidence" for that opioid at any other dose regimen, for any painful condition other than the one studied, for any population other than the one studied, for any other opioid than the one studied, at any doses other than those studied, or for any duration greater than that which was studied. If, however, the study were broad enough to include every painful condition in every conceivable population with all available opioids at the full range of doses one might ever encounter and for the longest imaginable duration of therapy, results favorable to long-term opioid therapy would still be rejected, because such a study would be too inhomogeneous to be useful.

So what? Even though large, high-quality, randomized, double-blind, placebo-controlled studies that are independently sponsored are ideal for predicting the likely effectiveness of a therapy, they are useless for evaluating the actual effectiveness in an applied clinical situation.

The beauty of the Bayesian approach is that it is unnecessary to prove that all opioids are effective for all dose regimens for all painful conditions in all populations for all durations, in the first place. It is only necessary to show evidence that a specific opioid is likely to have been effective in a specific dose regimen for a specific painful condition in a specific patient for the specific treatment duration that has already occurred. If that is likely to have been true, that the same will also be true for the next interval is a testable hypothesis. Once the likelihood that the hypothesis is true drops to an unacceptable level, one can (and, I would say, should) make a therapeutic change to test an alternative hypothesis (which may or may not include an opioid).

What is an unacceptable likelihood? That depends on the perceived relative costs of Type I (false positive) and Type II (false negative) errors in conclusions. For example, if one believes that the addition of prayer to other therapies might be beneficial, but that it is certainly harmless, then the evidentiary threshold for the effectiveness of add-on prayer therapy can be extremely low; in contrast, the evidentiary standards for the effectiveness of an extremely costly and risky therapy should be correspondingly high.

SB. Leavitt, MA, PhD said...

Thank you, Dr. Shank, for taking the discussion to a further level of understanding. I do try to keep the UPDATES articles as brief as possible, and your comments offer more to think about. --SBL

Mark S. Barletta said...

Maybe we should change the classic phrase "There is no evidence to suggest" to " There is evidence to suggest" that opiates do manage chronic pain in people suffering from non-cancer pain. If there was no evidence to suggest why is it people that suffer from chronic pain find relief with opioids when all else has failed them.
I for one tried all of the above to get my chronic pain under control to no avail. When my Pain Specialist started me on a aggressive pain management program and titrated me up to a level of relief then and only then did I finally get pain relief. Then and only then did my quality of life come back in to play, I was given the pain relief I deserved for so long.

SB. Leavitt, MA, PhD said...

Yes… there is evidence to suggest that opioids are effective for chronic noncancer pain; however, it is mostly anecdotal and not based on high-quality clinical research. The prior probability, in a Bayesian sense, is low. Yet, the same thing could be said of the evidence against opioids for this purpose; so, the only solution is to have better-quality clinical trials --- which the FDA has mandated for ER/LA opioid analgesics, but this will take time.

Janice Reynolds said...

There is no evidence to suggest that chronic pain in a non-cancer setting is (or isn't) different than chronic pain in a cancer setting-so, huh? What they are really saying when they saw there is no evidence to support using opioids in non-cancer related pain is "only people with cancer deserve pain management." All pain is different for each individual and everyone may respond differently to interventions whether medicine, CAM, or interventional. Research tells of likely effectiveness, adverse effects, and problems which may be associated (like the potential for GI bleeds and multiple other dangers in the use of NSAIDs)
I found the explanations very helpful.

Rav said...

In response to Mark Barletta's comment, I would like to point out that the reason why most people (including yourself) who SUFFER from chronic pain benefit from opioids is due to two reasons - firstly, suffering is an experience which is individualised and vary in severity....which affects various aspects of life including mood, affect, sleep etc. Opioids improve suffering by working not on pain but those other aspects of chronic pain which patients find hard to manage without recourse to drugs. Secondly, it has a huge placebo benefit which is the reason why the efficacy vain over time. The efficacy of opioid benefit is still questionable......however until we have a better outlook and approach to managing chronic noncancer pain we will rely on opioids. Regards. Harish

Myron Shank, M.D., Ph.D. said...

@Rav, what is your evidence that the waning of opioid analgesia is due to its dependence upon placebo effect? Tolerance and dependence are both related to dose, duration, and consistency dosing. If tolerance is due to the placebo effect, why is not dependence? Why does tolerance develop in non-humans for which we have no evidence of placebo expectations?

Tolerance can easily be explained in terms of classic down-regulation of receptors, which is readily demonstrated to occur in isolated tissues and even cells. It is associated with a reduction in the number of receptors and decreasing chemical responses to occupation of the receptors. Would you argue that the reduction in receptors and receptor responses that can be observed in tissues and isolated cells is an expression of a placebo effect?

Placebo effects can be very powerful, but they rarely last as long as the effectiveness of opioids usually does.

RachaelP said...

As a Pain Advocate, with current training and research updates, I find the cruel verbiage used to describe the lack of evidence supporting long-term opioid usage for non-cancer pain disturbing. It's only getting worse since this piece was written.

Where are the voices of the majority of pain treatment doctors, who witness the benefits? Oh I forgot, they are mostly frustrated and tired of the changes. They are in CYA mode, bless most of their hearts.

We need a better approach that incorporates the voices of patients, caregivers, doctors, nurses and researchers NOW! We need help now. With 2/3 of suicides being related to pain, who will take the responsibility for the life-long effects experienced by friends and family? When suicide rates start hitting the news as an effect of mistreated/undertreated pain this fall, more patients may view it as the easier option...that epidemic is much scarier in my opinion.

Of course those of us that are educated know why the majority of overdodes have occurred, and it isn't the fault of the true patient, following doctor's orders. We need to fully educate the public. It's time. This article is a good start, but it's reach is not wide enough.