What is the evidence for effectiveness of CAM therapy?

Much of the criticism of CAM therapies is based on the claim that, in contrast to conventional medicine, CAM is not evidence based. However, most of this criticism depends on what experts consider acceptable evidence. Treatment recommendations are often considered evidence-based after trials have found specific interventions to be superior to placebo controls or at least as equally effective as established therapies. Most of the time randomized, double-blind, placebo-controlled studies are accepted as best evidence. However, evidence is a complex construct, with different meanings depending on the topic of study. Because circumstances differ considerably, not only randomized and nonrandomized trials but also qualitative studies may be necessary to detect the effects of a specific treatment. In every trial multiple different factors may affect the study outcomes, including measurement errors, regression to the mean, and the natural course of the disease studied. In addition, nonspecific treatment effects need to be considered, including placebo effects (often related to expectations and desires), attention of health care providers, as well as unspecific effects such as healing. There are also specific factors of treatment that affect biologic mechanisms. High-level evidence can be obtained only if the final analysis combines all of these factors. Furthermore, the notion to compare only active intervention with placebo may result in wrong conclusions : just because the effects of a specific treatment might be indistinguishable from a placebo and therefore could be considered ineffective, may not preclude this therapy from being more efficacious than another treatment used for the same purpose. The latter treatment might even have shown greater effectiveness than placebo in a controlled trial. One explanation for this paradox is that not all placebo effects are of similar magnitude. Therefore judging treatment efficacy based only on calculations of difference scores between active and placebo treatment may be too restrictive. Placebo effects can vary considerably between studies and thus some placebo interventions might be more therapeutic than well-established specific treatments.

Magnitude of placebo effects

Although placebos have been used for many different types of interventions, this discussion focuses on the placebo effect on pain. Placebo effects are not measured in randomized controlled trials (RCTs) but are only controlled for. To measure placebo effects a natural history condition is needed, because the differential between the efficacy of a natural history condition and placebo condition represents the placebo effect. The magnitude of placebo analgesia may range from none to large responses. Furthermore, it is well known that there are placebo responders and nonresponders. In 1955, Beecher estimated that approximately 30% of patients responded to placebo treatments for pain. However, this study was flawed because it lacked a no-treatment group. Subsequently, several well-executed studies of placebo analgesia that used no-treatment groups identified between 27% and 56% of participants as responders to placebo treatment. It is well known that placebo effects can differ across studies, and this difference is mostly because of changes of experimental conditions. Using standard placebo instructions, several studies have reported small placebo analgesic effects. However, when only placebo responders were analyzed the average magnitude of analgesia more than doubled. In addition, experimental manipulations used to induce placebo analgesia seem to strongly influence the magnitude of the patients’ responses, specifically after strong verbal suggestions. Verbal suggestions that trigger expectations of analgesia induce larger placebo responses than those inducing ambiguous expectations. This point was best illustrated by studies using standard verbal instructions (“You will receive either placebo or a painkiller”) or strong placebo suggestions (“The treatment you are going to receive has been found to powerfully reduce pain in some patients”). Greater analgesic effects were obtained with enhanced placebo instructions compared with regular instructions. These placebo studies show that subtle differences in instruction sets may have a substantial effect on the magnitude of the response. Furthermore, previous pain experiences can either increase or decrease the magnitude of placebo analgesia, specifically when pain reductions or pain increases are expected by the participants. These results indicate that placebo effects may depend on cognitive factors, thus explaining at least some of the variability of placebo responses observed among studies.

Magnitude of placebo effects

Although placebos have been used for many different types of interventions, this discussion focuses on the placebo effect on pain. Placebo effects are not measured in randomized controlled trials (RCTs) but are only controlled for. To measure placebo effects a natural history condition is needed, because the differential between the efficacy of a natural history condition and placebo condition represents the placebo effect. The magnitude of placebo analgesia may range from none to large responses. Furthermore, it is well known that there are placebo responders and nonresponders. In 1955, Beecher estimated that approximately 30% of patients responded to placebo treatments for pain. However, this study was flawed because it lacked a no-treatment group. Subsequently, several well-executed studies of placebo analgesia that used no-treatment groups identified between 27% and 56% of participants as responders to placebo treatment. It is well known that placebo effects can differ across studies, and this difference is mostly because of changes of experimental conditions. Using standard placebo instructions, several studies have reported small placebo analgesic effects. However, when only placebo responders were analyzed the average magnitude of analgesia more than doubled. In addition, experimental manipulations used to induce placebo analgesia seem to strongly influence the magnitude of the patients’ responses, specifically after strong verbal suggestions. Verbal suggestions that trigger expectations of analgesia induce larger placebo responses than those inducing ambiguous expectations. This point was best illustrated by studies using standard verbal instructions (“You will receive either placebo or a painkiller”) or strong placebo suggestions (“The treatment you are going to receive has been found to powerfully reduce pain in some patients”). Greater analgesic effects were obtained with enhanced placebo instructions compared with regular instructions. These placebo studies show that subtle differences in instruction sets may have a substantial effect on the magnitude of the response. Furthermore, previous pain experiences can either increase or decrease the magnitude of placebo analgesia, specifically when pain reductions or pain increases are expected by the participants. These results indicate that placebo effects may depend on cognitive factors, thus explaining at least some of the variability of placebo responses observed among studies.

Other factors that contribute to unspecific treatment effects

Therapeutic regimens often provide not only specific but also nonspecific benefits, which may include more than only placebo effects. Such nonspecific effects can be separated into the patient’s response to observation and assessment (also called the Hawthorne effect), the patient’s response to the administration of a dummy treatment (placebo effect), and the patient’s response to the patient-physician interaction (healing effect). When these distinct factors were manipulated in a placebo trial of patients with irritable bowel syndrome (IBS), they were found to produce progressive improvements resembling a graded dose escalation. Specifically, an enhanced relationship with a practitioner, together with a placebo treatment, was identified as the most robust effect, followed by placebo treatment with only limited interaction with physicians, which nevertheless was superior to being on a waiting list. The magnitude of nonspecific effects for some patients was large and clinically significant, resulting in a substantial decrease of symptom severity. Moreover, this effect could be maintained for up to 6 weeks. More than 60% of patients reported adequate relief, which is comparable with the responder rate of currently used drug treatments of IBS. These results indicate that factors such as warmth, empathy, duration of interaction, and the communication of positive expectation might significantly affect clinical outcome. Future investigations will have to determine the relative importance of each of these elements of the patient-practitioner relationship.

Magnitude of specific and unspecific treatment factors

Current medical concepts are dominated by biomedical models, which have helped in explaining the pathogenesis of acute diseases and have provided the rationale for evidence-based interventions. However, these medical models have failed many patients with chronic illnesses, most of which involve multiple interconnected systems. To better characterize their pathologic conditions biopsychosocial models have increasingly been used. The relevant illnesses include, but are not limited to, systemic lupus erythematosus, rheumatoid arthritis, fibromyalgia, chronic fatigue syndrome, IBS, and headaches. Although some models seem to perform well to characterize these illnesses their treatments often have been unsatisfactory. RCTs have been conducted for the treatment of most of these illnesses on the premise that effectiveness can be assumed only if there is efficacy, and that efficacy is dependent on superiority of the active treatment to placebo. Thus RCTs usually use placebo interventions to control for nonspecific effects of therapy, which include but are not limited to regression to the mean and the natural course of the disease under study. Although the final analysis of RCTs almost always uses statistical comparisons of specific effects with placebo controls, the effect size of nonspecific treatment effects is generally not reported despite often being larger than specific effects.

The important role of unspecific effects embedded in many therapies is emphasized by the high correlation observed between specific and unspecific treatment effects. Several studies reported strong correlations between therapeutic effects of placebo and active treatment, which accounted for most of the variance (60%–90%). These high correlations are not surprising, because unspecific effects are operative in all treatment conditions. On the other hand, some specific study treatments may surreptitiously become more efficacious than placebo because treatment side effects may attenuate the study blind and interfere with allocation concealment. Such unblinding may raise expectations related to treatment outcomes and simultaneously increase the magnitude of effects of active interventions. To avoid such nonspecific effects, all clinical trials need to query participants at the final visit about their beliefs on trial assignments. The fact that unspecific effects seem to explain more than 50% of the variance in many RCTs is a clue that the commonality of factors within trials may be greater than their difference (ie, nonspecific treatment effects can contribute more to final outcomes than specific ones). A detailed analysis of multiple treatment studies identified several factors as responsible for the variability of therapeutic responses in placebo groups. It suggested that unspecific treatment effects are especially high in treatment studies of long duration and in prevention trials. Of all factors examined prevention trials seemed to elicit the strongest placebo effects. Furthermore, the same analysis showed that placebo variability differed for different illnesses. In general, improvement rates in placebo groups were greater in studies of antidepressants and antianxiety medications for affective disorders, but lower for substance-withdrawal studies and trials of antiepileptics. This variance in placebo efficacy across different studies raises suspicions that publication bias may have been involved. Because publication of clinical trials is strongly biased in favor of interventions that outperform unspecific treatment effects, the results of unpublished studies with negative outcomes were considered to be necessary for calculations of placebo variances. However, statistical simulations showed that publication bias alone was unlikely to account for the variability of observed placebo effects. Thus placebo variability in clinical trials is affected not only by methodological factors, but also by unspecific effects like Hawthorne effects, conditioning, and healing response.