of Traditional Randomized Controlled Clinical Trials in Rheumatology


Pragmatic limitations of clinical trials

1

A relatively short time frame in chronic diseases – sometimes too short to identify important clinical benefits or to recognize loss of efficacy

2

Inclusion and exclusion criteria may restrict eligibility to fewer than 10 % of patients with a particular diagnosis who may be considered eligible for a clinical trial

3

Differences between a medication and a placebo are required to be statistically significant but not necessarily robust – statistical significance may indicate only marginal clinical benefit

4

Clinically important differences may not be statistically significant due to insufficient numbers of patients for statistical power

5

Important variables affecting outcomes other than whether a patient was randomized to a medication versus another medication or placebo may be seen – but generally ignored in clinical trial reports

6

Traditional clinical trials with parallel designs have inflexible dosage schedules and restrict concomitant medications, although a flexible dosage schedule toward a target with multiple medications may provide optimal results

7

Surrogate markers and indices used in clinical trials may be suboptimal measures to detect changes in clinical status or predict important clinical outcomes

8

Rare side effects cannot be identified in most trials

Intrinsic limitations of clinical trials

1

The design can greatly influence results – availability of a control group does not eliminate bias

2

Data are reported in groups – ignore possible substantial variation in groups

3

No absolute criteria for the balance of risk and benefit for the therapy – different individuals may interpret very differently and all be “correct”

4

Loss of a placebo effect in a clinical trial (although gain of more extensive care which may offset and even surpass the usual “placebo effect”)





Eight Pragmatic Limitations of Randomized Clinical Trials in Chronic Diseases


Eight types of pragmatic limitations in chronic diseases are summarized below:

1.

The relatively short time frame of clinical trials in chronic diseases.

A prominent limitation of clinical trials in chronic diseases involves too short a time frame of observation to recognize meaningful clinical trends that develop only over longer periods. For example, a randomized controlled clinical trial in RA was conducted over 48 weeks to compare results of 3 regimens – methotrexate monotherapy, auranofin (oral gold) monotherapy, and a combination of methotrexate and auranofin [90]. No significant differences were found between results with any of these three regimens (Fig. 1) [90].

A304262_1_En_8_Fig1_HTML.gif


Fig. 1
Results of a randomized clinical trial in 297 patients with rheumatoid arthritis treated with auranofin (AUR), methotrexate (MTX), or auranofin plus methotrexate (Combo) [90]. The figure illustrates percentages of patients with ≥50 % meaningful improvement in tender or swollen joints. Final results showed no significant differences between the three groups over one year

A similar conclusion was reported from a far more extensive meta-analysis of 66 clinical trials reported in 1990 concerning the efficacy of disease-modifying antirheumatic drugs (DMARDs) in the treatment of RA [91] (Fig. 2). This meta-analysis included 117 treatment groups: 11 for antimalarial drugs (e.g., hydroxychloroquine), 23 for auranofin, 29 for in efficacy injectable gold, 7 for methotrexate, 19 for d-penicillamine, 6 for sulfasalazine, and 22 for placebo. The meta-analysis indicated no significant differences in efficacy between sulfasalazine, d-penicillamine, methotrexate, and injectable gold (Fig. 2) [91], i.e., that the efficacy of methotrexate for RA was equivalent to hydroxychloroquine, sulfasalazine, d-penicillamine, and injectable gold.

A304262_1_En_8_Fig2_HTML.gif


Fig. 2
Standard composite treatment effect (in standard units). Meta-analysis of 66 clinical trials reported in 1990 concerning the efficacy of DMARDs in the treatment of RA [91]. This meta-analysis included 117 treatment groups: 11 for antimalarial drugs (e.g., hydroxychloroquine), 23 for auranofin, 29 for injectable gold, 7 for methotrexate, 19 for d-penicillamine, 6 for sulfasalazine, and 22 for placebo. All drugs have greater efficacy than placebo in the management of RA, determined according to a composite of grip strength (a measure of effectiveness of grip), tender joint count, and erythrocyte sedimentation rate, adjusted for disease duration, trial length, initial tender joint count, and blinding. In these analyses, no significant differences were seen between sulfasalazine, d-penicillamine, methotrexate, and injectable gold (From Felson et al. [91] with permission)

Results of the meta-analysis did not appear translated into actual clinical practice over 5 years in an observational study of duration of treatment courses of DMARDs in 7 rheumatology practices reported in 1992 [92] (Fig. 3, Panel a). Duration of treatment courses in an incurable chronic disease such as RA can serve as a composite measure of effectiveness and safety of a medication. A formal analysis of estimated duration of continuation of 1,083 courses of 6 DMARDs over 60 months in 477 patients with RA indicated that approximately 80 % of methotrexate courses were continued after 2 years, compared to 50 % of courses of hydroxychloroquine, penicillamine, parenteral gold, and azathioprine and only 20 % of courses of oral gold (Fig. 3, Panel a). After 5 years, approximately 60 % of the methotrexate courses were continued versus approximately 20 % of the hydroxychloroquine, penicillamine, parenteral gold, and azathioprine courses, and virtually no course of oral gold (Fig. 3, Panel a) [92].

A304262_1_En_8_Fig3_HTML.gif


Fig. 3
(a) Estimated continuation of all 1,083 courses of DMARDs in 532 patients with rheumatoid arthritis over 60 months. Differences between methotrexate and all other drugs, as well as between oral gold (auranofin) and all other drugs, are statistically significant (P < 0.001), while differences among other drugs are not significant. (b) Estimated continuation of 477 courses of the initial DMARD used in the same 532 patients over 12 months. Differences between methotrexate versus oral gold (auranofin) are not statistically significant and are considerably less apparent than in A, in which estimated continuation was studied for all courses over 60 months [92]

The data from the observational study were analyzed over only 1 year for the initial 447 DMARD courses, conditions that mimic clinical trials (Fig. 3, Panel b), in contrast to the above analyses of all DMARD courses over 5 years (Fig. 3, Panel a) [92]. Continuation rates of courses of all 6 DMARDs were similar, including no difference between methotrexate versus parenteral versus oral gold (auranofin) (Fig. 3, Panel b), as seen in the clinical trial (Fig. 1).

The absence of statistically significant differences between DMARD courses over 1 year (seen in Fig. 3b) mimics results of clinical trials in Figs. 1 and 2 but differs considerably from the results seen in actual clinical care over 5 years (Fig. 3a). Therefore, results of both the clinical trials and observational study are accurate and “correct.” However, the accurate data in the clinical trials and meta-analysis were not translated into long-term clinical care over 5 years, and the clinical trial results were not applicable to routine clinical care.

These observations suggest caution in interpretation of data from clinical trials to physicians for routine care. Nonetheless, in 2008 (16 years after publication of the report of differences between results of clinical trials and clinical care [90]), a “systematic review” of DMARDs in the principal journal for internists, Annals of Internal Medicine, concluded that there was “moderate evidence that sulfasalazine, leflunomide, and methotrexate were equivalent in efficacy, with no obvious major differences in adverse events and discontinuation rates among these three DMARDs” [93].

This conclusion differs from contemporaneous clinical care in the international QUEST-RA database of many countries (Table 2), in which methotrexate was taken by 83 % of patients, sulfasalazine by 43 %, leflunomide by 21 %, and biological agents by 23 % [94]. These patterns were seen in countries in which patients do not pay for medications [94], so they could be explained only in small part on the basis of costs. A strict methodologist may conclude that the clinicians were in error and not practicing “evidence-based medicine,” since the systematic review concluded that the three agents were similar in efficacy and adverse events. However, if the conclusion of the systematic review were accurate, comparable usage of the 3 DMARDs might be expected in routine care, but that is not seen. These findings again indicate that data from short-term clinical trials may provide less accurate information about long-term results of therapies than long-term observational studies, as a result of limitations of the clinical trial methodology [87].


Table 2
The use of disease-modifying antirheumatic drugs (DMARDs) in the QUEST-RA countries


























































































































































































Country

Delay to start DMARDS, months, median

DMARD exposure years, mean

Selected DMARDs ever taken; percentage of patients in the QUEST-RA study per country

Prednisone (%)

MTX (%)

HCQ (%)

SSZ (%)

LEF (%)

Any biological agent (%)

Argentina

13

3.7

83

68

49

 6

16

3

Denmark

10

7.9

43

85

39

64

11

23

Finland

7

14.4

74

85

74

84

21

17

France

8

9.9

83

86

55

49

42

53

Germany

15

8.4

54

78

30

36

25

29

Ireland

11

6.3

71

92

15

33

24

41

Italy

9

7.1

69

79

42

14

31

26

The Netherlands

5

8.1

26

91

28

35

6

19

Poland

4

7.2

69

87

34

60

18

8

Serbia

11

6.6

88

69

55

17

7

2

Spain

14

7.3

67

82

43

29

34

27

Sweden

12

8.8

66

83

34

62

9

31

Turkey

12

8.9

69

88

27

61

22

7

UK

12

7.9

51

67

39

46

4

16

USA

9

7.9

77

85

49

12

19

33

Total

9

8.1

66

83

41

43

21

23


Adapted from Sokka et al. [94]

The highest percentage for each drug is indicated in bold and the lowest in bold italics

DMARD disease-modifying antirheumatic drug, HCQ hydroxychloroquine, LEF leflunomide, MTX methotrexate, QUESTRA Quantitative Standard Monitoring of Patients with Rheumatoid Arthritis, SSZ sulfasalazine

Limitations of a short time frame also are seen in a trial conducted in patients with polymyositis to compare therapeutic efficacy of a combination of prednisone plus azathioprine versus prednisone monotherapy (plus placebo) [95, 96]. The initial report concerning this clinical trial indicated no differences between the two groups after 3 months of treatment, according to three measures, i.e., days to normalize the creatinine phosphokinase (CPK) muscle enzyme, change in the muscle strength score, and reduction of inflammation on the muscle biopsy [95]. The authors concluded that “in a controlled, prospective, randomized, double-blind study…azathioprine does not afford any therapeutic advantage when used in addition to accepted prednisone dosages in the initial management of polymyositis” [95].

The randomized controlled trial was then continued further to 3 years. After 3 years (Table 3), improvement according to functional grade disability was significantly greater for patients treated with the combination of prednisone plus azathioprine versus those treated with prednisone monotherapy. In retrospect, differences were seen after 1 year according to functional status, but not according to CPK, muscle strength, or muscle biopsy. The authors concluded that “longer follow-up (3 years) has shown that the group given prednisone plus azathioprine has improved more with respect to functional disability; this group also requires less prednisone for disease control” [96].


Table 3
Comparison of results in treatment of polymyositis with prednisone + azathioprine versus prednisone + placebo over 3 years according to functional grade disability [96]


























 
Functional grade disability

Treatment

Onset

1 Year*

3 Years*

Prednisone + azathioprine

4.5

3.0

2.1

Prednisone only

4.1

3.6

3.0


Adapted from: Bunch [96]

*p < 0.01

These observations illustrate two important principles regarding analyses of treatments in a rheumatic disease: (a) Recognition of the possible advantages of combination second-line therapy may require periods of years, rather than months. (b) Differences in results of two treatments may be apparent according to measures of functional disability, rather than laboratory or biopsy data, as discussed below (see point 7 concerning surrogate markers). These principles may be relevant to studies of all rheumatic diseases.

A striking example of the importance of a long time frame in a clinical trial to assess treatment of a chronic disease is seen in a trial designed to prevent renal failure in patients with SLE nephritis using several treatment regimens, including prednisone monotherapy versus combinations of prednisone with azathioprine and/or cyclophosphamide (Fig. 4) [97]. Substantial advantages to cyclophosphamide plus prednisone were seen over 10 years, with preservation of renal function in about 90 % of patients versus only about 30 % of patients treated with prednisone monotherapy (Fig. 4). These results established cyclophosphamide as the standard of care for SLE nephritis for at least two decades in the 1980s and 1990s. It is not widely recognized, however, that even after 4 years, renal function was preserved in more than 90 % of patients in all groups, i.e., prednisone monotherapy appeared as effective as the combination with cyclophosphamide (Fig. 4).

A304262_1_En_8_Fig4_HTML.gif


Fig. 4
Probability of maintaining life-supporting renal function in long-term randomized clinical trials of 72 high-risk patients with active SLE nephritis, according to treatment group: PRED prednisone, AZA azathioprine, POCY oral cyclophosphamide, AZCY combined oral azathioprine and cyclophosphamide, IVCY intravenous cyclophosphamide [97]

As a result of this clinical trial, combination therapy with cyclophosphamide plus prednisone became the standard of cate for SLE nephritis over the next two decades. However, if this trial had been conducted over only a 3-year period or less, as is the case in more than 98 % of randomized controlled trials in rheumatology, it would have been concluded that no advantage is seen to the combination of prednisone plus cyclophosphamide over prednisone monotherapy! Only a clinical trial conducted in relatively asymptomatic individuals with support from the intramural program of the United States National Institutes of Health (NIH) allowed 10 years of observation, which would not be possible at most clinical settings to establish a new standard of care.

 

2.

Inclusion and exclusion criteria may restrict eligibility to fewer than 10 % of patients with a particular diagnosis who may be considered eligible for a clinical trial.

In theory, all individuals with a particular diagnosis should be eligible to participate in a clinical trial. This goal is more likely to be met in a short-term trial of an antibiotic to eradicate an infectious agent, rather than for a longer-term (but usually not long enough) trial of a medication that affects primarily mammalian cells. In practice, however, all clinical trials have inclusion and exclusion criteria, designed to ensure that a relatively homogeneous group of patients with sufficient disease activity and without severe confounding comorbidities are studied to document improvement.

In many, if not most, instances, inclusion criteria are rather stringent, so that only a small fraction of patients are eligible for the trial. For example, inclusion criteria in the Anti-TNF therapy in RA with concomitant therapy (ATTRACT) trial of infliximab – the first trial reported of a biological agent in RA – were found to exclude 95 % of patients seen in 2000 in the author’s clinical setting [98] (Fig. 5). The three inclusion criteria were six swollen and six tender joints, met by only about one-third of patients; morning stiffness of 45 min and elevated ESR or CRP (2 of 3), met by only half the patients who had six tender and swollen joints; and a methotrexate dose greater than 12.5 mg per week which was met by only one-third of these patients. Cumulatively, these three basic inclusion criteria allowed only 5 % of RA patients to be eligible for this trial [98] (Fig. 5). Similar data have been reported in other reports [99, 100].

A304262_1_En_8_Fig5_HTML.gif


Fig. 5
Analysis of patients with rheumatoid arthritis (RA) who were potential participants in the ATTRACT (anti-tumor necrosis factor α trial in rheumatoid arthritis with concomitant therapy) trial of infliximab plus methotrexate versus methotrexate monotherapy. Of the 152 patients in this consecutive patient cohort, 12 did not have a joint count recorded and another 2 patients were taking etanercept or infliximab at the time of the first joint count and would therefore have been ineligible for the ATTRACT study. Thus, 138 patients were analyzed for meeting the inclusion criteria of the ATTRACT trial: ≥6 tender joints and ≥6 swollen joints; 2 of the following 3 – morning stiffness of ≥45 min, ESR of ≥28 mm/h, or CRP of ≥2 mg/dl; and methotrexate (MTX) dose of ≥12.5 mg/week [98] (From Sokka and Pincus [98] with permission)

All clinical trials also list exclusion criteria, as many variables other than assignment to an intervention or a placebo may affect possible outcomes, such as high age, low education level, low or high disease severity, comorbidities, organ damage, fibromyalgia, previous and concomitant interventions, and many others. Exclusion criteria also restrict entry into the trial to certain possible subjects, in an effort to isolate observed differences to the treatment, and reduce effects of confounding variables.

In theory, the process of randomization should allow adjustment for other variables that might affect the results of a clinical trial. However, in practice, extensive exclusion criteria are common – and compromise the generalizability of results to all patients. For example, many clinicians would treat a patient with RA who is older than 80 years and has a history of breast cancer in remission for 20 years with a biological agent, the efficacy of which was documented in a clinical trial that excluded people who met both criteria for age and comorbidity.

 

3.

Differences between a medication and a placebo are required to be statistically significant but not necessarily robust – statistical significance may indicate only marginal clinical significance.

A clinical trial that includes large numbers of patients may indicate that marginal clinical differences are statistically significant. For example, hundreds of clinical trials conducted during the 1970s and early 1980s indicated that various nonsteroidal anti-inflammatory drugs (NSAIDs), such as ibuprofen, naproxen, piroxicam, and diclofenac, led to improvement in RA patients in the number of tender or swollen joints.

The data were highly statistically significant in clinical trials which included relatively large numbers of patients. However, NSAIDs provided only marginal benefits to most patients [101], although a few individual patients experienced great benefit with each of the medications. NSAIDs often are not used at all, or only on an “as needed” basis in contemporary management of RA, although efficacy was documented in dozens of clinical trials.

 

4.

Clinically important differences may not be statistically significant due to insufficient numbers of patients for statistical power.

Many important rheumatic conditions are seen in relatively small numbers by any individual health professional. Furthermore, enrollment in a clinical trial may be limited by inclusion and exclusion criteria. For example, during the early 1970s, four randomized controlled clinical trials were conducted in patients with SLE nephritis to compare mortality with combinations of prednisone plus azathioprine versus prednisone monotherapy (plus placebo). Two trials of Donadio et al. [102] and Hahn et al. [103] indicated no significant differences with combination versus prednisone monotherapy, while two others of Sztejnbok et al. [104] and Cade et al. [105] indicated lower mortality in patients treated with a combination of prednisone and azathioprine versus prednisone monotherapy (Table 4). Differences in results may be explained in part by differences in patients selected for the trials, as the two studies in which an advantage was seen to the combination included more patients with diffuse proliferative glomerulonephritis, the type of SLE nephritis with the poorest prognosis.


Table 4
Analysis of mortality in four randomized controlled clinical trials in SLE nephritis in which treatment with prednisone + azathioprine was compared to prednisone only
















































 
Randomized controlled trials

Trial characteristics/results

Sztejnbok et al. (1971) [104]

Cade et al. (1973) [105]

Donadio et al. (1974) [102]

Hahn et al. (1975) [103]

Period of observation

3 years

4 years

3 years

2 years

Prednisone monotherapy: % 4-year mortality (in N patients)

32 % (19)

73 % (15)

0 % (9)

30 % (13)

Prednisone + azathioprine: % 4-year mortality (in N patients)

0 % (16)

46 % (13)

0 % (9)

18 % (11)

Difference statistically significant?

Yes

Yes

No

No

Number with diffuse proliferative glomerulonephritis/total number

24/35

28/28

7/16

14/24


Adapted from Sztejnbok et al. [104], Cade et al. [105], Donadio et al. [102], and Hahn et al. [103]

Many individual trials in rheumatic diseases do not have sufficient statistical power to provide statistically significant conclusions. Such limitations in individual clinical trials in SLE nephritis have been overcome in part by a pooled analysis performed by Felson et al. (Fig. 6) [106]. The pooled analysis of eight studies indicated clear statistically significant advantages to combinations of corticosteroids plus immunosuppressive therapy versus corticosteroids alone in treatment of SLE nephritis (Fig. 6). Enhanced statistical power provided by a pooled analysis may overcome in part limitations of small numbers in individual clinical trials.

A304262_1_En_8_Fig6_HTML.gif


Fig. 6
Rate of renal deterioration in SLE nephritis patients treated with steroids alone and in those treated with a combination of immunosuppressive drugs and steroids. Each point represents 1 of 8 studies in a pooled analysis performed by Felson et al. [106]

Although “power calculations” are based on sophisticated mathematical computations, implying substantial precision, in actual fact, they must be based on estimates, which may involve incorrect assumptions. Furthermore, many rheumatic diseases are quite unusual, and it may be difficult to identify a sufficient number of patients to participate in the study, even if power calculations are valid. Therefore, many clinical trials may not show an effect simply because there is insufficient statistical power. Enhanced statistical power may be provided by a meta-analysis of many clinical trials, which may overcome in part limitations of small numbers in individual clinical trials [91], but even meta-analysis cannot overcome a short time frame, exclusion criteria, etc., as noted above.

 

5.

Important variables affecting outcomes other than whether a patient was randomized to a medication versus another medication or placebo may be seen – but usually ignored in reporting of the clinical trial.

The basic design of the randomized controlled clinical trial is focused on identifying differences in results using one intervention versus another or a placebo, and reports of results naturally emphasize this comparison. However, in some trials, outcomes are affected more by variables other than whether a patient was randomized to a drug versus another medication or placebo.

One example of this phenomenon is seen in the Beta-Blocker Heart Attack Trial (BHAT) study, designed to compare treatment with a beta-blocker medication, propranolol, versus placebo, to prevent death from a second heart attack in people who had suffered a recent heart attack [107]. The trial documented that propranolol was more effective than placebo (Fig. 7). However, the patients’ level of formal education – a surrogate for self-management, life stress, and social support – was associated with greater differences than medication versus placebo (Fig. 7) [108]. Of note, recognition of differences according to educational level is not nearly as widely known as differences according to the medication versus placebo.

A304262_1_En_8_Fig7_HTML.gif


Fig. 7
Life-table cumulative mortality curves in the Beta-Blocker Heart Attack Trial (BHAT) according to (a) placebo treatment, (b) propranolol treatment, (c) education level, and (d) life stress and social isolation [108]

Another example of a clinical trial in which other variables were more significant than differences between the medication and placebo involved analysis of clofibrate versus placebo to reduce lipid levels in cardiovascular disease [109]. The 5-year mortality of patients treated with clofibrate was 20 % compared to 21 % in patients treated with placebo, a nonsignificant difference [109] (Table 5). However, 5-year mortality of patients randomized to clofibrate who adhered to their prescriptions was 15 % versus 24 % in nonadherents and virtually identical in patients randomized to placebo 15 % in adherents to placebo versus 28 % in nonadherents (P <0.0001 for adherent vs. nonadherent within each treatment arm) (Table 5). These data indicate that adherence to a treatment regimen was far more powerful to explain a reduction in mortality than whether or not patients were assigned to a lipid-lowering medication versus a placebo.


Table 5
Five-year mortality in patients given clofibrate or placebo, according to cumulative adherence to protocol prescription [109]







































Adherencea

Treatment group

Clofibrate

Placebo

N

% mortalityb

N

% mortalityb

<80 %

357

24.6 ± 2.3 % (22.5 %)

882

28.2 ± 1.5 % (25.8 %)

≥80 %

708

15.0 ± 1.3 % (15.7 %)

1,813

15.1 ± 0.8 % (16.4 %)

Total study group

1,065

18.2 ± 1.2 % (18.0 %)

2,695

19.4 ± 0.8 % (19.5 %)


aA patient’s cumulative adherence was computed as the estimated number of capsules actually taken as a percentage of the number that should have taken according to the protocol during the first 5 years of follow-up or until death (if death occurred during the first 5 years)

bThe figures in parentheses are adjusted for 40 baseline characteristics. The figures given as percentages ±1 SE are unadjusted figures whose SEs are correcting to within 0.1 unit for the adjusted figures

 

6.

Traditional clinical trials with a parallel design have inflexible dosage schedules and restrict concomitant medications, although a flexible dosage schedule toward a target with multiple medications may provide optimal results.

Contemporary trials designed for registration of new therapies require that a new medication have statistically significantly greater efficacy than a placebo, with an acceptable profile for adverse events, in a parallel design [110]. This type of clinical trial may provide unequivocal documentation that a therapy under study is superior to placebo.

However, this parallel design does not allow testing combinations of therapies, which are recognized increasingly as optimal for most patients with inflammatory rheumatic diseases [111]. One approach to overcome this limitation involves a “strategy trial,” in which patients are treated with combinations of DMARDs versus monotherapy toward a target, generally a disease activity score – DAS [26] or DAS28 [27] – to indicate low disease activity or remission, with a protocol requiring adjustment of treatment at frequent visits.

Eight “strategy trials” have been reported in RA [112119] (Table 6), all of which documented significant advantages to intensification of therapies based on careful patient monitoring aimed at a target measure versus traditional therapy that was unchanged over longer periods. All eight trial results indicate that a strategy of aiming for low disease activity or remission appears more important than the agent used [120]. A “treat-to-target” strategy is emerging as the standard of care in RA [121]. Similarly, almost all patients with any inflammatory rheumatic disease are treated with combinations of medications, which cannot be studied optimally in clinical trials which restrict dosage and combinations.
Nov 27, 2016 | Posted by in RHEUMATOLOGY | Comments Off on of Traditional Randomized Controlled Clinical Trials in Rheumatology

Full access? Get Clinical Tree

Get Clinical Tree app for offline access