Introduction

Clinical trials provide a framework for evaluating the benefits and harms of interventions and contribute to the evidence base which informs clinical decision-making. They are designed to address at least one specific research question such as ‘Does early treatment with combination therapy improve physical function compared to monotherapy in patients with juvenile idiopathic arthritis (JIA)?’A number of clinical trial design characteristics are possible and the choice of which to adopt will depend on multiple factors including the research question, the phase of evaluation, the condition of interest, the type of intervention and comparator, ethical issues, available resources and logistical constraints.

The randomised controlled trial (RCT), and systematic review of RCTs, provides the highest level of evidence for judging the benefits of treatment . Randomly allocating treatments to patients ensures that groups of patients are comparable and any observed difference in outcome between groups can be attributed to the difference in treatment. Adequate concealment of allocation, whereby the knowledge about which treatment will be allocated to the next patient is concealed, is a further requirement to prevent the potential for selection bias. Comparative clinical trials evaluating the effects of interventions should aim to incorporate both randomisation and allocation concealment whenever possible.

An important consideration for almost any clinical trial is the sample size. If a trial is too small, there may be insufficient data to reliably answer the research question. If a trial is too big, resources are wasted and patients may potentially be exposed to ineffective treatments for longer than is absolutely necessary. Therefore, an estimate of the sample size required to detect a clinically relevant treatment effect of a particular magnitude with a certain level of power (chance of detecting a true difference) is usually calculated at the design stage. The sample size required will become larger if the power (chance of detecting a true difference) is increased, or if the significance level (chance of incorrectly finding a difference) or magnitude of treatment effect is reduced. For a full description of sample size in clinical trials, see Ref. .

Rare diseases are defined by a prevalence of <200,000 affected individuals in the USA or a condition that affects not more than five in 10,000 individuals in the European Union (EU) . There are an estimated 7000 rare diseases that may affect 30 million EU citizens with approximately 75% of rare diseases affecting children . Despite this, clinical trials to evaluate interventions for a rare disease have been, and remain, too often neglected and clinicians frequently have very limited, or absent, good quality evidence on which to base their clinical decisions. The reason for this paucity can be explained in part by the fact that traditional sample sizes that are estimated during the planning of a clinical trial will often be impracticable for rare diseases. Since the prospect of a comparative clinical trial with a definitive conclusion may therefore be unlikely, unless treatment differences are large, plans to even undertake robust RCTs may be abandoned. Nevertheless, even when adequately powered RCTs may be unfeasible, there is a strong argument that some level of randomised evidence is much better than none. Moreover, provided there is equipoise, such that there is a state of genuine uncertainty over which intervention is better, even small RCTs of existing therapies are likely to be advantageous and should not be viewed as unethical .

By way of illustration, JIA has an annual incidence of approximately 10/100,000 . It is defined as arthritis of unknown aetiology beginning before the child’s 16th birthday and persisting at least 6 weeks, where known causes have been excluded . Major advances have taken place in recent years in the management of JIA, and clinical trials have led to important breakthroughs in optimising interventions and developing a stronger evidence base for its treatment . The impact of international regulatory changes has brought about a sea change in the focus and delivery of clinical trials for this rare disease. Key factors contributed to this success, including the availability of two large international not-for-profit research networks working in close collaboration, validated outcome measures to evaluate response approved already by the European Medicines Agency (EMA) and US Food and Drug Administration (FDA), and the advent of biological agents that have revolutionised JIA treatment . However, despite this progress, challenges remain in matching appropriate trials to growing expectations of patients and clinicians (and indeed regulators) alike . Even more challenging in the field of paediatric rheumatology, for example, are the other much rarer connective tissue disorders including juvenile-onset systemic lupus erythematosus, juvenile dermatomyositis, childhood scleroderma and childhood vasculitis. Here, lack of strong evidence significantly undermines the clinician’s confidence in treating children with often lifelong conditions and significant associated morbidity and mortality .

Clinical trials in rare diseases per se are not really any different from those in other diseases. However, due to the smaller population available, there is a need to carefully consider alternative methodologies that are most appropriate for the rare disease and intervention in question so that trials are designed to maximise efficiency and make the most of what data there may be available, whilst recognising the limitations of results. We will now explore some of the most common approaches for comparative clinical trials, whilst recognising this is not an exhaustive list. Where relevant or available, we will use examples from paediatric rheumatic diseases as illustrative and an overall summary of advantages and disadvantages is displayed in Table 1 . In other cases, we will adopt examples from other clinical scenarios. A full review and comparison of methods are given elsewhere .

Within-patient designs: repeated measures, crossover trial, N-of-1 trial

One approach to improve efficiency is to recognise that clinical measurements taken within a patient are likely to be less variable than measurements taken between independent patients. The reduced variability effectively increases the power, or if power is set to a specific value, reduces the sample size required, a property which can be exploited by increasing the number of measurements taken on an individual over time, or at different locations of the body. As the number of measurements per patient increases, and the correlation between repeated measures reduces, so does the sample size required. Therefore, for rare disease clinical trials, incorporation of repeated measures within the design is a worthwhile consideration and may also provide an insight into the effect of treatment over time. Mowinckel demonstrated that most patients with rheumatoid arthritis record substantially different scores of pain, fatigue and global disease activity over time, and that taking repeated measurements can reduce the between-subject variation. The study showed that taking five measurements per patient in this clinical setting seems to be the optimal which would decrease the number of patients required in a two-armed clinical trial by as much as 22% .

Other examples of trial designs that specifically incorporate within-patient measurements include the crossover trial and the N-of-1 trial. Both of these involve each patient receiving all treatments which are being assessed within the trial, the order of which should be randomised. The benefit is that a patient acts as his/her own control, which improves efficiency. Furthermore, since patients have the opportunity to try both treatments, they can provide a judgement as to which they might prefer.

In a crossover trial with two treatments, experimental and control, an eligible patient would be randomised to group 1 which receives experimental treatment first followed by control treatment, or group 2 which receives control treatment first followed by experimental. Clinical measurements would be taken at the end of each treatment period and the analysis would contrast these measurements within a patient making sure that the non-independence of the paired observations are taken into account during the analysis.

A multicentre, double-blind, placebo-controlled, crossover study was undertaken to investigate the efficacy of methotrexate (MTX) in systemic-onset JIA and extended oligoarticular JIA . The trial publication specifically states that the crossover design was chosen due to the small number of eligible patients. Initial treatment (placebo or MTX) was given over a 4-month period followed by subsequent 4-month period with the alternative treatment (placebo or MTX). Results from the trial suggested that MTX 15–20 mg/m ² given orally once a week was effective for both extended oligoarticular and systemic JIA ( Table 1 ).

The N-of-1 trial is similar in spirit to the crossover trial in that a patient receives all treatments that are being explored in the trial. However, in contrast to the crossover trial, the N-of-1 trial would recruit an individual patient and would expose the patient to multiple periods of treatment. The unit of randomisation is the treatment rather than the patient. To take full account of the information available from multiple N-of-1 trials, a random effects meta-analysis has been proposed to fully exploit the data (see chapter 6 in Ref. ). N-of-1 trials should always be pre-planned and sequences of N-of-1 trials will always be more convincing and much preferred compared to those which are unplanned . The N-of-1 design is most suitable for ‘fast-acting symptomatic treatments and in diseases that quickly return to stable baseline values after treatment’ .

Huber et al. use a hierarchical Bayesian meta-analysis model to combine results from a series of N-of-1 trials in six children, each with three pairs of randomised, double-blinded treatments (amitriptyline 25 mg or placebo). The trial aimed to assess the effect of amitriptyline to relieve pain in JIA in the small subgroup of children that continue to experience pain despite apparent improvements in inflammatory findings. Even with such a small sample size, the study’s findings indicated that there was only a small probability that amitriptyline could provide pain relief in this particular population and that further research of this intervention was not recommended ( Table 1 ).

Crossover trials and N-of-1 trials suffer from similar limitations. They are only applicable for chronic diseases which are stable over time, where response is relatively swift in relation to the treatment time, and where the purpose of a treatment is to alleviate symptoms rather than to cure. The nature of all of the paediatric rheumatic diseases mentioned above, including JIA, is the often relapsing–remitting natural history. All are subject to generally unpredictable disease flares and at times, spontaneous remissions that may make interpretation of outcomes from these trials difficult. Increasingly, outcomes relevant to and expected by patients are long-term outcomes and maintenance of sustained remission over time, difficult to differentiate in crossover trial designs . Despite advances in knowledge, understanding disease mechanisms remains albeit partial and so none of the current treatments subject to clinical trials to date has been targeted at achieving a cure. The advent of biological agents can now selectively block the effects of cytokines, immune effector cells or their cell-to-cell interactions that form the basis of the autoimmune aetiopathogenesis of these disorders . Therefore, as treatment paradigms are constantly changing, expectations of trial designs that can cope with profound modifications of the disease mechanisms involved are needed.

A particular problem with the crossover and N-of-1 trial occurs when a treatment continues to have an effect even after the treatment has been administered. This potential ‘carry-over’ effect of treatment from one period to the next can make interpretation difficult and may require information collected during the second treatment period to be discarded which thus reduces efficiency and the original benefits of this design. The problem can be alleviated by incorporating a washout period where no treatment is administered in between each period. However, sufficient knowledge about the effects of treatment is required to establish how long this washout period should be. For novel agents, this may be difficult to estimate accurately and therefore carry-over effects may creep in making interpretation difficult. In estimating washout periods for starting trials of new biologics in JIA recently, this has tended to utilise a cut-off of five of the drug’s approximate half-lives. In the MTX trial in JIA , a 2-month washout period was assumed to be long enough to minimise the possibility of a carry-over effect. Arguably, the crossover and N-of-1 trial should not be used to assess the effect of treatments with strong carry-over effects. An example here would be the use of B-cell-depleting agents, where the effects can last upward of 6 months minimum before repopulation can be anticipated. Furthermore, it may be unethical to withhold treatment during this washout period where life-threatening or serious organ-related damage might otherwise ensue such as in the severe multi-system connective tissue disorders noted.

Within-patient designs: repeated measures, crossover trial, N-of-1 trial

One approach to improve efficiency is to recognise that clinical measurements taken within a patient are likely to be less variable than measurements taken between independent patients. The reduced variability effectively increases the power, or if power is set to a specific value, reduces the sample size required, a property which can be exploited by increasing the number of measurements taken on an individual over time, or at different locations of the body. As the number of measurements per patient increases, and the correlation between repeated measures reduces, so does the sample size required. Therefore, for rare disease clinical trials, incorporation of repeated measures within the design is a worthwhile consideration and may also provide an insight into the effect of treatment over time. Mowinckel demonstrated that most patients with rheumatoid arthritis record substantially different scores of pain, fatigue and global disease activity over time, and that taking repeated measurements can reduce the between-subject variation. The study showed that taking five measurements per patient in this clinical setting seems to be the optimal which would decrease the number of patients required in a two-armed clinical trial by as much as 22% .

Other examples of trial designs that specifically incorporate within-patient measurements include the crossover trial and the N-of-1 trial. Both of these involve each patient receiving all treatments which are being assessed within the trial, the order of which should be randomised. The benefit is that a patient acts as his/her own control, which improves efficiency. Furthermore, since patients have the opportunity to try both treatments, they can provide a judgement as to which they might prefer.

In a crossover trial with two treatments, experimental and control, an eligible patient would be randomised to group 1 which receives experimental treatment first followed by control treatment, or group 2 which receives control treatment first followed by experimental. Clinical measurements would be taken at the end of each treatment period and the analysis would contrast these measurements within a patient making sure that the non-independence of the paired observations are taken into account during the analysis.

A multicentre, double-blind, placebo-controlled, crossover study was undertaken to investigate the efficacy of methotrexate (MTX) in systemic-onset JIA and extended oligoarticular JIA . The trial publication specifically states that the crossover design was chosen due to the small number of eligible patients. Initial treatment (placebo or MTX) was given over a 4-month period followed by subsequent 4-month period with the alternative treatment (placebo or MTX). Results from the trial suggested that MTX 15–20 mg/m ² given orally once a week was effective for both extended oligoarticular and systemic JIA ( Table 1 ).

The N-of-1 trial is similar in spirit to the crossover trial in that a patient receives all treatments that are being explored in the trial. However, in contrast to the crossover trial, the N-of-1 trial would recruit an individual patient and would expose the patient to multiple periods of treatment. The unit of randomisation is the treatment rather than the patient. To take full account of the information available from multiple N-of-1 trials, a random effects meta-analysis has been proposed to fully exploit the data (see chapter 6 in Ref. ). N-of-1 trials should always be pre-planned and sequences of N-of-1 trials will always be more convincing and much preferred compared to those which are unplanned . The N-of-1 design is most suitable for ‘fast-acting symptomatic treatments and in diseases that quickly return to stable baseline values after treatment’ .

Huber et al. use a hierarchical Bayesian meta-analysis model to combine results from a series of N-of-1 trials in six children, each with three pairs of randomised, double-blinded treatments (amitriptyline 25 mg or placebo). The trial aimed to assess the effect of amitriptyline to relieve pain in JIA in the small subgroup of children that continue to experience pain despite apparent improvements in inflammatory findings. Even with such a small sample size, the study’s findings indicated that there was only a small probability that amitriptyline could provide pain relief in this particular population and that further research of this intervention was not recommended ( Table 1 ).

Crossover trials and N-of-1 trials suffer from similar limitations. They are only applicable for chronic diseases which are stable over time, where response is relatively swift in relation to the treatment time, and where the purpose of a treatment is to alleviate symptoms rather than to cure. The nature of all of the paediatric rheumatic diseases mentioned above, including JIA, is the often relapsing–remitting natural history. All are subject to generally unpredictable disease flares and at times, spontaneous remissions that may make interpretation of outcomes from these trials difficult. Increasingly, outcomes relevant to and expected by patients are long-term outcomes and maintenance of sustained remission over time, difficult to differentiate in crossover trial designs . Despite advances in knowledge, understanding disease mechanisms remains albeit partial and so none of the current treatments subject to clinical trials to date has been targeted at achieving a cure. The advent of biological agents can now selectively block the effects of cytokines, immune effector cells or their cell-to-cell interactions that form the basis of the autoimmune aetiopathogenesis of these disorders . Therefore, as treatment paradigms are constantly changing, expectations of trial designs that can cope with profound modifications of the disease mechanisms involved are needed.

A particular problem with the crossover and N-of-1 trial occurs when a treatment continues to have an effect even after the treatment has been administered. This potential ‘carry-over’ effect of treatment from one period to the next can make interpretation difficult and may require information collected during the second treatment period to be discarded which thus reduces efficiency and the original benefits of this design. The problem can be alleviated by incorporating a washout period where no treatment is administered in between each period. However, sufficient knowledge about the effects of treatment is required to establish how long this washout period should be. For novel agents, this may be difficult to estimate accurately and therefore carry-over effects may creep in making interpretation difficult. In estimating washout periods for starting trials of new biologics in JIA recently, this has tended to utilise a cut-off of five of the drug’s approximate half-lives. In the MTX trial in JIA , a 2-month washout period was assumed to be long enough to minimise the possibility of a carry-over effect. Arguably, the crossover and N-of-1 trial should not be used to assess the effect of treatments with strong carry-over effects. An example here would be the use of B-cell-depleting agents, where the effects can last upward of 6 months minimum before repopulation can be anticipated. Furthermore, it may be unethical to withhold treatment during this washout period where life-threatening or serious organ-related damage might otherwise ensue such as in the severe multi-system connective tissue disorders noted.

Adaptive methods

The EMA defines a study as adaptive if “statistical methodology allows the modification of a design element (e.g. sample-size, randomisation ratio, number of treatment arms) at an interim analysis with full control of the type I error” (the error made when we incorrectly reject the null hypothesis which is in fact true, in other words a false positive). Adaptive designs have the potential to reach conclusions more quickly and more efficiently than traditional designs and are therefore worthy of consideration for clinical trials of rare diseases.

Adaptive randomisation has been defined as a “change in randomization probabilities during the course of the trial to promote multiple experimental objectives, while protecting the study from bias and preserving inferential validity of the results” . Methodologies vary but commonly adapt randomisation probabilities according to patient characteristics (covariates), accumulating treatment allocation data or accumulating response data and are classified as follows by Hu and Rosenberger .

• ○
  

  Restricted randomisation : a randomisation procedure that uses past treatment assignments to select the probability of future treatment assignments, with the objective to balance numbers of subjects across treatment groups (e.g., balanced coin design).

  
  
• ○
  

  Covariate-adaptive randomisation : a randomisation procedure that uses past treatment assignments and patient covariate values to select the probability of future treatment assignments, with the objective to balance treatment assignments within covariate profiles (e.g., minimisation).

  
  
• ○
  

  Response-adaptive randomisation : a randomisation procedure that uses past treatment assignments and patient responses to select the probability of future treatment assignments, with the objective to maximise power and minimise expected treatment failures (e.g., randomised play the winner)

  
  
• ○
  

  Covariate-adjusted response-adaptive (CARA) randomisation : a combination of covariate-adaptive and response-adaptive randomisation procedures.

The use of an appropriate adaptive randomisation scheme may increase statistical power and may increase the chance of patients being allocated a more effective treatment, a property which is desirable for desperately ill patients. One of the earlier examples of a response-adaptive scheme is Zelen’s ‘play the winner’ approach which allocates the next patient to the same treatment as the previous patient if a successful outcome was observed or to the opposite treatment if a failure was previously observed. However, as this approach is prone to selection bias, the ‘randomised play the winner approach’, sometimes referred to as the ‘balls in urn method’, was proposed by Wei andDurham .

One of the most famous examples of the application of the play the winner approach is the Extracorporeal Membrane Oxygenation (ECMO) trial in babies with persistent pulmonary hypertension . The first baby was randomised to ECMO and survived. The second baby was randomised to conventional therapy but died, which meant that the third baby was allocated to ECMO. As the third baby survived the next baby was also allocated ECMO. This continued until eleven babies had been allocated ECMO and all had survived. Results from this trial suggested a large beneficial effect of ECMO which was subsequently confirmed in a larger-scale RCT which also used an adaptive randomisation scheme to balance the ethical and scientific concerns .

Some methods, particularly the response-adaptive approaches, have been criticised as they can in fact lead to less powerful tests under different conditions. In addition, the response-adaptive methods are only suitable if a response to treatment can be measured promptly and within the given recruitment period of the trial so that these data can be used to adapt the ongoing randomisation process. They may also lead to a drift in patient characteristics during the trial which can lead to bias in the treatment effect, and there is the potential for selection bias to be introduced as allocation may become predictable. Furthermore, the operating characteristics of many methods have not necessarily been examined for very small trials, and since the analysis is not as easily interpretable as when fixed randomisation probabilities are used , a careful and balanced consideration should be given to the use of adaptive randomisation in rare disease clinical trials. For a full review of methods, see Rosenberger et al. .

A sequential design is another example of an adaptive trial. In contrast to a conventional trial which would calculate a fixed sample size and analyse response data at the end of the trial, a sequential trial would analyse accumulating data collected on randomised patients throughout the trial and decide whether the trial should be stopped due to futility, efficacy or safety, according to some pre-specified criteria. Analyses may be undertaken continuously after each individual patient or after groups of patients have been entered. As multiple statistical tests are undertaken, the chance of incorrectly rejecting the null hypothesis increases, and a methodology such as the triangular test , which plots triangular stopping boundaries to allow stopping for evidence of a treatment difference or for lack of difference, is needed to adjust interim and final analyses to account for this.

The main attraction of a sequential trial is that, on average, fewer patients are required compared to a conventional fixed sample size design, and as a decision regarding the hypothesis of interest may be reached earlier, fewer patients may be exposed to ineffective or harmful treatments. Possible drawbacks of the sequential trial are that the reduced sample size can mean that less information is available for the analyses of secondary outcomes. Their increased complexity can lead to an increase in resources, they may require the use of specialist trial software and expert statistical advice and they may not be as easily understood as traditional fixed sample size designs. Furthermore, if the time between administering an intervention and measuring the outcome is long relative to the recruitment period, the sequential trial may not be feasible.

Examples of sequential trials in paediatric rheumatology are difficult to uncover. In the adult literature, Huber et al. explored the effectiveness of subcutaneous injections of a cartilage preparation, Articulatio Genus D5 (AG5), in osteoarthritis of the knee using a sequential RCT. Differences between treatments were evaluated each time a patient contributed outcome data using a one-sided triangle test. The trial demonstrated that pain was not significantly superior in the AG5 group compared to placebo after 4 weeks of treatment but did find a higher reduction of pain medication usage on AG5. In a different trial, Piepers et al. randomised 163 adult patients with amyotrophic lateral sclerosis to receive valproic acid or placebo. Recruitment took place over 21 months and the primary outcome was time to death or tracheostomy with permanent invasive ventilation or non-invasive ventilation >23 h/day. The authors quote a total fixed sample size of at least 336 patients for a traditional RCT based on detecting an improvement of 15% with a 5% significance level (one sided) and 90% power. In their sequential trial design, after 163 patients were randomised and 41 patients had reached a clinical end point, the sequential monitoring indicated that the test statistic had crossed the lower boundary and the null hypothesis of no difference between the two treatment arms could be accepted. The sequential trial design and analysis allowed inclusion of half the fixed sample size and the trial could be discontinued approximately 3 years earlier than a traditional fixed sample size RCT.

The flexibility of the adaptive design framework is attractive for rare diseases, particularly as many of the design options can lead to increased efficiency with trials that are smaller or of shorter duration. However, substantial care should be taken to ensure that adaptations are pre-planned and are undertaken using a rigorous methodology. The possibility of interim analyses being inadvertently disseminated is also of concern as this could lead to unintended changes in the participant recruitment which can lead to bias and subsequent difficulties interpreting results. Other possibilities of adaptive design which are not covered here include the ranking and selection design, multi-arm-multistage designs and seamless phase II/III designs, all of which are discussed in detail by Chow et al. and might be attractive in paediatric rheumatology if a number of alternative treatments are available for a particular condition. In the treatment of childhood connective tissue disorders or vasculitis, such approaches may be helpful as currently there are a number of immunosuppressant therapies available for either the induction or the maintenance phases of therapy ( Table 1 ).