Key points

Introduction

Outcomes research, also known as patient-centered outcomes research or comparative effectiveness research (CER), is oriented toward studies of health care practices such as specific medications, behavioral interventions, care delivery systems, and their impact on best outcomes for individual patients. For this research to make a difference, there must be investment in activities to disseminate this knowledge, to implement the findings into practice, and to measure and improve quality of care delivered.

Although this is not a new field of study, there has been increased emphasis and enthusiasm for this line of research as a result of changes in federal legislation and recent substantial funding for CER. The American Recovery and Reinvestment Act of 2009 allocated $1.1 billion to fund CER between the Agency for Healthcare Research and Quality, the Department of Health and Human Services, and the National Institutes of Health (NIH). In 2010, the Patient Protection and Affordable Care Act served to establish the Patient-Centered Outcomes Research Institute (PCORI), a not-for-profit, independent organization funded through the Patient-Centered Outcomes Research Trust Fund (PCORTF) with income from the Treasury and from a fee to Medicare, private insurers, or self-insurance plans. It is estimated that PCORI will receive $3.5 billion up to September 30, 2019.

The spirit and distinguishing characteristics of this research are evident in the definition of CER used by the Federal Coordinating Council for Comparative Effectiveness Research:

• 
  

  CER is the conduct and synthesis of systematic research comparing different interventions and strategies to prevent, diagnose, treat and monitor health conditions. The purpose of this research is to inform patients, providers, and decision-makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances. To provide this information, comparative effectiveness research must assess a comprehensive array of health-related outcomes for diverse patient populations. Defined interventions compared may include medications, procedures, medical and assistive devices and technologies, behavioral change strategies, and delivery system interventions. This research necessitates the development, expansion, and use of a variety of data sources and methods to assess comparative effectiveness.

The emphasis is for delivery of high-quality topics of relevance and importance to patients and families that allows them to make informed decisions about their health care. A central aspect of that evidence is the analysis of the impact of a particular intervention (ie, as defined earlier) on patient-relevant outcomes. These outcomes may include patient-reported outcomes (PROs) of symptoms or function, side effects, economic impact, or impact on social or societal participation. CER thus necessitates high-quality registries for longitudinal patient follow-up and outcome measures that have proved to be valid and responsive to change. Another distinguishing feature is the recognition that there are multiple stakeholders who require this information to make informed health care decisions, including not only clinicians, researchers, and patients but also policy makers and insurers.

Once information has been generated from outcomes research on which interventions or treatments are most effective, it needs to be efficiently distributed in such a way that it can identify in which setting the intervention will be most beneficial. There is a significant time lag in translation of research evidence into clinical practice. Furthermore, there are widespread disparities in care received in which people who might benefit from a treatment do not receive it. The CER definition places emphasis on the mission “to inform.” The quality of delivered care may be monitored and evaluated in part on whether or not patients who would benefit from a given treatment receive it.

Introduction

Outcomes research, also known as patient-centered outcomes research or comparative effectiveness research (CER), is oriented toward studies of health care practices such as specific medications, behavioral interventions, care delivery systems, and their impact on best outcomes for individual patients. For this research to make a difference, there must be investment in activities to disseminate this knowledge, to implement the findings into practice, and to measure and improve quality of care delivered.

Although this is not a new field of study, there has been increased emphasis and enthusiasm for this line of research as a result of changes in federal legislation and recent substantial funding for CER. The American Recovery and Reinvestment Act of 2009 allocated $1.1 billion to fund CER between the Agency for Healthcare Research and Quality, the Department of Health and Human Services, and the National Institutes of Health (NIH). In 2010, the Patient Protection and Affordable Care Act served to establish the Patient-Centered Outcomes Research Institute (PCORI), a not-for-profit, independent organization funded through the Patient-Centered Outcomes Research Trust Fund (PCORTF) with income from the Treasury and from a fee to Medicare, private insurers, or self-insurance plans. It is estimated that PCORI will receive $3.5 billion up to September 30, 2019.

The spirit and distinguishing characteristics of this research are evident in the definition of CER used by the Federal Coordinating Council for Comparative Effectiveness Research:

• 
  

  CER is the conduct and synthesis of systematic research comparing different interventions and strategies to prevent, diagnose, treat and monitor health conditions. The purpose of this research is to inform patients, providers, and decision-makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances. To provide this information, comparative effectiveness research must assess a comprehensive array of health-related outcomes for diverse patient populations. Defined interventions compared may include medications, procedures, medical and assistive devices and technologies, behavioral change strategies, and delivery system interventions. This research necessitates the development, expansion, and use of a variety of data sources and methods to assess comparative effectiveness.

The emphasis is for delivery of high-quality topics of relevance and importance to patients and families that allows them to make informed decisions about their health care. A central aspect of that evidence is the analysis of the impact of a particular intervention (ie, as defined earlier) on patient-relevant outcomes. These outcomes may include patient-reported outcomes (PROs) of symptoms or function, side effects, economic impact, or impact on social or societal participation. CER thus necessitates high-quality registries for longitudinal patient follow-up and outcome measures that have proved to be valid and responsive to change. Another distinguishing feature is the recognition that there are multiple stakeholders who require this information to make informed health care decisions, including not only clinicians, researchers, and patients but also policy makers and insurers.

Once information has been generated from outcomes research on which interventions or treatments are most effective, it needs to be efficiently distributed in such a way that it can identify in which setting the intervention will be most beneficial. There is a significant time lag in translation of research evidence into clinical practice. Furthermore, there are widespread disparities in care received in which people who might benefit from a treatment do not receive it. The CER definition places emphasis on the mission “to inform.” The quality of delivered care may be monitored and evaluated in part on whether or not patients who would benefit from a given treatment receive it.

Registries in pediatric rheumatology

Longitudinal registries with high-quality data are essential to the validity of patient outcomes studies. Clinic-based data sets, in contrast to administrative data sets such as medical insurance claims, allow for collection of information on disease severity such as clinician measures of disease activity and PRO measures.

There is a growing number of national and international registries in pediatric rheumatology, of which only a few examples are described here. The organizing principles, format, frequency of visits, measures collected, disease populations, funding sources, and other features vary across the registries ( Table 1 ). This variation renders aggregation of the data or replication of study findings challenging.

PharmaChild is a network of registries from multiple European Union (EU) countries for the purpose of pharmacovigilance regarding treatments of juvenile idiopathic arthritis (JIA). PharmaChild is funded by the EU and pharmaceutical companies and organized by the Pediatric Rheumatology International Trials Organization. In North America, the Pediatric Rheumatology Collaborative Study Group has maintained postmarketing surveillance registries after completion of randomized controlled trials for JIA treatments.

The example of national, population-based databases shows comparative effectiveness studies under favorable circumstances. Such registries confer advantages such as reduction in bias, full ascertainment of the population avoids selection bias, and ensuring representativeness of the sample. A national registry minimizes censoring and loss to follow-up. Equally important is the standardization of outcomes assessment for patients across different clinic sites.

Studies from Germany, such as the German Etanercept Registry, have described long-term safety and efficacy of etanercept in JIA. Another recent German registry examined the clinical experience with oral versus subcutaneous administration of methotrexate on disease control, as well as adverse effects of treatment. The investigators found no advantage of using subcutaneously injected methotrexate over orally administered methotrexate, despite practitioner debates to the contrary. Is this a question of interest to the patient? A study by Mulligan and colleagues showed the importance of addressing this question as far as implications for health-related quality of life (HRQOL) of patients and caregivers. In the study, methotrexate use in general was described as having a negative impact on HRQOL, but administration in the form of injections was identified as a significant independent predictor of distress.

The Dutch National Biologics Register provides another example of a registry providing longitudinal follow-up of etanercept-treated patients used to address a question likely of general interest to the public, payers, and policy makers: do the treatment benefits of etanercept justify the high cost of the biological medication? The study entailed an economic evaluation comparing estimates of costs and usefulness in the 12 months preceding the medication use and 27 months after initiation. The Health Utility Index Mark 3 was given to most sites that participated (7 of 9), in order to calculate quality-adjusted life years, in addition to routine collection of clinical variables and PROs. It was concluded that the medication was cost-effective. In the United States, it is stipulated that CER focuses on effectiveness of treatments rather than results to be used in policy decisions to minimize costs.

In the United States, the Childhood Arthritis and Rheumatology Research Alliance (CARRA) registry represents 59 pediatric rheumatology centers and more than 6000 patients with JIA (see Table 1 ). The CARRA registry also collects information on multiple childhood rheumatic diseases, including systemic lupus erythematosus, juvenile dermatomyositis, linear scleroderma, vasculitis, and fibromyalgia. One of the purposes of the registry is to identify patients eligible for clinical trials, to conduct CER and long-term safety studies. Cross-sectional studies have characterized a descriptive epidemiology of the study population, including disease and demographic characteristics and prescribing patterns in the United States. Longitudinal follow-up is planned and under way. The CARRA registry overlies an informatics framework based on the Informatics for Integrating Biology and the Bedside and NIH-funded National Center for Biomedical Computing. It is a scalable, federated system whereby individual centers have ownership of their own data and can contribute data to network studies with permission.

The ARChiVe (A Registry for Childhood Vasculitis), based in Canada, is a registry targeting children with ANCA (antineutrophil cytoplasmic antibody)-associated vasculitis, established in cooperation with the CARRA network.

Comparative effectiveness in observational databases creates significant challenges to the validity of findings. Principal among these challenges are bias, of which there are several types, and confounding. For example, confounding by indication occurs when both the study outcome of interest and the treatment selected are related to the severity of disease. A choice of treatment that is believed to be the strongest may thus go to the sickest patients, who are most refractory to treat, and the potential benefit of the treatment of the disease may be underestimated. There are statistical analytical techniques that minimize these challenges, such as use of propensity scores or instrumental variables, but these may not be able to sufficiently overcome the problem and risk of flawed conclusions.

Variation in how patients are treated in clinical practice also poses additional difficulty to analysis and interpretation of observational data. According to clinical practice, geography, and other considerations, there is wide variability in the care that patients receive. There is inconsistency in the way providers may approach patients that makes it difficult to analyze clinical data. For instance, there may be differences across providers in treatment selection, medication dosing, follow-up intervals, whether clinical variables and outcomes measures are assessed and if so which measures are used, missing data, all which introduce significant noise into the database. In the face of such unwarranted variability, even sophisticated statistical techniques are unlikely to detect whether an intervention is effective.

A way to try to minimize variability as well as bias is the use of treatment guidelines, which if followed drive standardization. For example, guidelines may provide specified treatment options and dosing, approach to intensification or taper of medication regimens, standard visit intervals, and assessment schedules that specify clinical laboratory and PRO measures. There are examples of treatment guidelines by various professional organizations, including consensus treatment plans generated by CARRA for systemic arthritis, juvenile dermatomyositis, localized scleroderma, lupus nephritis and polyarticular JIA (Ringold S, Weiss PF, Colbert RA, et al. Childhood Arthritis and Rheumatology Research Alliance Consensus Treatment Plans for New Onset Polyarticular Juvenile Idiopathic Arthritis. Submitted for publication).