Abstract
Measurement of health outcomes in pediatric rheumatic diseases is a critical component of clinical practice and research studies. Measures should include the biological, physical, and psychosocial dimensions of health. Health outcome measures are developed systematically, often using consensus methods. Prior to implementation into practice, health outcome measures must undergo evaluation of measurement properties such as reliability, validity, and responsiveness. There are several health outcome measures available for juvenile idiopathic arthritis, juvenile systemic lupus erythematosus, and juvenile dermatomyositis, many of which are composite measures of disease activity. In addition, tools exist for measuring physical functioning and health-related quality of life. There is increasing focus on the incorporation of patient-reported or parent-reported outcomes when measuring the health state of patients with pediatric rheumatic diseases. Further work is required to determine the optimal health outcome measures and approach for eliciting the patient’s perception of their health state in pediatric rheumatology.
Rationale for use of health outcome measures in pediatric rheumatic diseases
Measurement of health state is critical for understanding the health status of children and youths with rheumatic diseases, for making decisions in clinical practice, and for evaluating and comparing the effect of therapies in clinical research studies. As stated by Lord Kelvin, “when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot, your knowledge is of a meager and unsatisfactory kind.” Due to the complexity of rheumatic diseases, a single laboratory or clinical feature is often not sufficient to capture the complete health state, and thus composite or multidimensional outcome measures are common. Over the past several years, there has been an international collaborative effort to develop and validate standardized health outcome measures for pediatric rheumatic diseases, including measures of disease activity and damage, physical functioning, and health-related quality of life (HRQL) . Most recently, there has been an increased focus on the inclusion of patient and parent perspectives in health outcome measures through the use of parent-reported or patient-reported outcomes (PROs) . PROs are reports coming directly from patients about how they feel or function in relation to a health condition and its therapy without interpretation by health-care professionals or anyone else .
This chapter will discuss the theoretical basis of measuring health outcomes, methods for the development and validation of health outcome measures, and an overview of available measures for common pediatric rheumatic diseases.
Definitions of health outcomes in pediatric rheumatic diseases
Health is described as a state of biological, psychological, and social well-being, and an ability to meet the demands of life. The World Health Organization’s (WHO) International Classification of Functioning, Disability, and Health (ICF) is a conceptual framework for defining, measuring, and classifying health and disability . The ICF is based on the bio-psycho-social model for health, highlighting that a health condition interacts with internal and external factors to determine levels of biological, physical, and psychosocial functioning ( Fig. 1 ). The ICF framework suggests that health outcome measures should incorporate an assessment of the body’s anatomic and physiological functions and structures, physical functioning (activity), and psychosocial functioning (participation), with the consideration that these outcomes are influenced by contextual factors both within the individual (e.g., coping strategies) and the environment (e.g., physical barriers or social attitudes) . According to the ICF structure, disability arises if there is dysfunction at any of these levels. For example, a child with juvenile idiopathic arthritis (JIA) who has swelling and reduced range of motion (ROM) of a joint (biological level) may have difficulty running (physical level) and may not participate in gym class because of fear of injuring himself/herself (psychosocial level). Accordingly, interventions should also be aimed at each level of health: medical treatment to reduce joint inflammation, physiotherapy to improve running, and accommodations at school to allow the child to participate at his/her own pace.
In pediatric rheumatic diseases, the biological level of health is often assessed using measures of disease activity, damage, or severity; thus, these concepts also merit definition. “Disease activity” includes those aspects of a patient’s disease that are potentially reversible, and “disease damage” refers to persistent or permanent changes in anatomy, physiology, and function (and that usually develop from previously active disease or complications of therapy) . Both disease activity and damage contribute to “disease severity,” which has been defined as the total effect of the disease on the individual and often relates to prognosis . Given the multidimensionality of pediatric rheumatic diseases, a challenge exists when some components suggest that the disease is more active, while others suggest it is less active (see discussion below). Currently, pooled and appropriately weighted composite indices are often used to capture the various dimensions of disease activity (see below).
Ideally, in both clinical practice and research settings, disease activity measures should be combined with measures of physical and psychosocial functioning – and perhaps contextual factors, both personal and environmental – to compose a complete health outcome picture. This generally includes the use of PROs, which are self-reports that capture a broad array of the patient experience, including health symptoms (pain, fatigue), physical functioning, mental health, participation in activities, and self-efficacy . PROs can help to provide care that is responsive to patient needs and should also be included as important outcomes in therapeutic clinical trials . One important PRO is quality of life (QOL), which is defined by the WHO as “an individual’s perception of their position in life in the context of the culture and value systems in which they live, and in relation to their goals, expectations, standards, and concerns.” HRQL may be defined as “patient-reported quality of health,” and may refer, in some way, to how QOL is affected by health or illness .
Definitions of health outcomes in pediatric rheumatic diseases
Health is described as a state of biological, psychological, and social well-being, and an ability to meet the demands of life. The World Health Organization’s (WHO) International Classification of Functioning, Disability, and Health (ICF) is a conceptual framework for defining, measuring, and classifying health and disability . The ICF is based on the bio-psycho-social model for health, highlighting that a health condition interacts with internal and external factors to determine levels of biological, physical, and psychosocial functioning ( Fig. 1 ). The ICF framework suggests that health outcome measures should incorporate an assessment of the body’s anatomic and physiological functions and structures, physical functioning (activity), and psychosocial functioning (participation), with the consideration that these outcomes are influenced by contextual factors both within the individual (e.g., coping strategies) and the environment (e.g., physical barriers or social attitudes) . According to the ICF structure, disability arises if there is dysfunction at any of these levels. For example, a child with juvenile idiopathic arthritis (JIA) who has swelling and reduced range of motion (ROM) of a joint (biological level) may have difficulty running (physical level) and may not participate in gym class because of fear of injuring himself/herself (psychosocial level). Accordingly, interventions should also be aimed at each level of health: medical treatment to reduce joint inflammation, physiotherapy to improve running, and accommodations at school to allow the child to participate at his/her own pace.
In pediatric rheumatic diseases, the biological level of health is often assessed using measures of disease activity, damage, or severity; thus, these concepts also merit definition. “Disease activity” includes those aspects of a patient’s disease that are potentially reversible, and “disease damage” refers to persistent or permanent changes in anatomy, physiology, and function (and that usually develop from previously active disease or complications of therapy) . Both disease activity and damage contribute to “disease severity,” which has been defined as the total effect of the disease on the individual and often relates to prognosis . Given the multidimensionality of pediatric rheumatic diseases, a challenge exists when some components suggest that the disease is more active, while others suggest it is less active (see discussion below). Currently, pooled and appropriately weighted composite indices are often used to capture the various dimensions of disease activity (see below).
Ideally, in both clinical practice and research settings, disease activity measures should be combined with measures of physical and psychosocial functioning – and perhaps contextual factors, both personal and environmental – to compose a complete health outcome picture. This generally includes the use of PROs, which are self-reports that capture a broad array of the patient experience, including health symptoms (pain, fatigue), physical functioning, mental health, participation in activities, and self-efficacy . PROs can help to provide care that is responsive to patient needs and should also be included as important outcomes in therapeutic clinical trials . One important PRO is quality of life (QOL), which is defined by the WHO as “an individual’s perception of their position in life in the context of the culture and value systems in which they live, and in relation to their goals, expectations, standards, and concerns.” HRQL may be defined as “patient-reported quality of health,” and may refer, in some way, to how QOL is affected by health or illness .
Theory of outcome measure development
The development of health outcome measures is a systematic and an often lengthy and iterative process . Here we present a brief overview of traditional measurement theory as it applies to the development of health outcome measures in pediatric rheumatology ( Table 1 ).
| Step | Comments |
|---|---|
| Statement of purpose of measure | Should be explicitly stated a priori |
| Methods of item generation | From literature, expert opinion, patient opinion, consensus methods Item list should be comprehensive |
| Method to identify number of factors or constructs | For example, principal component and factor analysis |
| Derivation sample | Across spectrum of disease activity, academic and community clinics, ideally n = 100 cases and 100 controls |
| Clinicians involved | Clinicians deriving criteria should be different than those providing cases/controls |
| Definition of gold standard | Validated set of criteria or expert opinion |
| Technique for selection of final set of items | For example, sensitivity/specificity, receiver operating characteristic curve, regression models |
| Assessment of psychometric properties | Feasibility, face validity, content validity, reliability, construct validity, responsiveness |
| External validation of criteria set | Distinct validation sample |
Purpose of the measure
The first steps in health outcome measure development are to clearly define the construct to be measured, the population in which it is to be measured, and the purpose of the measure. Kirshner and Guyatt describe three main purposes of health status measures . “Discriminative” measures are used to identify individuals or groups with a certain attribute (e.g., criteria for inactive disease in JIA ). “Predictive” measures classify individuals into a set of predefined risk categories by a known comparison to a gold standard. This type of index is generally used as a screening or diagnostic instrument (e.g., dipstick proteinuria in lupus nephritis). Lastly, “evaluative” measures are used to assess the magnitude of longitudinal change in the health outcome of interest (e.g., manual muscle testing (MMT) in juvenile dermatomyositis (JDM) ).
Item generation and reduction
As an initial step, all possible items for inclusion into the outcome measure are generated, usually through literature review and opinions from key stakeholders, which should ideally include input from patients . Item generation is followed by item reduction, with the goal of including all the important items, but minimizing redundancy. In pediatric rheumatology, consensus methods have commonly been used for the item generation and reduction phases of measure development (see below). Other methods for item selection include principal component analysis or factor analysis. These are statistical procedures that determine the number of different aspects of the health outcome, or “factors,” being measured and then identify the items that explain the majority of the overall variance in the construct .
Consensus methods
Delphi surveys and the nominal group technique (NGT) are two common consensus procedures utilized for health outcome measure development. The Delphi technique is an iterative multistage method that allows the use of anonymous written or online responses . It involves a series of questionnaires, each of which is based on the results of the previous step, and the process stops when participants approach consensus. The definition of consensus should be stated a priori and used as criteria for termination of the process , although this is not always the case in published studies. NGT involves a highly structured face-to-face meeting to gather information from relevant stakeholders . Panelists rank, discuss, and then re-rank a series of items related to the topic. The results are analyzed for agreement in prioritization, and generally 70–80% consensus is required . These approaches have been used successfully to develop several health outcome measures in pediatric rheumatology such as the definition for improvement in JIA and disease activity core set measures for JDM and juvenile systemic lupus erythematosus (JSLE) .
Derivation study
Often there are several candidate sets of items for inclusion in the health outcome measure. In order to determine which of these sets best classifies the outcome under study, they should be applied to a large and diverse group of patients with the condition of interest . A comparator group is chosen based on the intended use of the criteria and should represent patients from whom the criteria aim to distinguish. In order to avoid circular reasoning, the group of clinicians who create the list of candidate items should be different from those who provide and classify the patients . The sample size suggested for derivation studies is approximately 100 per group .
Selection of final set of items
Statistical techniques such as comparing the sensitivity and specificity of candidate sets of items are often used to select the final set of items. For example, Brunner et al. determined the most sensitive and specific definition for flare in patients with JIA based on receiver operating characteristic curve analysis . Logistic regression may also be used to determine which variables best discriminate between more and less severely affected patients, as was done for the development of criteria for minimal disease activity (MDA) in JIA .
Assessment of measurement properties
Once a potential health outcome measure is developed, it should be piloted in a small group of users to ensure comprehensibility and relevance . The users should also evaluate the face and content validity of the tool, that is, whether the items are reasonable and cover all the important aspects of the construct to be measured. Feasibility should also be assessed, with consideration given to ease of use and to minimal burden on the patient and health care provider. Without these characteristics, the tool is unlikely to be accepted by the users or adopted into clinical care.
If a tool is found to be feasible, its measurement properties should be assessed in the target population, including an evaluation of the reliability, validity, responsiveness, and interpretability of the health outcome measure ( Table 2 ) . The results of these studies provide several key pieces of information about the measurement tool: the amount of measurement error it is associated with, whether it is measuring the construct that it is intended to measure, whether it is able to detect change in the underlying construct over time, and the significance of its scores. As a final step, health outcome measures should be applied to an independent sample to ensure external validation of these properties .
| Domain | Measurement property | Definition |
|---|---|---|
| Reliability | Internal consistency | The degree of interrelatedness among items |
| Reliability | The proportion of the total variance in measurements which is due to “true” differences among patients | |
| Measurement error | The systematic and random error of a patient’s score that is not attributed to true changes in the construct being measured | |
| Validity | Content validity | The degree to which the content of an instrument is an adequate reflection of the construct to be measured |
| Face validity | The degree to which the items look to be an adequate reflection of the construct to be measured | |
| Construct validity | The degree to which the scores of an instrument are consistent with hypotheses (e.g., relationships to scores of other instruments) based on the assumption that the instrument validly measures the construct to be measured | |
| Criterion validity | The degree to which the scores of an instrument are an adequate reflection of a “gold standard” | |
| Responsiveness | The ability of an instrument to detect change over time in the construct to be measured | |
| Interpretability | The degree to which one can assign qualitative meaning to an instrument’s quantitative scores or change in scores |
Disease activity measures for JIA ( Table 3 )
Core set criteria for JIA
The American College of Rheumatology (ACR) core outcome variables for juvenile arthritis are: (1) physician global assessment of disease activity (10-cm visual analog scale (VAS)), (2) parent/patient assessment of overall well-being (10-cm VAS), (3) functional ability (childhood health assessment questionnaire (CHAQ)), (4) number of joints with active arthritis (defined as joint effusion or limitation of motion accompanied by heat, pain, or tenderness), (5) number of joints with limited ROM, and (6) erythrocyte sedimentation rate (ESR) . Consensus methods were used to define improvement; an ACR Pedi30 response corresponds to ≥30% improvement from baseline in three of six variables, with no more than one remaining variable worsening by >30% . Similarly, the ACR Pedi 50, 70, 90, and 100 response definitions require 50%, 70%, 90%, and 100% improvement in at least three core set variables without worsening of more than one variable by >30%. Flare is defined as worsening of two variables by ≥40% without improvement in more than one variable by ≥30% . Note that achieving improvement or flare is relative to the patient’s baseline status, and thus the absolute level of disease activity may be different for each patient even though they meet the same criteria for improvement or flare (see discussion below). Additional limitations associated with the JIA core set criteria are that they are a dichotomous measure and not a continuous measure of health status, they are not straightforward to calculate, and practitioners may not collect the entire core set at each visit. As such, this measure is not typically used in routine clinical practice ; nonetheless, the ACR Pedi responses have generally been used as the primary outcome measures for therapeutic clinical trials in JIA.
| JIA | JSLE | JDM | |
|---|---|---|---|
| Relative disease activity measures | |||
| Core response variables (CRV) | ACR :
| PRINTO :
| PRINTO :
|
| Definition of improvement | ≥30% improvement from baseline in 3 of 6 CRVs, with ≤1 CRV worsening by >30% | ≥50% improvement from baseline in 2 of 5 CRVs, with ≤1 CRV worsening by >30% | PRINTO : ≥20% improvement in 3 of 6 CRVs, with ≤2 worsening by ≥ 20% (muscle strength excluded) IMACS : ≥15% improvement in muscle strength and physical function, ≥20% physician and patient global and extramuscular assessments, ≥30% improvement in muscle enzymes |
| Definition of flare | Worsening of 2 CRV by ≥40% without improvement in >1 CRV by ≥30% | ||
| Absolute disease activity measures | |||
| Global disease activity tools | JADAS :
| SLEDAI BILAG SLAM ECLAM | DAS MDAAT CAT |
| Minimal disease activity (MDA) | MDA :
| ||
| Inactive disease/remission | Wallace criteria Inactive disease:
| Inactive disease:
| PRINTO Inactive disease (on or off medication) requires ≥3/4 of:
|
a Including myalgia, arthralgia, fatigue and/or headache.
b Including statins, aspirin, angiotensin-converting enzyme inhibitors, angiotensin receptor blockers, bisophosphonates, vitamin D, omega 3 fatty acids.
Juvenile arthritis disease activity score
The juvenile arthritis disease activity score (JADAS) is a measure of absolute disease activity in JIA and consists of four components: (1) physician global assessment of disease activity, (2) parent/patient global assessment of well-being, (3) number of active joints, and (4) ESR . Unlike the ACR core set, it does not include functional status or number of joints with reduced ROM because these may be more reflective of disease damage . Depending on the number of joints assessed, the JADAS is scored on a continuous scale from 0 to 101 (71 joints), 0–57 (27 joints), or 0–40 (10 joints) and can be used to directly compare patients. Changes in JADAS score have been found to correlate with ACR Pedi response and have an excellent ability to predict flare and inactive disease . The minimally clinically important difference (MCID) in JADAS27 scores was found to be +5.5 for improvement (corresponding to ACR Pedi30 response) and −1.7 for worsening (corresponding to flare). In addition, cutoff scores of ≤2.7 and ≥6 have been proposed to distinguish low and high disease activity, respectively . Recently, the “JADAS3,” which excludes ESR and avoids the necessity of a blood sample, was found to correlate with individual measures of disease activity as well as the original JADAS .
Criteria for MDA, inactive disease, and remission in JIA
With improved success in the treatment of JIA, criteria for MDA, inactive disease, and remission have been defined. MDA is intended to represent a state between high disease activity and remission that is acceptable to the physician and patient ; however, there are no data on the prognostic significance of maintaining this state. Achieving MDA for oligoarticular disease requires a physician global assessment score of ≤2.5 cm and swollen joint count of zero, while the MDA for polyarticular disease is defined as physician global assessment score ≤3.4 cm, parent global assessment ≤2.1 cm, and swollen joint count of ≤1 .
Wallace et al. have defined three distinct health outcomes for JIA patients: inactive disease, clinical remission on medication, and clinical remission off medication. “Inactive disease” is achieved when a patient on medical therapy has: (1) no joints with active arthritis, (2) no fever, rash, serositis, splenomegaly, or generalized lymphadenopathy, (3) no active uveitis, (4) normal ESR and/or C-reactive protein (CRP), and (5) physician global assessment indicates no disease activity . If patients meet the inactive disease criteria for 6 continuous months while taking medication, they are classified as being in “clinical remission on medication,” and if they meet the criteria for 12 continuous months off medication, they are considered to be in “clinical remission off medication.” The authors established face and content validity for these criteria via consensus methods. They also showed moderate to high construct validity and responsiveness; however, reliability of the composite measures could not be estimated .
Disease activity in systemic-onset JIA
The characteristic features of fever, rash, serositis, organomegaly, and lymphadenopathy are additionally considered when evaluating disease activity of patients with systemic-onset JIA (SJIA). The core set specifies spiking fever (>38 °C during the past week) and the definition of inactive disease requires absence of SJIA symptoms as additional parameters for SJIA patients. The development of an SJIA-specific composite disease activity measure has been initiated, using patient and parent interviews , and Delphi survey of health professionals to generate items . The top items included fever, rash, increased CRP and ESR, and requirement for increasing medications. The final set of items and measurement properties of this measure are yet to be published.
Novel biomarkers for disease activity in JIA
There are emerging data on novel biomarkers for disease activity in JIA. For example, soluble ST2, the receptor for interleukin-33, is elevated in patients with SJIA and may serve as an indicator of disease activity . In addition, levels of the myeloid-related protein 8 and 14 (MRP8/14) complex (ligand for toll-like receptor 4) have been shown to predict disease flares and to correlate with disease activity in SJIA and enthesitis-related arthritis . Interestingly, high baseline MRP8/14 is also associated with increased odds for response to methotrexate in JIA (odds ratio of 16.07 for Pedi ACR50 response if MRP8/14 > 3000 ng/ml) . Elevated protein S100A12, another phagocyte-activation marker, may be another predictor for flare in JIA patients with clinically inactive disease . Certain biomarkers in the synovial fluid may be predictive of disease extension in oligoarticular JIA, including lower CD4:CD8 ratio and high level of chemokine CCL5 .
Related posts:
Genetics, genomics, and their relevance to pathology and therapy
Granulomatous inflammation: The overlap of immune deficiency and inflammation
Macrophage activation syndrome and cytokine-directed therapies
Update on research and clinical translation on specific clinical areas: From bench to bedside: How insight in immune pathogenesis can lead to precision medicine of severe juvenile idiopathic arthritis
Assessment and treatment of pain in children and adolescents
Therapeutic advancements in juvenile idiopathic arthritis
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
