Unni G. Narayanan MBBS MSc FRCS(C) Departments of Surgery & Rehabilitation Sciences Institute, Division of Orthopaedic Surgery, University of Toronto, Toronto, ON, Canada Division of Orthopaedic Surgery, Department of Surgery, University of Toronto, Toronto, ON, Canada The mandate of Evidence‐Based Orthopedics is to promote evidence‐based practice and to identify areas of clinical uncertainty that would benefit from quality research. The quality of evidence is underpinned by the tenets of evidence‐based medicine (EBM), a term first introduced by Gordon Guyatt,1 based on the principles of critical appraisal developed by David Sackett, one of the pioneers of EBM.2 For each clinical question tackled in this book, authors have used the PICO framework to guide their appraisal of the evidence pertinent to that question.3 The O in PICO stands for the outcome that should be used to evaluate the effectiveness of the intervention (I) of interest relative to the comparison intervention or control (C) applied to a specific patient population or problem (P). Fundamental to the practice of EBM is how we choose to define effectiveness, or the O. Sacket himself defined EBM “as the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.”4 The pillars of EBM include (i) using the best available external clinical evidence from systematic research integrated with (ii) individual clinical expertise. Often overlooked is the third pillar of EBM, which is (iii) the “compassionate use of individual patients’ predicaments, rights, and preferences in making clinical decisions about their care.” The emphasis on patient values and preferences in evidence based decision‐making5 evolved concurrently with the evolution of the outcomes research movement which set out “to sort out what works in medicine and to learn how to make clinical decisions that reflect more truly the needs and wants of the individual patient.”6 The imperative for patient‐centered care as a critical aspect of high‐quality healthcare is enshrined in the Institute of Medicine Report in 2001.7 The outcomes we measure should be meaningful to patients and consistent with their priorities and preferences. This chapter reviews current concepts of outcome measurement, highlighting frameworks for the conceptualization of outcomes, with an emphasis on patient‐reported outcomes (PROs) and the challenges pertinent to measuring these in children. The example of a child with ambulatory cerebral palsy (CP) will be used to demonstrate some of the advances and limitations of outcome measures for pediatric orthopedic patients. Outcomes are the consequences of an intervention to treat a condition or what happens over time (natural history) if the condition is untreated.8 An effective intervention is one which alters the natural history of that condition favorably and achieves the goal(s) for which that treatment was intended. The goals of an intervention can be reactive, to address a symptom or problem (e.g. eliminate knee pain, tripping, increase walking speed) or preventative or prophylactic to prevent some future problem associated with the natural history (e.g. preserves walking, prevents osteoarthritis in adulthood). Some outcomes occur early (e.g. improved gait pattern), while others occur later (e.g. improved walking speed and endurance several months after surgery). An intervention can be associated with many outcomes, some desirable (benefits) and others undesirable (harms). An undesirable outcome can be expected (e.g. postoperative pain, muscle weakness) or unexpected – complications or adverse events (e.g. dysesthesias in the feet following nerve stretch injury), which can be transient or permanent. How does one judge the overall effectiveness of an intervention for an individual patient or a group of patients with a given condition? Pediatric orthopedics has a relatively short history of recognizing the importance of measuring outcomes. In 1991, Michael Goldberg proposed a framework of outcomes assessment in which he made the distinction between the more immediate technical outcomes of an intervention (e.g. correcting excessive femoral anteversion, normalizing the migration percentage of a displaced hip, reducing the Cobb angle in scoliosis) from functional outcomes which were the reasons for which the operation was being done (e.g. decreased tripping, pain relief, improved appearance).9 He also recommended measurement of patient satisfaction to gauge patients’ perception of the intervention’s success and cost effectiveness to measure whether the intervention was worth the costs involved. This framework served the field well with the development of generic and condition‐specific patient‐reported outcome measures (PROMs) pertinent to pediatric orthopedics. More recently, the International Classification of Functioning, Disability and Health, or ICF model, introduced by the World Health Organization (WHO),10 and its pediatric equivalent, the ICF for Children and Youth (ICF‐CY),11,12 provide a unified language to classify health and health‐related domains and a framework to measure health outcomes associated with any health condition.11 In the ICF framework, body structures refer to the anatomic parts affected by the health condition of interest (e.g. periventricular leukomalacia in CP; spastic muscle; bone deformity), and body functions refer to physiological functions of body systems (e.g. decreased range of motion, lever arm dysfunction). Body functions and structures allow functional activities or specific tasks or actions (e.g. sitting, walking, running), which facilitate participation or doing the things that one wants to do to engage in life roles (e.g. being independent, going to school, playing sports). Implicit is the assumption that participation contributes to one’s quality of life (QOL). The impact of a health condition on the body functions and structures are the biophysical impairments that might lead to activity limitations (e.g. inability to run), which might result in participation restrictions (e.g. inability to keep up with friends or play sports). The ICF framework incorporates the influence of environmental factors (e.g. home/school/community, socioeconomic status, access to health care) and personal factors (e.g. demographic characteristics, culture, lifestyle preferences, motivation, personality).13 These contextual factors can explain the gap between what one can do (capacity) and what one actually does do in daily life (performance).14 Interventions act at the level of body functions and structures. In the management of ambulatory CP, the technical objective of multilevel surgery is to address the impairments such as muscle contractures and bony deformities. Correction of these impairments (technical outcome) is assumed to lead to (functional) outcomes that patients and parents want, which is to achieve activities and participation with fewer restrictions.15 A technically successful outcome may not necessarily result in a functionally successful outcome. Also, a positive impact on activities and participation can be achieved without an intervention to correct impairments. For example, the use of powered mobility (wheelchair) might provide an alternative means of efficient locomotion, which may increase participation by allowing an individual to be independent and able to move around faster and with less effort. These gains in participation might be accompanied by a negative impact at the level of body function and structure such as decreased cardiovascular fitness or increased knee flexion contractures. Ultimately, to make meaningful judgments about effectiveness, it is necessary to base these on specific goals that are aligned with the patient’s or parents’ priorities. Patient priorities are the concerns, needs, desires, and expectations associated with living with a health condition. Patient and parent goals are the product of these priorities and informed by input from clinicians. Understanding priorities and goals is crucial for making decisions about interventions that will best address these. There may be multiple perspectives (e.g. the child’s or the parent’s) which may not be completely concordant. Figure 173.1 The Priority Framework for Outcomes Assessment. Source: Modified from Narayanan.16 Outcomes are most meaningful when they are aligned with patient priorities and goals. If our interventions are intended to address these priorities and goals, their effectiveness must be evaluated using outcome measures which specifically incorporate the goals and priorities of the patient population.16 Patient‐reported outcome measures (PROMs) are the gold standard to evaluate the effectiveness of interventions. The content of PROMs should have been derived from patients themselves, and in the case of children, parents and caregivers as well. When the perspectives of patients are not accessible because they are very young or cognitively unable, one has to rely on the report of the child’s parent(s). The views of older children can and must be taken into consideration, but their perspectives might differ from those of their parents. The level of agreement between parents and children is usually better for domains reflecting physical activity, functioning, and symptoms, but poorer for domains which reflect more social or emotional issues.17 Whenever possible, both perspectives should be considered during decision‐making, and for measuring outcomes as well. Generic PROMs evaluate the impact of physical, mental, and social function; health status; and wellbeing across different health conditions. These are useful to policymakers as they can be used to compare outcomes across different clinical conditions and interventions to understand the relative value of some types of interventions over others for purposes of healthcare utilization, planning, and resource allocation. However, generic measures are less sensitive to change than condition‐specific measures, which focus on issues directly relevant to the condition. When the primary goal of an intervention is to save or extend life, measuring mortality (survival) must be the primary outcome. However, adding years of life alone is insufficient if the life saved is not worth living. Quality of life (QOL) is defined as “individuals’ perceptions of their position in life in the context of the culture and value systems in which they live, and in relation to their goals, expectations, standards and concerns.”18 Health‐related quality of life (HRQoL) refers to the health‐related factors that contribute to the goodness and meaning of life, where health is “a state of complete physical, mental and social well‐being, and not merely the absence of disease or infirmity.”19 HRQoL is multidimensional, encompassing physical, mental, and social domains and their relation to as role attainment, daily functioning, and participation in community life.20,21 Reliability is a fundamental requirement and is the property of the measure to produce the same result when no change has occurred.22 Internal consistency refers to a special type of reliability which assesses how well items within a scale correlate with each other to measure a single construct (e.g. physical function).23 An outcome measure is valid when it measures the phenomenon it was designed to measure. An outcome measure has face validity when its items appear to be measuring what they are supposed to.23 Content validity examines the extent to which all relevant and important content or domains pertinent to the attribute of interest are adequately addressed by the items in the measure. Face and content validity represent the sensibility of the measure,24 and are established by using qualitative research with patients (and their parents) and experts who work with these patients to develop the outcome to ensure it adequately captures what’s important to patients and parents. Criterion validity is the correlation of an outcome measure with another measure that is regarded as a more accurate measure (gold standard) of the criterion. When such a gold standard measure does not exist, particularly for subjective attributes, construct validity requires a process of hypothesis testing to examine the logical relationship that should exist between a measure and characteristics of patients and patient groups.22 For instance, one can test the hypothesis that the outcome measure should generate different scores when tested on two groups, known to be different in severity of that condition (known groups or extreme groups validity). Convergent validity is a type of construct validity demonstrated when the scales of a measure correlate with the related scales of another measure, but not to unrelated scales (divergent validity). A discriminative outcome measure is sufficiently sensitive to detect small (but meaningful) differences between patients and is free from ceiling effects (i.e. many subjects rate the highest possible score on the measure because it is unable to distinguish higher functioning subjects from each other) and floor effects (i.e. less discriminative of lower functioning subjects who are rated at the lower end of the scale). Finally, an outcome instrument that is intended to measure effectiveness is an evaluative measure, which must be responsive or sensitive to change that occurs following an intervention.25,26 The presenting history of deterioration in gait in a 10‐year‐old boy with CP is consistent with the functional decline and symptoms that many ambulatory children experience in adolescence. Multilevel surgery is recommended to improve gait or preserve function and a number of outcome measures have been used to evaluate whether these interventions have been effective.
173 Outcomes in Pediatric Orthopedics
Measuring outcomes that matter in pediatric orthopedics
Clinical scenario
What are outcomes?
Frameworks of health and disease and the evaluation of outcomes
The Priority Framework for Outcomes Evaluation (Figure 173.1)
Outcome measures in pediatric orthopedics: general considerations
Generic versus condition‐specific measures
Mortality, health, and quality of life
Psychometric properties of an outcome measure (See also in Chapter 5)
Outcome measures for ambulatory cerebral palsy
Related posts:
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
