Measuring outcomes

Outcomes instruments that attempt to assess function and quality of life in orthopedic patients are multiplying. Today, there are nearly 400 general orthopedic musculoskeletal outcomes instruments being used for research or clinical purposes and more than 100 measures applicable to the spine alone. Outcomes instruments can play an important role in the development of new procedures, techniques, and protocols in addition to providing some measure of quality. However, the musculoskeletal literature is filled with clinical justifications based on outcome results, such as “excellent,” “good”, or “poor” that can be at best, difficult to verify and at worst, misleading. Further, without a common language and standards, it is nearly impossible to adequately compare results against each other.

Taking into account the results of an appropriate outcomes measure is a critical step in recommending a course of treatment for musculoskeletal care. However, this can be a challenging task. In the process, one treatment protocol or intervention may be deemed better than another based on a specific desired end point (eg, range of motion), but not as good when based on another end point (eg, pain relief).

Clinician-based outcomes

One can think about outcome measures as being either clinician-based or patient-reported. Clinician-based outcomes are often physiologic and can be measured directly by the clinician. Examples include muscle strength, joint range of motion, gait abnormalities, limb length, and bony alignment. These physiologic measures, also known as “hard” or “objective” findings, are often used to infer a patient’s functional ability. In contrast, patient-reported outcomes reflect a patient’s perception of their functional ability, symptoms and quality of life. Because these are considered “soft” or “subjective,” there has been some reluctance in the past to trust these types of outcomes measures.

A commonly held belief is that clinician-based outcomes are inherently objective. After all, clinicians measure directly a patient’s motion, strength, or alignment. However, the key attribute to outcome objectivity is not dependent on who makes the assessment but rather on the reliability or reproducibility of a finding. There is substantial variability in many clinician-based outcomes. For example, interobserver agreement in determining motion of the spine or extremities is often poor. Muscle strength can be difficult to reproduce, particularly manually, but also in some cases when a dynamometer is used. Variability for simple imaging tests has also been documented. On the other hand, reproducibility of many patient-reported outcomes can be quite high. By the standard of reliability, patient-reported outcomes can be as reliable as or more reliable than clinician-based outcomes, and therefore as or more objective.

Physicians have relied on clinician-based outcomes based on the belief that a strong link exists between those outcomes and patient well-being. However, such is often not the case. For example, the severity of knee osteoarthritis is typically determined from orthogonal standing radiographs. Measurements of joint space compromise and alterations in the mechanical axis are assumed to correlate with a patient’s overall quality of life. Yet, Bruyere and colleagues showed that measurement of mean joint space width and the narrowest joint space point did not significantly correlate with pain, stiffness, or function derived from a patient-reported outcome measure, the Western Ontario and McMaster Osteoarthritis Index (WOMAC). In another example, 18 nonrheumatoid patients who underwent limited wrist fusion had poor wrist scores based on range of motion and grip strength that did not correlate highly with patient satisfaction or self-assessment of wrist performance.

Clinician-based outcomes

One can think about outcome measures as being either clinician-based or patient-reported. Clinician-based outcomes are often physiologic and can be measured directly by the clinician. Examples include muscle strength, joint range of motion, gait abnormalities, limb length, and bony alignment. These physiologic measures, also known as “hard” or “objective” findings, are often used to infer a patient’s functional ability. In contrast, patient-reported outcomes reflect a patient’s perception of their functional ability, symptoms and quality of life. Because these are considered “soft” or “subjective,” there has been some reluctance in the past to trust these types of outcomes measures.

A commonly held belief is that clinician-based outcomes are inherently objective. After all, clinicians measure directly a patient’s motion, strength, or alignment. However, the key attribute to outcome objectivity is not dependent on who makes the assessment but rather on the reliability or reproducibility of a finding. There is substantial variability in many clinician-based outcomes. For example, interobserver agreement in determining motion of the spine or extremities is often poor. Muscle strength can be difficult to reproduce, particularly manually, but also in some cases when a dynamometer is used. Variability for simple imaging tests has also been documented. On the other hand, reproducibility of many patient-reported outcomes can be quite high. By the standard of reliability, patient-reported outcomes can be as reliable as or more reliable than clinician-based outcomes, and therefore as or more objective.

Physicians have relied on clinician-based outcomes based on the belief that a strong link exists between those outcomes and patient well-being. However, such is often not the case. For example, the severity of knee osteoarthritis is typically determined from orthogonal standing radiographs. Measurements of joint space compromise and alterations in the mechanical axis are assumed to correlate with a patient’s overall quality of life. Yet, Bruyere and colleagues showed that measurement of mean joint space width and the narrowest joint space point did not significantly correlate with pain, stiffness, or function derived from a patient-reported outcome measure, the Western Ontario and McMaster Osteoarthritis Index (WOMAC). In another example, 18 nonrheumatoid patients who underwent limited wrist fusion had poor wrist scores based on range of motion and grip strength that did not correlate highly with patient satisfaction or self-assessment of wrist performance.

Patient-reported outcomes

It is increasingly recognized that traditional clinician-based outcome measures need to be complemented by measures that focus on the patient’s concerns to evaluate interventions and identify whether one treatment is better than another. Patient-reported outcomes are classified as either “general” or “disease-specific” measures of health-related quality of life. General measures are designed to be used across different diseases and across different demographic and cultural subgroups. They are usually multidimensional and are designed to give a comprehensive and general overview of health-related quality of life. The most well known general measure of health-related quality of life is the Medical Outcomes Study Short Form-36, typically known as the SF-36. General measures of health-related quality of life permit comparisons across populations with different health conditions and are more likely to detect unexpected effects of an intervention. An important limitation of these measures is that they tend to be less responsive to changes in health status and are therefore less likely to detect the effects of a specific intervention compared with disease-specific measures of health-related quality of life.

Musculoskeletal disease-specific measures of health-related quality of life, on the other hand, focus on aspects of health that are specific to an injury (eg, fracture), disease (eg, osteoarthritis), anatomic area (eg, knee), or population of interest (eg, athletes). This specificity has been shown to contribute to a more responsive measure, and is more able to detect smaller or important changes that occur over time in the particular disease studied. For example, a hip-specific instrument designed for patients with osteoarthritis should be particularly responsive to important changes in patients receiving total hip arthroplasty because it focuses only on the most relevant items. Further assuming that the instrument has clear relevance to the patient’s health problem, a disease-specific outcomes instrument will lead to greater patient acceptance, higher response rates, and improved data collection. To ensure an adequate assessment of a patient’s entire health-related quality of life, it is recommended that generic and disease-specific patient-reported outcomes measures be administered.