Outcome Measures in Upper Limb Prosthetics

Laura A. Miller PhD, CP

Linda Resnik PT, PhD, FAPTA

Dr. Miller or an immediate family member serves as a board member, owner, officer, or committee member of US Chapter of the International Society of Prosthetics and Orthotics. Neither Dr. Resnik nor any immediate family member has received anything of value from or has stock or stock options held in a commercial company or institution related directly or indirectly to the subject of this chapter.

ABSTRACT

Upper limb prosthetic outcome measures are needed for clinical care and research in upper limb amputation. However, the field of amputation rehabilitation has yet to reach consensus on a recommended suite, or toolbox, of measures for routine use. It is important to provide a brief overview of measurement properties to consider when selecting an outcome measure and to highlight outcome measures that have been recommended for use in upper limb amputation rehabilitation, along with emerging measures and important constructs that may be relevant to upper limb amputation rehabilitation. The clinician should be aware of barriers to implementing outcome measures and select strategies to improve adoption.

Introduction

An outcome measure is a means of systematically collecting data through a testing procedure that has emerged through a formal development process, which includes an evaluation of psychometric (ie, measurement) properties.¹^,² Outcome measures are essential clinical tools that can be used to evaluate patient status and assess change over time, provide data to identify patient needs, and help establish treatment priorities.³ Outcome measures can also be used to gauge effectiveness of a treatment during usual care delivery and are often used in research to describe populations and to assess effectiveness or efficacy. Collections of standardized outcome measures across practices or systems of care can be used to evaluate and improve quality.⁴

Outcome measures can be generic, population specific, or patient specific. Generic measures are designed for use in a broad variety of patient populations. However, condition-specific measures are designed for use in specific patient populations (eg, conditions, diseases), such as patients who are prosthesis users, or persons with upper limb conditions. As such, condition-specific measures are designed to target content areas that are most relevant to the disease or condition. Because of this they may be more responsive to change compared with generic instruments. Finally, patientspecific measures are used to assess activities and participation and goals that are identified and valued by the individual.

Some outcome measures assess performance (performance-based measures) and are administered and scored by a tester. Other outcome measures, particularly those that assess aspects of the client/patient experience, are self-reported and can be administered by an interviewer or self-administered. All outcome measures have formal scoring rules, some measures are scored with a single summary score, and other measures are scored via multiple subscales and/or component scores. Developing, refining, and testing a psychometrically strong outcome measure is a rigorous process that can take 10 to 15 years.

Researchers and clinicians have many choices for outcome measures. The best outcome measures must demonstrate reliability, validity, and responsiveness to change. Reliability means that outcome results are consistent and relatively free from errors. Validity means that a measure is gauging the concepts that it intends to measure. Responsiveness is the ability of a measure to detect meaningful change over time. More details about these key psychometric attributes of measures are provided later and in Table 1. Standards for evaluating the sufficiency of these psychometric properties have been described in detail in the scientific literature, and will not be summarized in this chapter. Because the psychometric properties of outcome measures are not fixed, measures must be studied in their intended population because measures may be reliable or valid in one patient population, but not in another.

Historically, it has been common for those working in upper limb amputation research and clinical care to create their own outcome measures for patient evaluation, either by adapting an existing measure and/or using
selected portions of a measure in an attempt to tailor it to a specific question or clientele. However, this practice may threaten a measure’s reliability and validity and is generally frowned upon. Although there may be a need for new, revised, and innovative measures in the field, their use should be supported by psychometric evaluation, which is generally outside the scope of clinical care and most research projects. There is also widespread recognition of the value of using standardized outcome measures with strong psychometric properties within the patient/target population whenever possible. Use of such standardized measures makes tracking progress or comparing outcomes across patients and groups possible. Collection of standardized data can enable pooling of outcomes across research studies in future systematic reviews and maximize usefulness of research findings in development of clinical practice guidelines.

TABLE 1 Key Psychometric Attributes of Measures

Psychometric Attribute	Methodologic Requirements
Reliability
Internal consistency	Internal consistency is a measure based on the correlation between items of a measure. It measures whether separate items are similar enough that they are capturing the same general construct.
Test-retest reliability	Test-retest reliability (or repeatability) is a measure of stability of a test over time, under the same conditions.
Interrater reliability	Interrater reliability is the repeatability of a test when administered by more than one rater.
Intrarater reliability	Intrarater reliability is the degree of agreement among repeated measurements by a single rater.
Validity
Face validity	Face validity is a subjective determination of how well the content of the measures covers the target construct and whether the test is intuitively meaningful to the tester and patient.
Content validity	Content validity is similar to face validity and is typically established by expert evaluation.
Criterion validity	Criterion validity is a measure of good agreement between test scores and scores of a current gold standard. Gold standards are not available for most rehabilitation measures.
Predictive validity	Predictive validity is a measure’s ability to predict outcomes or scores of another measure or significant event at a future point in time.
Construct validity	Construct validity is the degree to which a measure assesses what it intends to be measuring. There are several ways to demonstrate construct validity including the known-groups method, hypotheses testing, factor analysis, and contemporary measurement methods such as Rasch analysis.
Concurrent/discriminant validity	Concurrent and discriminant validity assess how much a measure correlates with other validated measures of similar or different constructs.
Minimal detectable change (MDC)	The MDC is the minimal amount of change in a measure’s score that exceeds measurement error. The MDC can be used to help interpret change scores in repeated measures.
Responsiveness evaluation	Responsiveness is the ability of a measure to detect change that is meaningful to the patient over time.
Examination of scale difficulty	A good measure should be well targeted to its intended population. If it is too difficult or easy, then it will be difficult to determine if decline or improvement has occurred. Scale difficulty can be examined, in part by evaluating floor and ceiling effects. These refer to the lower and upper bounds of a measurement past which the measure cannot be considered accurate or reliable.

There are many standardized measures available for use in upper limb amputation rehabilitation. Although strong evidence of reliability, validity, and responsiveness is key, it is also critical that users of outcome measures select those that address the constructs (eg, activity performance, prosthesis satisfaction) and questions that are most important to patients, clinicians, and payers. The choice of measures is also contingent on other factors related to the measure’s utility, for example, the time it takes to administer the measure (administrative burden), the need for and availability of specialized testing equipment, the need for special training for the test administrator, as well as the ease of scoring and interpretation of the scores.

Using Outcomes in Prosthetics

As mentioned previously, there are multiple reasons that a researcher or clinician might want to use outcome measures. The goal of using outcome measures may be to evaluate and improve quality of care, to predict or detect change over time within patients or between groups, or to distinguish between types of devices. Measures may be used to understand the breadth of function, application to activities, use in daily life, and quality of life (QOL). To measure multiple domains, a suite or toolkit of measures will be required.

When choosing outcome measures, it is important to carefully consider whether the measure provides the most useful information to address specific questions of interest or targets of treatment. For example, data from a measure that focuses on general QOL may not be able to detect the differences between the function of two different prosthetic hands.

Various groups of researchers and clinicians continue to work to evaluate the psychometric properties of measures and categorize their content. Although there is no current consensus, a variety of efforts have been made to help guide the selection measures that should be used in routine clinical care and/or research. Given the small population of users of upper limb prosthetic devices, the use of a core set of measures could also provide a larger, uniform pool of data for the profession to help advance the field of research in upper limb prosthetics.

Efforts to Evaluate and Categorize Upper Limb Outcomes Measures

Over nearly 20 years, a variety of groups have worked to categorize and evaluate outcome measures for upper limb amputation rehabilitation. The framework of the World Health Organization
International Classification of Functioning, Disability and Health (ICF) model has been used to describe the content of outcome measures used in upper limb prosthetics. The ICF framework describes the relationships between a health condition and the associated effects on the components of Function, Activity, and Participation, as well as Environmental and Personal Factors, and presents a taxonomy for defining each of these components of functioning and health.⁵^,⁶ Table 2 shows how the broad taxonomy of the ICF may be useful in selecting outcome measures to answer specific questions.

The Upper Limb Prosthetic Outcome Measures group formed at the Myoelectric Controls Symposium⁶ endeavored to critically evaluate outcome measures routinely used by clinicians and researchers. The purpose, clinical utility, and psychometric properties of each measure were documented.⁵^,⁶ A State of the Science Conference (SSC), sponsored by the American Academy of Orthotists and Prosthetists, combined work of the Upper Limb Prosthetic Outcome Measures group⁶ with an evidence-based review of the literature on outcome measures⁷ and proposed an early toolbox of recommended and to-be-considered outcome measures.⁸ These outcome measures were categorized by stakeholder questions, the ICF domain they addressed, and the field of application (development, clinical research, or patient care).

Another review of upper limb outcome measures focusing only on the domain of physical function was published by the US Department of Veterans Affairs as part of a clinical practice guideline for the management of upper extremity amputation rehabilitation.⁹ The measures in this review were specific to adult users of upper limb prostheses. Tables in the Guidelines summarized the ease of use and content of the physical function measures and rated the strength of evidence supporting the psychometric properties of the measures in persons with upper limb amputation. Additionally, the tables summarized the minimal detectable change for those measures in which it had been reported.

Two additional systematic reviews of measures for persons with upper limb trauma and amputation have been completed.¹⁰^,¹¹ One review addressed measures of impairment and activity limitation and the other addressed community integration/participation in life roles. These two reviews identified measures with the strongest psychometric properties and classified the content of each measure using ICF categories of body function, activity, and participation.

Summary of Highlighted Outcome Measures

Outcome measures that were recommended by the Academy’s SSC (indicated as SSC),⁸ were considered strong measures in the 2014 Veterans Administration/Department of Defense clinical practice guidelines (indicated as VA),⁹ or were used in upper limb amputation population and rated highly in either of the two systematic reviews described previously (indicated as SystRev1¹¹ or SystRev2¹⁰) are described. Table 3 provides a synopsis of the content of the subset of measures that have application to adults. Table 4 provides a list of known minimal detectable change values for the outcome measures shown in Table 3.

Performance Measures

Activities Measure for Upper Limb Amputees: VA and SystRev1

The Activities Measure for Upper Limb Amputees (AM-ULA) is a performance-based measure of activity.¹² It includes 18 self-care and activity of daily living tasks, such as pouring from a can, tying a shoe, zipping a jacket, and buttoning a shirt. Every item is scored between 0 and 4, based on a scoring rubric that takes into account task completion, speed, movement quality, and skillfulness of prosthesis use.

Assessment of Capacity for Myoelectric Control: SSC and VA

The Assessment of Capacity for Myoelectric Control (ACMC and ACMC 2.0) is a performance-based measure that uses observational analysis, typically from video, of an adult or pediatric patient with an upper limb prosthesis during functional bimanual activities.¹³^,¹⁴^,¹⁵ A 4-point scale is used to evaluate 30 aspects of myoelectric prosthesis use (eg, the ability to hold an object over the course of a task, with or without support; coordination with both hands; and the ability to adjust grip force, with or without visual feedback). Any bimanual task can be evaluated provided it allows all aspects of use to be observed. ACMC 2.0 reduced the aspects of control scored from 30 to 22 and clarified item definition. Final scores are calculated via the ACMC website, which uses a Rasch analysis. Originally, raw scores were converted into a logit scale. The ACMC version 3.0 manual updated the scoring to transform the logit scale to a range of 0 to 100, with higher scores indicating improved capacity.¹⁶ Raters are required to complete a training course to obtain certification.

Box and Blocks: SSC, VA, and SystRev1

The Box and Blocks (BB) is a performance-based measure that evaluates gross manual dexterity.¹⁷^,¹⁸ Individuals are asked to transfer as many 1-inch wooden blocks as possible over a divider from one side of a box to another, within 1 minute (Figure 1). The number of blocks represents the score. No specialized training is required to administer the test, and the necessary equipment can be purchased or constructed.

Jebsen-Taylor Hand Function Test: SSC and SystRev1

The Jebsen-Taylor Hand Function Test (JTHF) is a seven-part timed dexterity test.¹⁹ The tasks include writing, flipping index cards, picking up small objects, spooning beans into a jar, stacking checkers, and moving light and heavy cans (Figure 2). The time to complete each task is recorded, and each of the activities is scored separately. Recent work on the Jebsen-Taylor Hand Function Test (specific to prosthetics) resulted in modification to the scoring method of the original measure.¹⁷ The prosthesis-specific scoring involves counting the number of items completed (eg, checkers stacked,

cards flipped) as well as the time it takes to complete the test. The score is calculated by dividing the number of items completed by the number of seconds required to complete. Because of the difficulty of some tasks for persons using prostheses, the time allowed for each task is capped at 2 minutes. No specialized training is required to administer the outcome measure, and the necessary equipment can be purchased or made.

TABLE 2 Questions of Interest in Upper Limb Outcomes^a

Question

ICF Category

B. Body Functions

D. Activities and Participation

Other

Sensation of Pain (B280)

b7. Neuromuscular and Movement-Related Functions

d4. Mobility^b

Self-care (d5)

Domestic Life (d6)

Interpersonal Interactions and Relationships (d7)

Major Life Areas (d8)

Community, Social, and Civic (d9)

Unspecified Activities (Patient Selected)

Quality of Life

Goals

Satisfaction

Prosthesis Use Measures

What is the effectiveness of prosthetic or occupational treatment for improving…

How does the choice of prosthetic componentry affect…

How do my patients or my patient care compare to benchmarks for…

Movement control?

—

Activity performance?

—

Hours of use?

—

Prosthesis adoption?

—

Satisfaction with device?

—

Quality of life?

—

Pain?

—

How well does an individual control their prosthesis?

—

Are there areas of control/function that could be improved?

—

Is one type of prosthesis control better than another type of control?

—

What level of control/function should a prosthetic user expect?

—

Are users satisfied with the device?

—

How well does amputation rehabilitation help clients achieve their goals?

—

What is the quality of life for an individual with upper limb loss/deficiency?

—

With how much difficulty and how often does the individual use their upper limb prosthesis in everyday activities?

—

Is the upper limb prosthesis useful to the individual for specific tasks?

—

What is the physical function for an individual with upper limb loss/deficiency not using a prosthesis?

—

^aTable shows what ICF categories are included in many of the clinical and research questions of interest when evaluating upper limb prosthetic devices. (Only B and D ICF categories listed here.) An X indicates that the question addresses the ICF category or ‘other’ category. A dash indicates that the question does not directly address the category.

^bd4 Mobility examples: fine motor tasks (d440), hand and arm use gross motor (d448), carrying and handling objects (d430-d439), using transportation (d470).

Patient-Reported Measures

Disabilities of the Arm, Shoulder and Hand: SystRev1

The Disabilities of the Arm, Shoulder and Hand (DASH) is a self-report measure that assesses symptoms and disability in individuals with upper limb musculoskeletal disorders (disease or injury).²⁰ It consists of 30 questions covering function, symptoms, and social/role functioning. Total scores range from 0 to 100, with higher scores indicating greater disability. There are two optional modules for sports/performing arts and work.

QuickDASH: SystRev1

The QuickDASH is a shorter version of the DASH questionnaire. It is a self-report survey designed for persons with upper limb impairments.²¹ The QuickDASH, which has been validated in persons with upper limb amputation, contains 11 items assessing functioning and systems in musculoskeletal disorders of the upper limb.²² The QuickDASH also includes two optional scales for work activities and sports or playing an instrument.

Trinity Amputation and Prosthesis Experience Scales: SSC and SystRev2

Trinity Amputation and Prosthesis Experience Scales (TAPES) is a self-report measure originally designed to assess a user’s adaptation to lower limb amputation and prosthesis use.²³ Desmond and MacLachlan used items from the TAPES in a study of persons with upper limb amputation and proposed a different scoring approach.²⁴ The result, the TAPES-ULA, has four psychosocial adjustment scales: General Adjustment, Social Adjustment, Adjustment to Limitation, and the Optimal Adjustment scale reflecting the development of an optimistic outlook and the positive appraisal of life despite the trauma associated with amputation and the use of an artificial limb. There are four activity restriction scales: athletic activity restriction reflecting the limitation of activities that involve more dynamic physical effort, for instance, sport and recreation and running for a bus; social restriction that addresses limitation of social activities such as visiting friends and working on hobbies; mobility restriction that address physical function and mobility; and a new occupational restriction scale relating to restrictions in occupational performance.

Only gold members can continue reading. Log In or Register to continue