Clinical outcome assessment tools

The need to measure the outcome of treatment has long been recognized.⁴ Relying on a patient’s comments to the treating clinician is an unreliable way of gaining an objective insight into the results of treatment. Purpose-developed assessment tools have been designed to give a better picture of a patient’s status. These tools may be generic, or specific to a particular anatomical region or disease process.^5–12 The tools will generally consider a number of domains (subsections) in order to obtain information about different issues that might be relevant to the outcome or a patient’s status. There is a balance to be struck between a tool that covers many domains, and so gives a comprehensive assessment of a patient’s status, and one that is sufficiently concise to be of practical use (i.e. can be completed accurately in a reasonable time).

Measures may consider the response of patients to questions posed to them (patient-reported outcome measures – PROMs) or obtain information by an assessor evaluating aspects of the patient’s condition (surgeon-reported outcome measures – SROMs). While some clinicians feel that the information from a SROM should be more objective, a well-designed and validated PROM will provide a measure of a subjective construct (the patient’s impression of their status) and has the advantage of being completed free from the potential confounding of the personal relationship between the clinician and their patient (whether or not the patient responds in a manner to ‘please’ their clinician). As there is no examination-dependent domain, the PROM has the added advantage of it being possible to complete long-range, making data capture possible from patients who cannot travel for assessment.

In assessing a patient’s outcome after treatment, there may be many factors (or domains) that need to be considered. For example, the perceived outcome of surgical treatment of a painful, unstable arthritic elbow joint will depend on the degree of pain control achieved, the range of motion possible in the treated joint, the stability of the treated joint and the function of the hand on the side treated. These variables are likely to require different questions and assessments to determine their status (a question about pain from an elbow may not address whether or not an elbow is stable). Equally, questions may be dependent on one of several domains to determine the response given (the response to a question about the ability to lift objects will be influenced by the level of pain, the range of motion in and the stability of the elbow).

Some of these tools are global in nature, giving a picture of a person’s overall well-being, with domains to address particular features, such as mental state, physical well-being, pain, etc. The responses obtained by measures like this will therefore consider many variables, and so the impact of treatment of one diseased joint may well produce only limited changes in the score returned. To better determine the effect of an intervention on a particular joint, use of an outcome measure specific to that joint or region (and so more likely to be sensitive to changes in the function of the treated joint) or a tool specific to the disease process (and so more likely to be sensitive to changes in the severity of a disease) is more appropriate. In practice, there is again a balance to be struck between using a measure that is most likely to detect change following a given intervention, and using a tool that is applicable to several conditions and so of more practical use.

Survival analysis

Another way of assessing the outcome of a treatment is to undertake survival analysis. Commonly applied to studies considering the long-term survival following treatment for malignant diseases, this mechanism of standardizing the ‘start date’ for follow-up after treatment allows information regarding the long-term success (proportion of patients alive, in oncology studies) of treatments allows the relative effect of different treatments to be compared. This technique has been applied to the survival of joint replacement implants, but could equally well be applied to the long-term success of treatments designed to control symptoms (although in practice this is rarely, if ever, done due to difficulties determining an unambiguous definition of failure – see below).

Broadly speaking, two different methods of undertaking survival analysis are in common clinical use. One, the Kaplan–Meier technique, recalculates the proportion surviving (and the confidence intervals) each time an event occurs; this may be revision of an implant, and so failure, or withdrawal of a patient through unrelated death or loss to follow-up, either of which reduces the pool of patients under consideration, and so the denominator for calculations. (Loss to follow-up is a difficult area, considered in more detail below.)

A second technique involves determining a life table, considering all events that occur in a given year of follow-up together, and so recalculating the proportion surviving only once per given year of follow-up. This gives a simpler to follow, although annually averaged, picture of survival. An example is given in Figure 46.1.

Figure 46.1 Survivorship for the Souter–Strathclyde implant. 95% confidence intervals are only shown for the curve with revision as the sole endpoint.

(Redrawn with permission from Little CP, Graham AJ, Woods DA. Outcomes of Total Elbow Arthroplasty for Rheumatoid Arthritis: Comparative Study of Three Implants. The Journal of Bone and Joint Surgery (American). 2005;87:2439–2448.)

Whichever method is used, simply presenting the surviving proportion is misleading without giving the confidence intervals of the calculations. The smaller the pool of patients being considered, the greater the potential influence of a further event on the survival proportions, and so the wider the confidence interval. This concept of the ‘number at risk’ is clearly defined in a life table. The confidence intervals can give a clear impression of the relevance (if any) of apparent survival benefits to be gained through the use of a particular implant or technique over the alternatives.

A clear definition of what defines ‘failure’ is important in survival analysis. This sounds self-evident, and in oncology studies, the hard outcome measure of patient death is used. For survival analysis of arthroplasties, however, the apparently hard measure of implant revision is not, in fact, as good a measure as it might sound. A clinically failed or failing implant would be defined as a success until the revision surgery had been undertaken; therefore included with the ‘successes’ would be loose, asymptomatic implants (perhaps appropriately) and loose, symptomatic implants in whom the patient declined surgery, was not offered surgery for clinician-based reasons or was not deemed fit for surgery (all less appropriately). These factors confound the analysis.

Another area of difficulty, as with all follow-up studies, is knowing how to deal with loss to follow-up. A best-case scenario (that all patients lost are considered to be doing well, and so not responding to follow-up requests) may be taken; an equally reasonable worst-case scenario (that all patients not attending for follow-up are in fact doing badly, and chose not to attend or have sought treatment elsewhere) could also be undertaken, and the results of all three analyses considered.

While survival analysis is an important tool for considering the results of joint replacement surgery, its application to other areas of treatment is rare, and so it will not be addressed further in this chapter.

Joint registries

There is an increasing interest in gathering data about joint replacement function and survival on a national level, with national joint registries existing for lower limb implants in Scandinavia, the UK, Australia and New Zealand. These log patients receiving implants and then monitor implant performance, by registering when a component requires revision (and the indication for the revision) and by administering PROMs to determine patients’ function. The Oxford PROMs are used as the outcome assessment tool for many of these registries, being easily administered without the need for patient assessment.