Chapter 22F Standardization of Autoantibody Testing in Systemic Rheumatic Diseases

Marvin J. Fritzler, MD, PhD , Alan Wiik, MD, PhD

INTRODUCTION

Autoantibody testing has become a powerful tool for clinicians in the management of patients with systemic rheumatic diseases, but these tests are currently not being used optimally due to lack of widely used standardization of kit manufacturing, assay performance, workplace conditions, post-marketing surveillance, and test interpretation. Regrettably, because of significant logistical challenges there have been very few prospective, unbiased, and multicenter studies. Thus, the clinical accuracy of certain laboratory diagnostic approaches is still unknown.

It is clear that standardization of kits, reagents, and protocols could markedly improve these discrepancies. New serologic technologies and assays are being developed at a very rapid pace and may eventually be accompanied by acceptance of global standards of practice. To achieve this goal, the accuracy of such testing must be demonstrated in studies involving sufficiently large patient cohorts before new assays are approved or accepted for clinical use. This chapter provides an overview of the challenges involved in and proposed solutions put forward for the standardization of autoantibody testing. It is most important to appreciate that the primary goal of standardization is to improve the quality of patient care through the correct use and interpretation of autoantibody testing.

AUTOANTIBODY ASSAY KITS AND PROCEDURES

For over half a century, the detection of human autoantibodies has become an increasingly important approach to the diagnosis and management of patients with a variety of autoimmune conditions. Although some diagnostic laboratories still use assays that are developed in-house, there is growing and widespread use of commercial diagnostic kits from a large number of manufacturers and distributors.¹ These commercial kits are cost effective, are easy to use, and for the most part have been tested and validated according to established protocols and criteria. Most kits come ready to use and provide all of the necessary reagents and clearly written protocols for performance of the assay.

Commercial autoantibody assay kits employ a variety of technological platforms that include indirect immunofluorescence (IIF), immunodiffusion (ID), immunoblotting (IB), enzyme-linked immunoassays (ELISA), LINE assays, addressable laser bead immunoassays (ALBIA), and more recently antigen arrays in a number of formats.^2,³ One of the current popular technology platforms is based on the ELISA because ELISAs are available through a global network of manufacturers and distributors, and they offer sensitivity, high throughput, relatively low cost, and modest laboratory equipment needed to perform the assay. Unfortunately, there has been little done to standardize these kits,^4,⁵ and post-marketing surveillance and quality assurance is largely left to the manufacturers.

It might be assumed that assays that have been in use for several decades (such as IIF) have achieved a high level of inter-laboratory consistency, but this is not the case. A number of studies have evaluated the performance characteristics of IIF ANA ⁶^–¹⁷ and ANCA¹⁸ kits. In addition, studies that compared ELISA kits from different manufacturers to conventional assays such as IIF and ID concluded that there was significant discordance between these conventional assays and ELISA.^7,⁹ This lack of test result correlation is compounded by variability of the performance of diagnostic kits from different manufacturers.^8,¹⁹ However, at least one study that used a cross section of serum referred to a rheumatology laboratory found moderate to good agreement between ANA-IIF and anti-DNA results with two commercial ELISA kits.¹⁰ Analysis of the design of some studies suggests that lack of agreement between ELISA and conventional assays may depend on the diagnosis and/or the selection bias of the patients under study.^6,^7,^8,¹⁰

A study of the Serology Committee of the International Union of Immunological Societies and the Arthritis Foundation focused on the ELISA kits themselves and deficiencies in intrinsic properties of the kits (sensitivities and specificities) were identified as a specific concern.¹⁹ A more recent study by the same committee focused on academic clinical laboratories and found surprising lack of consistent inter-laboratory results even when highly characterized sera were distributed and tested.¹⁶ As in other studies,⁴ it was suggested that quality control procedures for daily performance of tests in the clinical laboratory setting should be adhered to and that a minimal performance target of correlation values in ELISAs should be established.¹⁶

MANUFACTURING AND ADOPTION OF DIAGNOSTIC KITS BY THE CLINICAL LABORATORY

In many countries, the delivery of health care has been “streamlined” through consolidation of services, including those provided by diagnostic laboratories. This has often translated into higher workloads and pressure to improve the interval between when the test is requested to the time the test result is reported to the ordering physician (turnaround time). To meet these market demands, manufacturers have developed a wide variety of easy-to-use and relatively inexpensive diagnostic kits designed for high throughput and the detection of a spectrum of more autoantibodies. Unfortunately, the demand for prepackaged diagnostic kits has led to a fast-track approach in the development, validation, and approval of commercial kits. Regulatory agencies such as the Food and Drug Administration (FDA) in the USA have attempted to maintain a reasonable level of quality before allowing the release and marketing of diagnostic kits by manufacturers.

It is in the context of this activity that some problems with kit performance occur because kit validation at this level may only be discovered during post-marketing performance. Validation studies would be significantly facilitated by a centralized collection of highly characterized normal and disease cohort sera that could be used in evaluation of kit performance and establishing appropriate levels of positive and negative boundaries. Currently, validation is often performed on state-of-the-art equipment, using the freshest reagents and kits just off the assembly line. A second level of validation involves providing the kits to laboratories that are willing and able to “beta test” the kit. If the results from the external beta tests are in agreement with internal data, the kit is submitted for approval by regulatory agencies. Upon approval, marketing and manufacturing rapidly begins.

Another source of variation is that manufacturers tend to purchase kit components from a wide variety of suppliers and not all kit manufacturers use the same supplier. The decision to produce or purchase critical components such as purified antigens and secondary antibodies is based on a number of factors that include cost and performance. When a reagent satisfies these features, manufacturers tend to purchase large lots of these reagents to minimize variability between production lots. After the commercial kit has been developed, validated by the manufacturer, and marketed, it is then left to the clinical laboratories to evaluate the products available and make an “informed” decision about which kit is suited to that laboratory’s particular environment so that cost/performance issues are adequately addressed.

The continuous development and adoption of new technologies provides yet another challenge. New technologies commonly focus on achieving high diagnostic (nosographic) sensitivity but less emphasis on high diagnostic specificity. Unfortunately, in practice diagnostic specificity generally decreases when a certain assay achieves high sensitivity. Therefore, when a new test is introduced, laboratories and clinicians may overlook the loss of diagnostic specificity and focus on the sensitivity of the assay. Because new assays are frequently released before the ability of the assay to accurately predict a specific diagnosis is fully known, it is imperative that testing sera from local control patients with inflammatory rheumatic diseases be used to measure the predictive performance of a new assay. This clinical evaluation must be performed by expert laboratories in close collaboration with experienced clinicians who strive for an accurate diagnosis. Defined patient samples provided by the experienced clinicians tend to produce superior coefficients of variation and receiver operator analyses that are used to determine performance characteristics of new assays and kits.¹

AUTOANTIBODY TESTING IN THE CLINICAL LABORATORY

The clinical diagnostic laboratory is usually dependent on the integrated and optimal performance of a number of individuals that perform different tasks, ranging from the laboratory manager, to the clinical laboratory specialist (usually a clinical laboratory immunologist with a PhD or MD degree and/or other certification), to the technologist and support staff that handle the specimens and perform the assay. As noted previously, the decision to adopt a particular kit is often based on fiscal matters and is left to the discretion of the manager in consultation with the laboratory specialist.

In a modern clinical laboratory where knowledge and highly specialized technologies are rapidly expanding and specialized products are being produced, the clinical laboratory specialist is expected to be an expert. These laboratory clinicians are doing their best to maintain and advance their level of competence, while carrying out burdensome but necessary quality assurance and accreditation standards. Meanwhile, industry is advancing quickly to adapt, modify, and advance diagnostic technologies as well as offer new diagnostic kits to make the laboratory clinician’s life more manageable. It is in this dynamic that further challenges to standardization of autoantibody testing arise.

The characteristics of diagnostic laboratories that perform well are not clearly understood. Even though manufacturers provide clear directions on how to optimally use their kit and perform a certain assay, some laboratories adopt kits but use their own variation of the protocols.¹⁶ One study showed that when the laboratories are a part of or affiliated with academic institutions they might perform better, but this is not always the case.¹⁶ It is clear that when laboratories perform well, they invariably engage in a process of standardization and internal validation of new kits using clinically defined serum samples before they are adopted. As noted previously, even in ideal manufacturing circumstances (after internal evaluations and external beta testing) the capability of the assay to accurately predict a specific diagnosis is not fully known. Therefore, it is incumbent on the clinical diagnostic laboratory to evaluate each new kit with test sera from local patients with inflammatory rheumatic diseases. This process needs to be attended by close collaboration of diagnostic laboratories, experienced clinicians who strive for an accurate diagnosis and patients that willingly donate their blood for testing and research purposes.

In the very early phase of diagnostics of inflammatory rheumatic diseases, the finding of a particular autoantibody has a great impact on the setting of diagnosis and estimation of prognosis in a patient.²⁰ Therefore, serum samples taken at the very early onset of inflammatory disease symptoms should be stored together with clinical data seen at that time so that later follow-up and final diagnosis can help us focus on the most relevant tests to order in early phases of disease.²¹

There are a number of other factors that impact assay performance. The first is the equipment used to perform the test, but of pivotal importance are microscopes used for IIF and spectrophotometers and plate readers used for ELISA. These instruments vary in performance, not only with respect to intra-laboratory configurations but inter-laboratory configurations. A key element related to microscopy is the use of a transmitted UV light source rather than incident (epifluorescence) light. In addition, the quality of microscope objective lenses (numerical aperture) varies from manufacturer to manufacturer and the choice of objectives is often dictated by cost. Some laboratories use oil immersion objectives and others insist that dry objectives are adequate.

Some laboratories use their own mounting medium and cover slips even when these are supplied as components of kits. ELISA and dot blot protocols are often performed on equipment with varying capacity and photonics that bear little resemblance to the equipment used by manufacturers. In addition, equipment is becoming more sophisticated and that generally results in assays becoming more sensitive. As that occurs, the diagnostic specificity decreases and cut-off points must be adjusted. For years it was thought that the cut-off and appropriate screening dilution for ANA on HEp-2 substrates was 1/40 or 1/80. However, after a multicenter study showed that 32% of normal sera were positive at 1/40 it was recommended that a cut-off of 1/160 is more appropriate.²²

The impact of variations in equipment on assay performance is compounded when the test requires a subjective assessment by the technologist or laboratory immunologist. Unless there are stringent in-house validation procedures, this can lead to a wide range of results. A typical example of this variation is the discrepant results that may occur when a different technologist in the same lab performs and interprets the results one day and another performs and interprets the results on weekends. It might be concluded that this results in a complete breakdown of inter-laboratory and inter-test performance. Thankfully, that is not always the case because despite these variables performance characteristics might be considered remarkably high. This in large part is due to quality assurance and accreditation programs, discussed in material following.

INTERPRETATION OF THE AUTOANTIBODY RESULT

The next significant problem leading to poor laboratory standardization is the content and design of autoantibody test reports. Clinicians must be able to understand the reported results without having to pour over the report. If the report is not easily understood, it can be erroneously interpreted. In some instances, the clinician may take unnecessary time to contact the laboratory to request an interpretation or (even worse) file the report as useless information. The results must be clearly expressed and an indication of whether it represents a high, moderate, low, borderline, or negative result. In most cases, there is little clinical value in reporting numerical results such as the OD values for ELISA.

Clinicians usually want to know if the result is positive or not, if it is highly abnormal or borderline, and a general interpretation of the result. Some laboratories also provide current literature references to support the result so that clinicians can inform themselves further if they desire. When an autoantibody is found, the positive result is communicated to the clinician as a printed report or a secured digital report sent directly to the doctors. Other general information to aid in the interpretation, such as the sensitivity and specificity of a positive result, should be tabulated in a printed or an Internet-based guide or printed handbook. It is clear that a uniform internationally approved autoantibody reporting format would be desirable.

Even if the result is accurately and clearly reported, other factors contribute to how a test is interpreted. Of utmost importance, the result must be interpreted in the context of the patient’s symptoms and/or clinical findings. An autoantibody test on its own rarely establishes a diagnosis because systemic rheumatic diseases involve multiple organ systems and, particularly early in the disease course, there is rarely a pathognomic feature. Therefore, multiple criteria have been developed and must be fulfilled to confirm a particular diagnosis. Importantly, each disease is associated with different autoantibody profiles and specificities,²³^–²⁸ and there are ongoing efforts to develop improved criteria for particular groups of patients by reevaluating or adding new autoantibody markers.^29,³⁰ The goal is to increase the likelihood that a diagnosis or tentative classification of the disease is correct. The more knowledgeable the clinician is with regard to the clinical and laboratory characteristics of diseases the greater the chance a diagnosis will be correct. The use and revision of disease classification criteria is one example of progress in the area of standardization.

Although there is progress in the evaluation and updating of widely used classification criteria there is a growing need for practical evidence-based and reasoned approaches for diagnostic testing. Hence, clinical practice guidelines (CPGs) are needed for a rational, judicious, and cost-effective use of serologic testing.³¹^–³⁵ A large number of variables must be considered before CPGs are developed. First, CPGs should take into full account evidence-based research. Second, CPGs should be based on consensus of experts in the field. The conclusions from some studies are dependent on the quality of the data, and it is necessary to develop inclusion and exclusion criteria to evaluate a large body of data.¹ The value and application of conclusions based on grouped literature review in the setting of rapidly changing technology is open to debate. There seems to be limited clinical value in developing CPGs when high-performance laboratories are grouped with poor performance laboratories. Third, the application of CPGs is difficult to apply in all settings. In small laboratories and clinical service environments it is easier to achieve consensus on testing strategies than it is for large laboratories that provide service to more extensive populations.

A CPG may include a guide to clinicians suggesting which screening tests should be used for a particular patient with a certain tentative diagnosis and those tests that are useful to monitor disease activity or progression (Table 22F.1). This guide could also provide a flow diagram of tests necessary in support of a diagnosis and estimate prognosis. A positive screening test may lead to the referral of a patient to a specialist, who will then initiate further serological testing. It is important to realize that the pre-test probability of detecting a useful diagnostic laboratory result increases with each clinical feature that has been incorporated into the tentative diagnosis.^36,³⁷ These issues emphasize the importance of developing CPGs that are current and in tune with conventional autoantibody testing.

TABLE 22F.1 AN APPROACH TO AUTOANTIBODY TESTING