Neuropsychological Testing in Concussion

Andrea Pana

INTRODUCTION

It is estimated that 1.6-3.8 million sports concussions occur annually in the United States (1,9). This number may underestimate the total number of sports concussions because the athlete, coach, or parent may not report or may not recognize the symptoms of concussion.
Animal studies have demonstrated a cascade of physiologic events that adversely affect cerebral functioning for a period of days to weeks after concussion (22).
Diagnosis and management of concussion involve a careful assessment of the history of the injury and subsequent symptoms, a thorough clinical examination, as well as the consideration of advanced diagnostic testing. Neuropsychological (NP) testing is one of several tools currently used in the management of concussion.
The Zurich Consensus statement on concussion states: “The application of neuropsychological (NP) testing in concussion has been shown to be of clinical value and continues to contribute significant information in concussion evaluation. It must be emphasized, however, that NP assessment should not be the sole basis of management decisions; rather, it should be seen as an aid to the clinical decision-making process in conjunction with a range of clinical domains and investigational results” (24).
The Team Physician Consensus Statement on Concussion states that NP testing “is recommended as an aid to clinical decision-making but not a requirement for concussion management.” The document also states that “the value of NP testing is enhanced when used as part of a multifaceted assessment and treatment program” and that it is “one component of the evaluation process and should not be used as a stand-alone tool to diagnose, manage or make RTP decisions in concussion” (12).

WHY IS NEUROPSYCHOLOGICAL TESTING USEFUL IN THE EVALUATION OF CONCUSSION?

Returning an athlete to participation before complete recovery may increase the risk of a second concussive injury, a catastrophic injury, or chronic, long-term cognitive impairment (18). Postconcussive symptoms and clinical recovery patterns vary in individual athletes, and not all have a typical course. Up to 10%-20% of concussed athletes may have a postconcussion syndrome in which symptoms of concussion such as headache, fatigue, and memory impairment persist for a month or more after the injury (18).
Self-reported symptoms are not reliable as a sole indicator of resolution of concussion. Several studies have shown that NP testing has a greater sensitivity than self-reported concussion symptoms in detecting resolution of concussion. Van Kampen et al. (33) found that 64% of their concussed athletes reported symptoms at 2 days, whereas 83% had deficits on NP testing compared to their baseline. Broglio et al. (2) showed that the sensitivity of symptoms in detecting concussion at 24 hours post injury was 68% compared to 78.6% and 79.2% for HeadMinder test and Immediate Post-Concussion Assessment and Cognitive Test, respectively.
There are many reasons why athletes may underreport symptoms. Athletes may blatantly deny symptoms due to internal or external pressures to compete (fear of losing position, losing respect, seeming weak, letting team down). They may mislabel or fail to identify symptoms when they occur; attributing them to stress, dehydration, and tight-fitting helmet. Mild postconcussive symptoms may be viewed as their baseline level of functioning. Athletes may not be aware that symptoms of fatigue or sleep deprivation are postconcussive. Lastly, an athlete may truly be asymptomatic but still have neurocognitive deficits associated with concussion (11).
Absence or resolution of symptoms is not indicative of complete recovery. Significant cognitive deficits remained in approximately 35% of concussive injuries when players were tested on computerized test battery after symptoms had resolved (20).
NP testing is an objective measure of cognition that is more sensitive than sideline cognitive tests in identifying cognitive deficits and tracking recovery in concussed athletes.

TYPES OF NEUROPSYCHOLOGICAL TESTING

Sideline assessment tools: These are basic NP tests designed for sideline assessment of players with concussion. Specifically, they involve tests of orientation, memory, concentration, and delayed recall.
Clinician-administered NP tests: Consist of paper-and-pencil NP tests to assess cognitive function. Concussion assessment batteries typically focus on cognitive function, but additional tests assessing related psychological dysfunction can be added in cases where more extensive NP testing may be warranted in a given patient.
Computerized NP tests: These are computer-based NP tests that measure various aspects of memory (new learning), cognitive processing speed, working memory, or executive functions. (The rationale for choosing tests from these domains is that these are functions typically affected by traumatic brain injury, as opposed to language or visuospatial skills, which are more resistant to the effects of brain injury) (9,28).

STATISTICAL FACTORS NECESSARY IN CHOOSING A NEUROPSYCHOLOGICAL TEST

For NP testing to be valuable in the evaluation and treatment of concussion, certain statistical parameters should be met to demonstrate that as a clinical tool it has added value to the clinical armamentarium. Ultimately, one would like to be able to say that using this tool changes outcomes in the individuals one is treating with concussion.
In the literature, statistical parameters applied in assessing NP test batteries include: reliability, validity, sensitivity, reliable change, and clinical utility.

Reliability

This is a measure of the stability of a score or test over time.
Test-retest reliability measures should ideally reflect clinically relevant intervals. For concussed athletes, that means scores need to be stable for months to more than a year in time.
Test-retest reliability measures can be affected by practice effects. Some of these effects can be accounted for in the statistical measure used.
Pearson coefficient and intraclass correlation coefficient (ICC) are two measures used in reliability. A minimally acceptable score for reliability is 0.60, with 0.90 being the ideal score to achieve (1,29).
Pearson coefficient is a statistic that is a bivariate measurement of the relationship between two independent variables. It is limited by insensitivity to systematic changes in the score means due to learning or practice effects. It is known to overestimate the correlation when sample sizes are limited (1).
The ICC is a univariate measure estimate of the agreement between two scores on the same test at two points in time. It accounts for some of the practice effects and is more commonly used in current studies as a measure of test-retest reliability.

Validity

The basic concept behind validity is establishing that a test measures what it is supposed to measure.
Concurrent validity: the degree to which a test under development correlates with established measures of the ability in question.
Validity may also be established by demonstrating that a test is sensitive to the impairment in clinical populations with known defects. (For example, is a test of memory decreased in patients with known memory deficits?)

Sensitivity and Specificity

One must establish that the test can differentiate between clinical patients (athletes with concussion) and controls (normal athletes). Is the test “positive in disease” (sensitivity) and “negative in health” (specificity)?
One can measure sensitivity of a test with analysis of a group (comparing group means in those with and without concussion) or in analysis of individuals (requires a prospective age-matched study of controls to athletes with concussion).
Specificity goes hand in hand with discussing sensitivity. In specificity, one looks at whether a test is negative in individuals without the disease in question. Is the test negative in an athlete who does not have a concussion? If not, then one has a false positive.

Change Scores

This is a measure of the variability of a test over time.
Variability is made up of a “real or normal fluctuation” of a test from one session to the next and an “error variance” or change in the test that is due to flaws in the measurement technique.
The degree of change can be measured with the following measures: simple change scores, true change scores, standard deviations, standard error of measurement index, simple regression, multiple regression, reliable change index (RCI), and modified RCI. These techniques provide a degree of change measurement that accounts for one or more of the following: measurement error, regression to the mean, practice effects, and variables such as age, education, socioeconomic status, and history of concussion (only multiple regression).
In the sports medicine/neuropsychology literature, the two most commonly applied techniques are RCIs and regression analyses (6).
RCIs provide a value above which an observed change can be said to be meaningful. A standard RCI does not correct for the effects of measurement error caused by practice effects or other confounding variables. A modified RCI controls for measurement errors and practice effects.
Simple and multiple regression techniques can be used to calculate or predict a subject’s score after concussion. Multiple regression equations may include estimates of effects of variables such as age, education, socioeconomic status, and prior concussion history. A significant change is said to occur when the difference between the observed and predicted score is greater than a certain criterion (6).

Clinical Utility

Is the test clinically useful? A test can have high reliability and high sensitivity but not necessarily be useful as a clinical decision-making tool.
For concussion, an NP test would be clinically useful if it is sensitive in detecting neurocognitive impairments once concussion symptoms have resolved.
It could also be added that a test would be clinically useful if the decision made from using it changes clinical outcomes. Does holding the athlete beyond the time during which they are asymptomatic until the time in which their NP tests return to baseline change their clinical outcome?

SIDELINE ASSESSMENT: BRIEF NEUROPSYCHOLOGICAL TESTS

How Are Neuropsychological Tests Used in Sideline Assessment of Concussion?

Sideline NP assessment consists of brief tests of orientation, memory, concentration, and delayed recall used to assess an athlete’s cognitive functioning to determine if the athlete sustained a concussion as well as to follow the concussion while on the field. It is used in conjunction with history, symptom assessment, and physical exam in the initial diagnosis and assessment of a concussed athlete.
Serial cognitive sideline assessment of concussion was also used along with signs and symptoms to determine whether an athlete could return to the competition that day. However, recent changes in legislation (National Football League [NFL], National Collegiate Athletic Association [NCAA], state athletic associations) and medical practice have by in large restricted the return of an athlete into the competition in which he or she sustained a concussion.
Serial NP assessment, along with serial symptom and exam evaluation, is used on the sideline to assess for worsening of cognitive status and necessity for further emergent evaluation such as imaging for a bleed.

Common Sideline Assessment Tools Used in the Evaluation of Concussion

Maddocks questions, which are composed of questions of orientation and recent memory, were validated in a 1995 study. It was in this study that it was concluded that questions relating to orientation (person, date of birth, age, and month) were not sensitive in discriminating between a concussed and nonconcussed athlete. Questions relating to recall of recently acquired events (ground, quarter, how far into quarter, last team to score, team played last, who won last) were sensitive in detecting concussion (19).
Standardized Assessment of Concussion (SAC) was developed in 1996 in response to the need for a standardized concussion assessment tool. It is a validated tool assessing orientation, immediate and delayed memory, and concentration. It is a scored tool that is optimally used when compared to a preseason baseline (26).
Sport Concussion Assessment Tool (SCAT) is a validated tool developed from the Second International Consensus Conference on Concussion in Sport (Prague). It was created by combining several existing assessment tools used by medical and sports organizations into a new standardized tool. SCAT consists of evaluation of signs, memory assessment, symptoms, cognitive assessment, and neurologic screening (26).
SCAT2 was a product of the Third International Consensus Conference on Concussion in Sport (Zurich). This tool consists of eight sections: symptoms, physical signs, Glasgow Coma Scale, sideline assessment-Maddocks score, cognitive assessment-SAC, balance examination, coordination examination, and cognitive assessment-delayed recall. Each section is scored with an overall total possible score of 100. This tool is not yet validated (24).

COMPREHENSIVE (CLINICAL) NEUROPSYCHOLOGICAL TESTING

Paper-and-Pencil (Clinician-Administered) Neuropsychological Testing

A variety of paper-and-pencil test batteries have been used for neurocognitive assessment. In patients with concussion, tests that measure various aspects of memory (new learning), cognitive processing speed, working memory, attention, or executive functions have been most commonly used. The rationale for choosing tests from these domains is that these are functions typically affected by traumatic brain injury, as opposed to language or visuospatial skills, which are more resistant to the effects of brain injury (28).
Examples of traditional paper-and-pencil neurocognitive tests (28):
- Hopkin’s Verbal Learning (memory/verbal learning)
- Brief Visuospatial Memory Test (memory)
- Wechsler Adult Intelligence Scale, Third Edition (WAIS-III) Digit Symbol subtest (processing speed)
- Symbol Digit Modalities Test (SDMT) (processing speed)
- Trail Making Test (processing speed, executive)
- Controlled Oral Word Association (processing speed, executive)
- Stroop Color Word Test (executive)
- WAIS-III Digit Span Test (working memory)
- WAIS-III Letter-Number Sequencing Test (working memory)
- Paced Auditory Stimulation Test (working memory, speed of processing)

Statistical support for use:

Paper-and-pencil tests have been used for years and have many studies looking at their validity. The validity of the tests has been well documented (28).
Reliability of individual tests have been documented through various studies. Reliability measures range from 0.39-0.93 for these individual tests, with most being in the 0.63-0.75 range; almost all are below the ideal of 0.90 but above the minimally acceptable 0.60 (9,13). There have been no attempts to combine several tests with similar cognitive domain measures into a composite score. This method is used with computer NP batteries and increases the reliability.

Randolph et al. (28) reviewed seven studies involving paper-and-pencil NP testing with respect to sensitivity. They concluded that the studies demonstrated “some evidence that standard paper-and-pencil tests are sensitive to the effects of concussions, at least within the first five days of concussion” (28).

Table 26.1 Advantages and Disadvantages of Neuropsychological (NP) Tests

Paper-and-Pencil NP Test

Computerized NP Test

Advantages

▪ Extensive/thorough battery

▪ Validity in diagnosis

▪ Lower cost of equipment

▪ Instant scoring; instant information to provider

▪ More precise timing measures (reaction time measured to 1/100 of second)

▪ Multiple equivalent forms, which decreases practice effects

▪ Standardization of stimuli

▪ Efficient in sports medicine setting

▪ Practitioner need not be present; athletic training staff/medical assistants may administer

▪ Large numbers of athletes can be tested at once

▪ Short administration time (< 25 minutes)

▪ Useful in serial testing

▪ Centralized data storage, analysis, and reporting

▪ Internet-based delivery possible

▪ Do not need neuropsychologist to interpret

Disadvantages

▪ Not ideal for serial use

▪ Lack of equivalent alternative forms

▪ Poor test-retest reliability

▪ Susceptibility to interrater biases and practice effects

▪ Time required by athlete and neuropsychologist

▪ Must be interpreted by neuropsychologist

▪ Cost of administration and interpretation

▪ Practitioner need not be present

▪ Tests specific domains, and at times, patient may need more global assessment of cognitive functioning

▪ Baseline testing needed (to provide a better individual comparison)

▪ Cost of computers, software

Clinical utility has not been demonstrated; most studies have not demonstrated that paper-and-pencil NP tests can detect concussion once players are asymptomatic.

Computerized Neuropsychological Testing

Computerized batteries measure domains of memory, attention, concentration, processing speed, and reaction time, as well as symptoms.
Five computerized NP tests are currently available for the evaluation of sports-related concussions:
- Automated Neuropsychological Assessment Metric (ANAM)
- CogSport/Axon Sports Computerized Cognitive Assessment Tool (CCAT)
- HeadMinder Concussion Resolution Index (CRI)
- Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT)
- Concussion Vital Signs
  - □ This test will not be discussed; as of October 2011, there were no evidence-based studies in the literature on its use in sports-related concussions.
Advantages and Disadvantages of Paper-and-Pencil, and Computerized NP Testing: see Table 26.1.