Abstract
Objectives
Number-processing may be altered following brain injury and might affect the everyday life of patients. We developed the first ecological tool to assess number-processing disorders in brain-injured patients, the Ecological Assessment Battery for Numbers (EABN; in French, the BENQ). The aim of the present study was to standardize and validate this new tool.
Material and methods
Standardization included 126 healthy controls equally distributed by age, sex and sociocultural level. First, 17 patients were evaluated by the EABN; then scores for a subgroup of 10 were compared with those from a French analytical calculation test, the Évaluation Clinique des Aptitudes Numériques (ECAN). The concordance between the EABN and the ECAN was analyzed to determine construct validity. Discrimination indexes were calculated to assess the sensitivity of the subtests.
Results
Standardization highlighted a major effect of sociocultural level. In total, 9 of 17 patients had a pathological EABN score, with difficulties in telling time, making appointments and reading numerical data. The results of both the EABN and ECAN tests were concordant (Kendall’s w = 0.97). Finally, the discriminatory power was good, particularly for going to the movies, cheque-writing and following a recipe: scores were > 0.4.
Conclusion
The EABN is a new tool to assess number-processing disorders in adults. This tool has been standardized and has good psychometric properties for patients with brain injury.
1
Introduction
Numbers play a major role in our daily lives. Number-processing may be altered after a brain injury. Hécaen et al. presented the first classification of acalculia syndromes . Later, theoretical calculation models were developed, particularly by McCloskey and Caramazza and Dehaene et al. .
Routine evaluation of cognitive functions revealed a significant gap between patients’ performance in analytical laboratory-based tests and in daily life . Awareness of such a discrepancy has resulted in studies of the notion of ecological validity, which has become an increasingly relevant concept after the revision of the International Classification of Functioning, Disability and Health (ICF) . The ICF reflects a recent evolution from a medical and rehabilitative approach to disability toward an ecological approach : disability is based on cognitive deficits as well as interactions with the environment. Therefore, evaluation of disability is insufficient when focusing on only a simple assessment of the person’s analytically investigated cognitive dysfunction or impairment.
The current analytical tools for cognitive evaluation of number-processing are the EC301R-F , the Talk, Listen Connect: Phase II (TLC2) and the Évaluation Clinique des Aptitudes Numériques (ECAN) , the latter developed by Auzou . These tests facilitate identification of impaired processes, but to our knowledge, no study has investigated an “ecological” tool. Both approaches are complementary: ecological evaluation aims to quantify the impact of the deficits highlighted by analytical evaluation. Ecological assessment often places the patient in a more demanding situation in terms of the attention cost or the strategies required to reach the goal set by the examiner . As a result, the executive functions are solicited to a greater extent. The same patients may show different performance patterns during ecological and analytical evaluation. At times, the routine nature of some daily tasks can help the patient perform them. As regards to their complaints, Darrigrand et al. studied the scores of 127 aphasic patients on the Bordeaux Scale of Verbal Communication. Disorders were particularly predominant in writing cheques or using bank cards (57% of patients). This finding underscores the need for new evaluation tools that can objectify patients’ problems.
The Ecological Assessment Battery for Numbers (EABN) is an ecological protocol consisting of 18 subtests grouped together in 8 main tests. The tool was constructed to underscore the number-processing difficulties encountered in daily life by adults with brain injury. Test construction was based on neuropsychological number-processing models and involved the difficulties brain-injured patients have in daily life, particularly with numbers. The first version of the test, involving 83 subjects, was stratified and standardized, and resulted in data for the performance of 12 aphasic patients . This first study highlighted the sensitivity of the EABN to capture global changes as well as specific disorders experienced by aphasic patents. We analyzed the validity of this version by consulting various experts (occupational therapists, neuropsychologists and speech therapists), who graded the relevance of the different tests. The recorded scores underlined the good validity of most of the tests, and the expert judgments were concordant. Inter-judge reliability was analyzed, with slight variability observed. Finally, the study showed good discriminatory power for most of the tests, except those pertaining to following a recipe, going to the movies, and telling time.
However, the study revealed a number of limitations; some tests were considered by experts to be less relevant than others. Experts also considered that certain supports needed improvement and that some of the rating criteria were insufficiently precise and therefore likely to produce subjectivity. To enhance the psychometric qualities of the EABN and using previously mentioned results, we endeavored to modify some of the tests. We eliminated those that received the poorest grades from the expert judges and modified some of the rating criteria to improve inter-judge reliability and increase the discriminatory power of the least sensitive tests.
The aim of the present work was to study the standardization and provide an initial validation of the number-processing assessment battery EABN in daily life situations. The secondary aim was to study the effect of demographic variables on the scores of a healthy control population, establish stratified standards, and compare the performance of a group of brain-injured patients on the EABN and an analytical test used as a gold standard.
2
Patients and methods
2.1
EABN
The EABN is an ecological test battery addressing the use of numbers in daily life; it consists of 8 tests ( Table 1 ) corresponding to common situations in everyday life involving numbers: telling time, going shopping, writing a cheque, writing down contact information, cooking etc. All tests and subtests are timed; however, subjects are asked not to take time measurement into account and to proceed as usual, thereby not being under pressure. The maximum total score is 40. The EABN material includes the rating sheet and the testing handbook, a chronometer, a calculator, a schedule, and play money. The complete test battery is completed individually by control subjects. The test generally takes place in a calm room and lasts 10 to 25 min. In most cases, professionals in charge of the EABN test went to the homes of control subjects.
Tests | Max score | Subtests |
---|---|---|
Telling time | 4 | Analog dials |
Digital dials | ||
Shopping | 9 | Evaluating the prices of everyday items |
Approximate calculation of a total amount | ||
Cash payment of an exact amount of money | ||
Verification with a calculator a total amount | ||
Cheque-writing | 6 | Calculating a percentage |
Payment by cheque an amount of money specified orally | ||
Making appointments | 9 | Calculating a duration |
Placing a date and an hour on a schedule | ||
Writing down dictated contact information | ||
Providing one’s own contact information | ||
Going to the movies | 4 | Choosing an hour according to temporal data |
Verifying and calculating the change given | ||
Composing a digital code | 1 | Composing a digital code given orally in digicode |
Following a recipe | 2 | Placing specified quantities on a scale |
Reading numerical data | 5 | Reading contact information |
Reading sentences containing numerical data |
2.2
Standardization of the EABN
The subjects included for the standardization phase were 20 to 79 years old with equal distribution by age (20–39, 40–59 and 60–79 years), sex and sociocultural level (SCL1: ≤ French certificate of higher education; SCL2, baccalaureate degree; SCL3, higher studies). Subjects had to understand and use the French language and have no abnormal neurological or psychiatric history. We produced descriptive (mean, SD, percentile 5) and inferential statistics using Student’s t test to estimate the effect of demographic variables on the EABN score and to establish standards according to the recorded effects. We preliminarily determined that if the score distribution were non-normal, the 5th percentile would be considered the pathological threshold.
2.3
Concurrent validity analysis
We performed a validity study with brain-injured patients who were hospitalized between September 2011 and December 2012 in the physical and rehabilitation medicine unit of La Pitié-Salpêtrière Hospital, Paris. Inclusion criteria were age 20 to 79 years, with no sensory or motor disability that disallowed taking the tests.
The patients were evaluated by both the EABN and a French analytical calculation test, the ECAN . The ECAN is an analytical battery designed to test abilities in calculation and number-processing. It was developed and standardized by Auzou et al. according to the French language adaptation of the Number-Processing and Calculation (NPC) battery . It includes 35 tasks assessing the different abilities in counting, the different aspects of number comprehension, numerical transcoding, calculation, arithmetic reasoning and conceptual knowledge. It assesses the calculation skills needed for 4 operations (simple fact retrieval, rule-based processing, mental calculation, written calculation) as well as problem resolution. The ECAN is more complete and provides more accurate evaluation of impaired processes than the EC301 and has been standardized with 423 healthy control subjects. It is based on 4 main areas: knowledge of numbers, transcoding, calculation and common knowledge of numbers. All tests are timed. They differ in several ways from the NPC: there are more items for the transcoding tests but some common items for different tasks, thereby allowing comparison of performance. Finally, the NPC approximate test, which practically all control subjects failed, has been replaced by a test derived from the Stanescu-Cosson test .
To study the degree of concordance between the ecological battery (EABN) and the analytical battery (ECAN), our reference battery, we calculated Kendall’s w coefficient . The Kendall w coefficient ranges from 0 to 1: the closer to 1, the greater the degree of concordance. The objective was to classify patients by rank on the test based on comparisons of the recorded grades.
2.4
Construct validity analysis
To establish construct validity, we analyzed the discriminatory power of each subtest. We calculated the discrimination indexes corresponding to the difference between the difficulty indexes of the 2 extreme groups, using the standardization data and the data collected from patients. The values for the P index for relative difficulty of the items range from 0 to 1: values close to 0 indicate an item for which few subjects successfully performed, and values close to 1 denote an item for which a high proportion of subjects successfully performed. When an item is rated on a scale containing several points, the difficulty index corresponds to the mean rating achieved for this item by the subjects as a whole. The mean rating for the item is divided by the maximum grade: for example, if the maximum grade for the item is 5, the mean grade for that item is divided by 5.
The P index equals the total sum of the ratings divided by the number of subjects. The item discrimination index D corresponds to the difference between the difficulty index for an item with regard to the “strong” group ( P +) and the difficulty index for the “weak” group ( P –). The larger the difference in D , the more the item discriminates between subjects with a high total score and those with a low total score. The “strong” group consists of subjects with a total score placing them in the upper 27% of the total population, and the “weak” group consists of those whose score places them in the lower 27% . To constitute these groups, we used the detailed standardization data ( n = 126) and the results recorded for patients ( n = 17), for a relatively large sample.
The discrimination index D can take on any value from –1 to +1. A value of zero means that a given item was as successfully negotiated by subjects with a low total score as by those with a high total score. A negative value means that with respect to an item, weaker subjects were more successful than stronger subjects. A discrimination index D > 0.4 is very good; 0.2 to 0.4, good; 0.1 to 0.2, average; and < 0.1 or negative, insufficient.
2
Patients and methods
2.1
EABN
The EABN is an ecological test battery addressing the use of numbers in daily life; it consists of 8 tests ( Table 1 ) corresponding to common situations in everyday life involving numbers: telling time, going shopping, writing a cheque, writing down contact information, cooking etc. All tests and subtests are timed; however, subjects are asked not to take time measurement into account and to proceed as usual, thereby not being under pressure. The maximum total score is 40. The EABN material includes the rating sheet and the testing handbook, a chronometer, a calculator, a schedule, and play money. The complete test battery is completed individually by control subjects. The test generally takes place in a calm room and lasts 10 to 25 min. In most cases, professionals in charge of the EABN test went to the homes of control subjects.
Tests | Max score | Subtests |
---|---|---|
Telling time | 4 | Analog dials |
Digital dials | ||
Shopping | 9 | Evaluating the prices of everyday items |
Approximate calculation of a total amount | ||
Cash payment of an exact amount of money | ||
Verification with a calculator a total amount | ||
Cheque-writing | 6 | Calculating a percentage |
Payment by cheque an amount of money specified orally | ||
Making appointments | 9 | Calculating a duration |
Placing a date and an hour on a schedule | ||
Writing down dictated contact information | ||
Providing one’s own contact information | ||
Going to the movies | 4 | Choosing an hour according to temporal data |
Verifying and calculating the change given | ||
Composing a digital code | 1 | Composing a digital code given orally in digicode |
Following a recipe | 2 | Placing specified quantities on a scale |
Reading numerical data | 5 | Reading contact information |
Reading sentences containing numerical data |
2.2
Standardization of the EABN
The subjects included for the standardization phase were 20 to 79 years old with equal distribution by age (20–39, 40–59 and 60–79 years), sex and sociocultural level (SCL1: ≤ French certificate of higher education; SCL2, baccalaureate degree; SCL3, higher studies). Subjects had to understand and use the French language and have no abnormal neurological or psychiatric history. We produced descriptive (mean, SD, percentile 5) and inferential statistics using Student’s t test to estimate the effect of demographic variables on the EABN score and to establish standards according to the recorded effects. We preliminarily determined that if the score distribution were non-normal, the 5th percentile would be considered the pathological threshold.
2.3
Concurrent validity analysis
We performed a validity study with brain-injured patients who were hospitalized between September 2011 and December 2012 in the physical and rehabilitation medicine unit of La Pitié-Salpêtrière Hospital, Paris. Inclusion criteria were age 20 to 79 years, with no sensory or motor disability that disallowed taking the tests.
The patients were evaluated by both the EABN and a French analytical calculation test, the ECAN . The ECAN is an analytical battery designed to test abilities in calculation and number-processing. It was developed and standardized by Auzou et al. according to the French language adaptation of the Number-Processing and Calculation (NPC) battery . It includes 35 tasks assessing the different abilities in counting, the different aspects of number comprehension, numerical transcoding, calculation, arithmetic reasoning and conceptual knowledge. It assesses the calculation skills needed for 4 operations (simple fact retrieval, rule-based processing, mental calculation, written calculation) as well as problem resolution. The ECAN is more complete and provides more accurate evaluation of impaired processes than the EC301 and has been standardized with 423 healthy control subjects. It is based on 4 main areas: knowledge of numbers, transcoding, calculation and common knowledge of numbers. All tests are timed. They differ in several ways from the NPC: there are more items for the transcoding tests but some common items for different tasks, thereby allowing comparison of performance. Finally, the NPC approximate test, which practically all control subjects failed, has been replaced by a test derived from the Stanescu-Cosson test .
To study the degree of concordance between the ecological battery (EABN) and the analytical battery (ECAN), our reference battery, we calculated Kendall’s w coefficient . The Kendall w coefficient ranges from 0 to 1: the closer to 1, the greater the degree of concordance. The objective was to classify patients by rank on the test based on comparisons of the recorded grades.
2.4
Construct validity analysis
To establish construct validity, we analyzed the discriminatory power of each subtest. We calculated the discrimination indexes corresponding to the difference between the difficulty indexes of the 2 extreme groups, using the standardization data and the data collected from patients. The values for the P index for relative difficulty of the items range from 0 to 1: values close to 0 indicate an item for which few subjects successfully performed, and values close to 1 denote an item for which a high proportion of subjects successfully performed. When an item is rated on a scale containing several points, the difficulty index corresponds to the mean rating achieved for this item by the subjects as a whole. The mean rating for the item is divided by the maximum grade: for example, if the maximum grade for the item is 5, the mean grade for that item is divided by 5.
The P index equals the total sum of the ratings divided by the number of subjects. The item discrimination index D corresponds to the difference between the difficulty index for an item with regard to the “strong” group ( P +) and the difficulty index for the “weak” group ( P –). The larger the difference in D , the more the item discriminates between subjects with a high total score and those with a low total score. The “strong” group consists of subjects with a total score placing them in the upper 27% of the total population, and the “weak” group consists of those whose score places them in the lower 27% . To constitute these groups, we used the detailed standardization data ( n = 126) and the results recorded for patients ( n = 17), for a relatively large sample.
The discrimination index D can take on any value from –1 to +1. A value of zero means that a given item was as successfully negotiated by subjects with a low total score as by those with a high total score. A negative value means that with respect to an item, weaker subjects were more successful than stronger subjects. A discrimination index D > 0.4 is very good; 0.2 to 0.4, good; 0.1 to 0.2, average; and < 0.1 or negative, insufficient.