Improving BCI performance through co-adaptation: Applications to the P300-speller




Abstract


A well-known neurophysiological marker that can easily be captured with electroencephalography (EEG) is the so-called P300: a positive signal deflection occurring at about 300 ms after a relevant stimulus. This brain response is particularly salient when the target stimulus is rare among a series of distracting stimuli, whatever the type of sensory input. Therefore, it has been proposed and extensively studied as a possible feature for direct brain-computer communication. The most advanced non-invasive BCI application based on this principle is the P300-speller. However, it is still a matter of debate whether this application will prove relevant to any population of patients. In a series of recent theoretical and empirical studies, we have been using this P300-based paradigm to push forward the performance of non-invasive BCI. This paper summarizes the proposed improvements and obtained results. Importantly, those could be generalized to many kinds of BCI, beyond this particular application. Indeed, they relate to most of the key components of a closed-loop BCI, namely: improving the accuracy of the system by trying to detect and correct for errors automatically; optimizing the computer’s speed-accuracy trade-off by endowing it with adaptive behavior; but also simplifying the hardware and time for set-up in the aim of routine use in patients. Our results emphasize the importance of the closed-loop interaction and of the ensuing co-adaptation between the user and the machine whenever possible. Most of our evaluations have been conducted in healthy subjects. We conclude with perspectives for clinical applications.


1


Introduction


A Brain-Computer Interface (BCI) is a system that connects the brain to a computer directly and avoids the need for peripheral nerve and muscle activities to execute user’s actions. A major aim of BCI research is to allow patients with severe motor disabilities to regain autonomy and communication abilities . This raises the crucial challenge of achieving a reliable control by measuring and interpreting brain activity on the fly. Due to the highly complex, noisy and variable nature of brain signals, especially those obtained with noninvasive recordings using scalp EEG, the computer sometimes misinterprets the signals and makes a decision that does not match the user’s intention. BCI is still a young field that is currently maturing by borrowing from several disciplines such as engineering, computational sciences, signal processing and neurophysiology. As a matter of fact, no BCI application has yet fully succeeded in being accurate and robust enough to be used routinely in clinical applications on impaired patients.


EEG is the most popular technique for BCI applications, simply because it is non-invasive, cheap and fairly easy to use at patient’s bedside. Moreover, tremendous efforts are being put into wireless and gel free EEG nowadays. EEG-based BCI are being explored for several years and a few neurophysiological markers have proved useful and promising. Probably the most prominent application that has emerged so far is the so-called P300-speller whose aim is to enable partially or fully locked-in patients to communicate . Although efficient, this application remains limited in several aspects . A central limitation lies in the need for high signal-to-noise ratio in order to make an accurate decision. This yields a challenging compromise between the speed and the accuracy of the spelling. In this paper, we synthetize results from our own recent online studies that aimed at optimizing this speed-accuracy trade-off. Two complementary strategies were used.


On the one hand, we made use of additional signals from the user. These are EEG responses to the display of the machine’s decision. Interestingly, these responses evoked by the machine’s feedback reflect whether the decision is erroneous or not. We proposed to measure them online in order to implement some automatic error correction.


On the other hand, we endowed the machine with adaptive behavior in order to make it more flexible and yet able to explicitly optimize the speed-accuracy trade-off at each trial. This approach relies on evidence accumulation such that the higher the signal-to-noise ratio in the EEG command, the faster the spelling. Evaluating this strategy online, we could evaluate the effect of this optimization on the user’s performance and motivation.


In both studies, conducted with healthy subjects, online data processing was performed within the OpenViBE software environment .


This paper is organized as follows. The Methods section starts with a short description of the P300-speller application. Then the two above approaches are introduced. Study 1 focusses on error signals during spelling through brain-computer interaction and evaluates the usefulness of such signals to correct for errors online, in an automated fashion. Study 2 introduces and validates an adaptive P300-speller and highlights the importance of co-adaptation. The results section summarizes the outcomes of these studies. In the last section, we discuss the implications and perspectives offered by those complementary studies.




2


Methods


2.1


General principle of the P300-Speller


The P300 signal is an EEG positive deflection that occurs approximately 300 ms after stimulus onset and is typically recorded over centro–parietal electrodes. This response is evoked by attention to rare stimuli in a random series of stimulus events (the oddball paradigm) and is even stronger when the subject is instructed to count the rare stimuli . It can be used to select items displayed on a computer screen . In practice, all possible items are displayed while the user focuses his attention (and gaze) onto the target item. Groups of items are successively and repeatedly flashed, but only the group that contains the target will elicit a P300 response. Correct spelling thus relies on both the user’s attentional state and the ability of the BCI to detect the P300 response.


We call a trial the succession of stimulations and observations that are needed to select one item. Each trial is made of several sequences, depending on the stopping criterion. A sequence of stimulations corresponds to the successive flashing of all the groups once, in a pseudo-random order. The longer the trial (i.e. the more sequences per trial), the more observations to rely on to find the target. Fig. 1 illustrates the general principle of the P300-Speller and the notion of sequence of stimulations to spell an item. Between 10 and up to 15 sequences are typically used in common implementations of the P300-Speller.




Fig. 1


Illustration of the general principle of a P300-Speller BCI: when the user focuses on the target and the target is flashed (e.g. letter H), the typical EEG evoked response will exhibit a strong N1 component followed by a P300 waveform (A); when the user focuses on the target but the target is not flashed, the typical EEG evoked response should be weaker with a smaller N1 component and no P300 waveform (B). Each group of letter is flashed, one after the other. One sequence is obtained as soon as every group has been flashed once. One trial or item spelling may consists in several sequences (C).


2.2


Study 1: making use of EEG error signals


We conducted this study to evaluate the benefits of automatic error correction during P300-based spelling. Error correction can be implemented in BCI thanks to EEG responses evoked by the feedback . Indeed, such evoked responses differ depending on whether the feedback is correct or not, that is whether the item detected by the BCI is indeed the one that the user wanted to spell. This can easily be measured in the case of copy spelling, when the machine knows what is the target letter. Hence, online feedbacks can be readily labelled as correct or incorrect for subsequent analysis of feedback evoked responses.


Fig. 2 shows the typical (averaged) evoked responses for correct and incorrect feedbacks as obtained during P300-based spelling. They are typically measured on fronto-central recording sites, thus requiring more anterior electrodes than the ones needed for spelling only.




Fig. 2


Typical (group averaged) EEG evoked responses observed on central sensors, to good (in green) and bad (in red) feedbacks. The difference between those two waveforms is due to a first component known as the Feedback Related Negativity (FRN) and a later P3-like component (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article).


A three-step procedure is required to evaluate automatic error correction online, as follows:



  • •

    an initial phase is used to calibrate the error detection algorithm. This requires acquiring samples of responses to both correct and incorrect feedbacks while the user is spelling in the absence of automated correction, so as to learn those responses for this particular individual;


  • •

    the initial training phase enables to optimize spatial filters and a probabilistic classifier that can then be used to detect error signals from each feedback related response;


  • •

    once an error has been detected, the automated correction consists in replacing the presumable erroneous item with another item, without user’s intervention. Since P300-based spelling also relies on probabilistic classification, items are ranked according to their probability of being the target. A natural strategy when an error is detected is then to propose the second most probable item according to this ranking.



In this online study, we evaluated error detection and correction at both the group and the individual level. In order to promote errors, we made the spelling challenging by considering very short (2 sequence-long) and short (4 sequence-long) trials.


Sixteen healthy volunteers participated in this study. Thirty-two EEG sensors were used for both spelling and error correction. Their placement followed the extended 10–20 systems. At the end of the experiment, the subjects were asked to answer a short questionnaire about their perception of the BCI performance in terms of both spelling and error correction.


All the details about this first study can be found in .


2.3


Study 2: the benefits of optimal stopping


In this second study, we endowed our P300-Speller BCI with adaptive decision-making. Instead of keeping the number of sequences constant, we enabled the BCI to stop in an optimal fashion . We wanted the BCI to be fast when it is confident about its decision and conversely, to be slow and to keep acquiring data when it is not yet clearly decided. This can be done by implementing some evidence accumulation process and an original stopping criterion that explicitly trades speed and accuracy.


Importantly, this approach relies on updating after each flash (instead of each sequence) the probability for each item to be the target. This is performed using Bayesian learning, which yields an evolutionary posterior probability distribution whose entropy reflects the confidence over the current target estimation or, in other words, the accumulated evidence in favor of each item. This information theoretic criterion is convenient since it is bounded and can thus easily be used to adjust the speed-accuracy trade-off.


Eleven healthy and BCI-naive subjects took part in this study. We compared our new adaptive mode with a traditional fixed mode. In the latter, the spelling was based on five sequences, while in the adaptive condition the speed-accuracy trade-off was individually tuned so as to reach roughly the same speed (five sequences) on average.


We also considered a further optimized BCI in terms of stimulations and EEG setup.


Regarding flashes, like in study 1, we departed from the traditional row/column way of grouping items. Instead, we grouped letters in a pseudo-random fashion and in a way that prevents from flashing neighboring items . This reduces errors due to distractions.


Besides, unlike in study 1, we reduced our number of EEG sensors down to 9, focusing on parieto-central, parietal, parieto-occipital and occipital sites, i.e. the back of the head where most of the relevant information come from .


We performed two complementary analyses out of this experiment.


The first (online) analysis simply enabled us to compare the two modes in terms of spelling speed and accuracy, as well as to ask the subjects about their preferences.


The second (offline) analysis consisted in reprocessing part of the data from both modes, but using a fixed and identical amount of evidence. In other words, instead of using an optimal stopping criterion, this second analysis considered a time-based criterion in order to compare modes based on the same number of trials. As a consequence, if a difference in performance between modes remains, it won’t be due to a difference in the criterion itself but to a virtuous effect of it onto the subject’s engagement or motivation.


More details about this second study can be found in .




2


Methods


2.1


General principle of the P300-Speller


The P300 signal is an EEG positive deflection that occurs approximately 300 ms after stimulus onset and is typically recorded over centro–parietal electrodes. This response is evoked by attention to rare stimuli in a random series of stimulus events (the oddball paradigm) and is even stronger when the subject is instructed to count the rare stimuli . It can be used to select items displayed on a computer screen . In practice, all possible items are displayed while the user focuses his attention (and gaze) onto the target item. Groups of items are successively and repeatedly flashed, but only the group that contains the target will elicit a P300 response. Correct spelling thus relies on both the user’s attentional state and the ability of the BCI to detect the P300 response.


We call a trial the succession of stimulations and observations that are needed to select one item. Each trial is made of several sequences, depending on the stopping criterion. A sequence of stimulations corresponds to the successive flashing of all the groups once, in a pseudo-random order. The longer the trial (i.e. the more sequences per trial), the more observations to rely on to find the target. Fig. 1 illustrates the general principle of the P300-Speller and the notion of sequence of stimulations to spell an item. Between 10 and up to 15 sequences are typically used in common implementations of the P300-Speller.


Tags:
Apr 23, 2017 | Posted by in PHYSICAL MEDICINE & REHABILITATION | Comments Off on Improving BCI performance through co-adaptation: Applications to the P300-speller

Full access? Get Clinical Tree
Get Clinical Tree app for offline access