Virtual reality and gaming technology for sensorimotor and cognitive rehabilitation

The effectiveness of sensorimotor function retraining is influenced by the quantity, duration, and intensity of practice. High-intensity exercise programs are often fraught with low compliance and adherence. Exercise adherence is a significant hurdle to overcome, especially in the presence of a long-term chronic illness. Maintaining motivation and engagement are central to long-term functional improvement and success. Self-worth, motivation, and activity enjoyment have been reported to be vital to long-term exercise adherence. It has also been suggested that when a client “focuses” on a game than his or her impairment, exercise becomes more enjoyable, motivating, and is more likely to be maintained over the many trials needed to induce plastic changes in the nervous system. Providing a treatment that is fun, motivating, and distracting while simultaneously enhancing function would serve to improve exercise adherence and, therefore, effect motor learning and functional outcomes. One method that is showing great potential to meet this need is the use of VR games for rehabilitation.

VR is defined as “an immersive and interactive system that provides users with the illusion of entering a virtual world.” Immersion and interactivity suggest a virtual or imaginary world that can be entered and explored. Gobbetti and Scateni 1998 state the “goal of virtual reality is to put the user in the loop of a real-time simulation, immersed in a world that can be both autonomous and responsive to its actions.” The user is connected to the VR system as part of the input/output loop, allowing individuals to provide input to the virtual environment (VE) and experience the result of that input. In 2003 Burdea described VR as “a high-end user-computer interface that involves real-time simulation and interactions through multiple sensorial channels.” These definitions require that legitimate VR systems provide a computer simulated situation or environment with which the user can interact in real time, giving a sense of actually being in that situation or environment. To place the user within a loop of real-time simulation, VR systems require an output device or visual interface (flat screen or head mounted display) and input devices for interaction (mouse, joystick, data glove) and tracking (tracking device). The software or VE can be viewed through the output device and manipulated or interacted with via the input devices. Fig. 2 shows the components of a VR system.

Components of a VR system (I, input; O, output).

From Burdea G, Coiffet P. Virtual reality technology. 2nd edition. New York: Wiley; 1994, copyright 2003. Reprinted with permission of John Wiley & Sons, Inc.

VR applications, historically used for the training of motor tasks involving highly complex activities such as surgical techniques, flight simulation, and military exercises allow the user to enter a simulated world through multimodal sensory feedback. With the reduction in cost of computing hardware and software technologies, and the increasing expansion of the video game market, video game consoles and video games can be labeled as VR systems. These VR systems are interactive, immersive, and provide the user with a sense of presence within a VE. The use of video games and VR systems for rehabilitation has expanded rapidly over the past few years. Early research in the area of the use of VR systems to assist people to relearn how to move suggests that VR game-based technology can be used effectively to improve motor skill rehabilitation of a range of functional deficits. VR systems demand focus and attention, but can motivate the user to move, and can provide the user with a sense of achievement even if they cannot perform that task in the “real world.”

There is a compelling and ethical motivation to address the needs of individuals who are aging with disabilities by promoting home-based access to low-cost, interactive VR systems designed to engage and motivate individuals to participate with game-driven physical activities and rehabilitation programming. The creation of such systems can serve to enhance, maintain, and rehabilitate the motor processes that underlie the integrated functional behaviors that are needed to maximize independence and quality of life beyond what exists with currently available, labor intensive, underutilized, and more costly approaches. There are fundamental and dramatic advantages inherent in the use of game-based approaches for treating cognitive, motor, and behavioral impairments. VR offers an ideal core technology because it allows for the creation of computer-generated 3-dimensional simulations, within which hierarchical task relevant challenges can be delivered and titrated across a range of difficulty levels. In this way, an individual’s treatment plan can be customized to begin at a stimulus challenge level that is attainable and comfortable for them, and then proceed with a gradual progression of challenge that is informed by the individual’s performance in real time. Furthermore, VR game-based environments allow for the presentation of more ecologically relevant stimuli that are embedded in meaningful and familiar contexts. By designing VEs that not only “look like” the real world but also actually incorporate challenges that require real-world functional behaviors, the ecological validity of cognitive/motor interventions can be enhanced. Within such simulations, the complexity of stimulus challenges found in naturalistic settings can be delivered while still maintaining the experimental control required for rigorous scientific analysis and replication. Alternatively, it is possible that the use of novel environments and game play can engage and motivate the user to perform activities that they would not normally perform of their own accord. VR technology also supports precise and detailed capture and analysis of complex responses (ie, kinematic assessments of speed, accuracy, timing, consistency).

VR has recently been explored as a therapeutic tool to retrain faulty movement patterns resulting from neurologic dysfunction and to augment rehabilitation of the upper limb of patients in the chronic phase after stroke. VR technology using specialized interface devices has been applied to improve motor skill rehabilitation of functional deficits including reaching, hand function, and walking. Substantial effort and expertise have been focused on developing these innovative technology/interventions to exploit the neuroplastic properties associated with the sensorimotor systems in the adult brain. It has been proposed that such VR-based activities can be delivered in the home via a telerehabilitation approach to support these patients’ increased access to rehabilitation and preventative exercise programming. Moreover, when such VR training is embedded in an interactive game-based context, there is a potential to enhance the engagement and motivation needed to drive neuroplastic changes that underlie motor process maintenance and improvement. However, home-based VR systems need to be affordable, and easy to deploy and maintain, while still providing the interactional fidelity required to produce the meaningful motor activity required to foster rehabilitative aims and promote transfer to real-world activities.

Use of computer-based cognitive fitness and assessment technologies have rapidly expanded over the past 5 years with the introduction of a range of game-based training tools, such as BrainAge (Nintendo of America, Redmond, WA, USA), BrainFitness (PositScience Corporation, San Francisco, CA, USA), and Posit Science (Posit Science Corporation, San Francisco, CA, USA). Sales of cognitive fitness and assessment products in the United States have grown an estimated $100 million to $265 million between the period of 2005 and 2008. Many products offer cognitive fitness games for regular long-term use. These brain-training games are often based on common neuropsychological assessment tools. Common areas of focus for games include long-term and short-term memory, language, executive function, visuospatial orientation, and critical thinking. The Posit Science program has been shown to reduce the effects of age-related cognitive decline through training in areas of memory, concentration, language, executive function, reaction time, and visual attention. BrainFitness by Dakim ( http://www.dakim.com/dakim ) is a touch-screen device that assesses and trains long- and short-term memory, language, computation, visuospatial orientation, and critical thinking. These types of applications may include the neuropsychological tests that are often used for assessment, raising some concern that these tasks are training the user for the test rather than training the cognitive domain itself. Playing casual computer games has also been shown to improve cognitive abilities in older adults. Older adults who play video games have shown improvements in reaction time, cognitive functioning, intelligence, visuomotor coordination, attention and concentration, and self-esteem and quality of life. The use of physical computer games for improvement of cognitive and functional deficits has to be shown; however, several researchers across the globe are currently exploring the use of video game consoles, such as the Sony Playstation2 EyeToy (Sony Corporation, Minato, Tokyo, Japan) and the Nintendo Wii and WiiFit (Nintendo Co. Ltd, Kyoto, Japan) with groups of individuals who are healthy aged and those aging with a disability.

One of the issues with the use of commercial games and consoles such as the Nintendo Wii, Nintendo WiiFit, and Sony Playstation2 EyeToy is that these applications have been developed and tested for the purpose of entertainment. Commercial video game consoles and games have been tested on a diverse audience; however, they were not designed as medical devices or with a primary focus of a rehabilitation tool. Because these games were initially designed for entertainment, the game-play mechanics are not entirely applicable to those with disabilities. Thus, there is a need to gain a better understanding of the qualities of the commercial games and gaming consoles before these devices can be approved as appropriate rehabilitation tools for people with disabilities. Many of these applications are too difficult to use as a therapy tool by people with disabilities, and cannot be accessed or altered to improve usability. The development of low-cost VR applications for rehabilitation must provide versatile and accurate interaction devices with the ability to track the user’s movements and present information about performance to the user and their therapist in an appropriate and user-friendly format. The games and VEs must allow the user to interact in a way that is appropriate for their level of impairment, and must be easily changed to increase the level of challenge as the user improves.

The potential of the use of VR game-based applications, as they pertain to those aging with and into disability, involves the integration of cognitive and physical tasks within an engaging and meaningful environment in which it is safe to (or in some cases the system makes the user unable to) make mistakes and errors. The operating premise is that cognitive demand for daily functional behaviors increases with age-related physical decline. The approach is to enhance physical function in meaningful activities by harnessing advances in science and technology (especially immersive technologies) that increase sensorimotor capacity and unload cognitive demands in those aging with and into disability. By taking advantage of VR technologies for facilitating focused task-specific practice and gaming for enjoyment and adherence, it is anticipated that the core processes at the body function level and activities level as represented in the ICF model will be affected, which can enable active participation and enhance quality of life for the intended beneficiaries. Examples of clinical situations in which the use of VR and gaming technologies can serve to complement and improve on existing assessment and training applications are described in the following sections.

VR rehabilitation for balance impairments

Balance, defined as the ability to maintain the body’s center of gravity over the base of support, requires full integrity of an elegant and complicated system. Postural control is the ability to maintain an upright position (in sitting or standing) when stationary or when performing activities. The postural control system incorporates (1) sensory detection (through the visual, vestibular, and somatosensory systems), (2) integration of sensorimotor information within the central nervous system, and (3) proper coordinated movement patterns and responses. When deterioration or damage occurs to one or more of these systems, impairment of balance occurs. More specifically, an ability to accurately assess (sense) the position of the center of gravity relative to the base of support will disrupt balance. Second, when automatic movements to maintain balance are triggered too slowly or ineffectively, balance becomes distorted. Impairments affecting balance, such as muscular weakness, proprioceptive deficits, limitations in range of motion, and visual/vestibular deficits, can lead to falls and restrict an individual’s normal motor activities, thereby limiting one’s sense of independence, and adversely affecting the quality of life. The prevalence of balance disorders in the overall population of the United States is unknown. It is estimated that at least half of the overall population of the United States are affected by a balance or vestibular disorder sometime during their life ( http://www.cdc.gov ). Thus, there is a high probability that individuals aging with and into disability will experience balance problems sometime in their lifetime. Finding low-cost and safe balance training tools is imperative as the baby boomers continue to age and potentially increase the risk of falls.

The recent release and worldwide acceptance and enjoyment of Nintendo’s WiiFit has provided significant evidence for the notion that exercise can be fun, provided it is presented in a manner that is entertaining, motivating, and distracting. The Nintendo WiiFit challenges balance in a variety of ways. By using a low-tech forceplate platform, the player controls the movements of an onscreen avatar by shifting their weight on the platform. When the player shifts right, the avatar shifts right and vice versa. Games such as hoola-hooping, defending a soccer goal, floating down a river, skiing down a mountain, and a variety of yoga positions are used in this device. This type of game challenges balance in a variety of ways. The player must watch, that is, engage the visual system, in a way that he or she can interpret the images presented and decide if movements to the right or to the left would be most successful. Visual feedback is given by, for example, seeing the avatar move between ski flags. Furthermore, knowledge of performance is provided visually as an individual moves into a yoga position, and is challenged to maintain his or her balance as precisely as possible by keeping a small red dot inside a yellow circle. Challenging the player to maintain the center of pressure and center of balance in a precise location while using visual feedback to guide the cursor on the screen tests the player’s balance symmetry. The games challenge a person’s vestibular system as the person moves through the challenging positions, and the vestibular system is being activated as the player moves into the various positions. If an individual has a deficit in any of these systems (visual, vestibular, or somatosensory) these games become a significant challenge. The Sony Playstation EyeToy has shown promising results as a low-cost tool for balance rehabilitation. The EyeToy (Sony Computer Entertainment, Europe) is a projected video capture system that uses a motion-sensitive USB camera to display mirror image of the player into the game, allowing the player to interact with the VE using the entire body. In a feasibility study in 2007, Flynn and colleagues applied the EyeToy with an individual with chronic stroke for 20 1-hour sessions. The game’s task requirements included target-based motion, dynamic balance, and motor planning. The study showed clinically relevant improvements in the Dynamic Gait Index, and trends toward improvement on the Fugl-Meyer Assessment, Berg Balance Scale, UE Functional Index, Motor Activity Log, and Beck Depression Inventory.

Physical activities, including strengthening exercises, tai chi, dancing, and walking have been shown to improve balance and decrease the risk of falls in older adults. Dancing is a fun filled, physical, and expressive activity open for participation by people of all levels of coordination. Modified dance-based exercises have been shown to improve endurance and balance in older adults. Dancing can improve and maintain strength, endurance, balance, and coordination. Dancing can also increase body awareness and kinesthetic awareness, challenge attention, and improve self-esteem and confidence. Dance-based training has been shown to be beneficial in improving static balance and reducing the risk of a fall. The effectiveness of physical activity and aerobic exercise in preservation of balance and reduction of falls has been well documented. A systematic review of prevention of falls exercise regimes found that the greatest relative effects of exercise on fall rates were seen in programs that included a combination of a higher total dose of exercise (>50 hours over the trial period) and consisted of challenging balance exercises (exercises conducted while standing in which people aimed to stand with their feet closer together or on one leg, minimize use of their hands to assist, and practice controlled movements of the center of mass). Dancing can be considered a challenging balance exercise, because dancers are required to change their base of support while moving to music, not looking at their feet, and controlling their weight shift between poses or movements. Dancing can increase body awareness, kinesthetic awareness, and improve and maintain flexibility, balance, coordination, endurance, strength, and self-esteem. Hopkins and colleagues found in 1990 that a 12-week low-impact aerobic dance program significantly improved cardiorespiratory endurance, strength/endurance, body agility, flexibility, body fat, and balance in a sample of 53 women older than 65 years. Shigematsu and colleagues in 2002 similarly found that, compared with a control group, a 12-week danced-based exercise class showed significantly greater improvements in balance measures (single leg balance, functional reach, and walking time around obstacles) in 38 women aged 72 to 87 years. Hui and colleagues in 2009 found that a 12-week danced-based exercise program of a sample of 111 older adults significantly improved balance measures (6-minute walk test, flexibility, timed up-and-go test), resting heart rate, endurance, and physical and mental well-being (SF-36) compared with a control group. Most of the dance group felt the intervention improved their health status. These findings show that dancing has physical and psychological benefits, and should be promoted as a form of leisure activity for older adults. Dance games, such as Dance-Dance Revolution, first developed by Konami in 1998, are physically active video games that challenge endurance and dynamic balance. However, the primary function of current dance and music-based video games such as Dance-Dance Revolution is entertainment. These games are beneficial from a rehabilitative perspective because they require skills related to timing, rhythm, balance, endurance, and strength. Using a dance mat with areas that the player must step on in time with cues on the screen, these games have the ability to get the player to perform specific moves in a specific pattern, and have been shown to improve activity and mood as well as reduce weight in children and youth. Dance-Dance Revolution has been shown to increase heart rate and motivate adolescents to exercise in an effort to improve motor coordination, self-esteem, and aerobic fitness. Dance games are currently being used in schools and a variety of workplaces across the United States to encourage children and adults to become more active. These games have the potential to be used as a part of a balance training prevention program for older adults (with and without disability) in clinics or within the home.

VR rehabilitation for balance impairments

Balance, defined as the ability to maintain the body’s center of gravity over the base of support, requires full integrity of an elegant and complicated system. Postural control is the ability to maintain an upright position (in sitting or standing) when stationary or when performing activities. The postural control system incorporates (1) sensory detection (through the visual, vestibular, and somatosensory systems), (2) integration of sensorimotor information within the central nervous system, and (3) proper coordinated movement patterns and responses. When deterioration or damage occurs to one or more of these systems, impairment of balance occurs. More specifically, an ability to accurately assess (sense) the position of the center of gravity relative to the base of support will disrupt balance. Second, when automatic movements to maintain balance are triggered too slowly or ineffectively, balance becomes distorted. Impairments affecting balance, such as muscular weakness, proprioceptive deficits, limitations in range of motion, and visual/vestibular deficits, can lead to falls and restrict an individual’s normal motor activities, thereby limiting one’s sense of independence, and adversely affecting the quality of life. The prevalence of balance disorders in the overall population of the United States is unknown. It is estimated that at least half of the overall population of the United States are affected by a balance or vestibular disorder sometime during their life ( http://www.cdc.gov ). Thus, there is a high probability that individuals aging with and into disability will experience balance problems sometime in their lifetime. Finding low-cost and safe balance training tools is imperative as the baby boomers continue to age and potentially increase the risk of falls.

The recent release and worldwide acceptance and enjoyment of Nintendo’s WiiFit has provided significant evidence for the notion that exercise can be fun, provided it is presented in a manner that is entertaining, motivating, and distracting. The Nintendo WiiFit challenges balance in a variety of ways. By using a low-tech forceplate platform, the player controls the movements of an onscreen avatar by shifting their weight on the platform. When the player shifts right, the avatar shifts right and vice versa. Games such as hoola-hooping, defending a soccer goal, floating down a river, skiing down a mountain, and a variety of yoga positions are used in this device. This type of game challenges balance in a variety of ways. The player must watch, that is, engage the visual system, in a way that he or she can interpret the images presented and decide if movements to the right or to the left would be most successful. Visual feedback is given by, for example, seeing the avatar move between ski flags. Furthermore, knowledge of performance is provided visually as an individual moves into a yoga position, and is challenged to maintain his or her balance as precisely as possible by keeping a small red dot inside a yellow circle. Challenging the player to maintain the center of pressure and center of balance in a precise location while using visual feedback to guide the cursor on the screen tests the player’s balance symmetry. The games challenge a person’s vestibular system as the person moves through the challenging positions, and the vestibular system is being activated as the player moves into the various positions. If an individual has a deficit in any of these systems (visual, vestibular, or somatosensory) these games become a significant challenge. The Sony Playstation EyeToy has shown promising results as a low-cost tool for balance rehabilitation. The EyeToy (Sony Computer Entertainment, Europe) is a projected video capture system that uses a motion-sensitive USB camera to display mirror image of the player into the game, allowing the player to interact with the VE using the entire body. In a feasibility study in 2007, Flynn and colleagues applied the EyeToy with an individual with chronic stroke for 20 1-hour sessions. The game’s task requirements included target-based motion, dynamic balance, and motor planning. The study showed clinically relevant improvements in the Dynamic Gait Index, and trends toward improvement on the Fugl-Meyer Assessment, Berg Balance Scale, UE Functional Index, Motor Activity Log, and Beck Depression Inventory.

Physical activities, including strengthening exercises, tai chi, dancing, and walking have been shown to improve balance and decrease the risk of falls in older adults. Dancing is a fun filled, physical, and expressive activity open for participation by people of all levels of coordination. Modified dance-based exercises have been shown to improve endurance and balance in older adults. Dancing can improve and maintain strength, endurance, balance, and coordination. Dancing can also increase body awareness and kinesthetic awareness, challenge attention, and improve self-esteem and confidence. Dance-based training has been shown to be beneficial in improving static balance and reducing the risk of a fall. The effectiveness of physical activity and aerobic exercise in preservation of balance and reduction of falls has been well documented. A systematic review of prevention of falls exercise regimes found that the greatest relative effects of exercise on fall rates were seen in programs that included a combination of a higher total dose of exercise (>50 hours over the trial period) and consisted of challenging balance exercises (exercises conducted while standing in which people aimed to stand with their feet closer together or on one leg, minimize use of their hands to assist, and practice controlled movements of the center of mass). Dancing can be considered a challenging balance exercise, because dancers are required to change their base of support while moving to music, not looking at their feet, and controlling their weight shift between poses or movements. Dancing can increase body awareness, kinesthetic awareness, and improve and maintain flexibility, balance, coordination, endurance, strength, and self-esteem. Hopkins and colleagues found in 1990 that a 12-week low-impact aerobic dance program significantly improved cardiorespiratory endurance, strength/endurance, body agility, flexibility, body fat, and balance in a sample of 53 women older than 65 years. Shigematsu and colleagues in 2002 similarly found that, compared with a control group, a 12-week danced-based exercise class showed significantly greater improvements in balance measures (single leg balance, functional reach, and walking time around obstacles) in 38 women aged 72 to 87 years. Hui and colleagues in 2009 found that a 12-week danced-based exercise program of a sample of 111 older adults significantly improved balance measures (6-minute walk test, flexibility, timed up-and-go test), resting heart rate, endurance, and physical and mental well-being (SF-36) compared with a control group. Most of the dance group felt the intervention improved their health status. These findings show that dancing has physical and psychological benefits, and should be promoted as a form of leisure activity for older adults. Dance games, such as Dance-Dance Revolution, first developed by Konami in 1998, are physically active video games that challenge endurance and dynamic balance. However, the primary function of current dance and music-based video games such as Dance-Dance Revolution is entertainment. These games are beneficial from a rehabilitative perspective because they require skills related to timing, rhythm, balance, endurance, and strength. Using a dance mat with areas that the player must step on in time with cues on the screen, these games have the ability to get the player to perform specific moves in a specific pattern, and have been shown to improve activity and mood as well as reduce weight in children and youth. Dance-Dance Revolution has been shown to increase heart rate and motivate adolescents to exercise in an effort to improve motor coordination, self-esteem, and aerobic fitness. Dance games are currently being used in schools and a variety of workplaces across the United States to encourage children and adults to become more active. These games have the potential to be used as a part of a balance training prevention program for older adults (with and without disability) in clinics or within the home.