Computer Access – Augmenting the Keyboard and Mouse

Key Terms

Control Interface

Continuous Input

Digital Recording

Emulation

Graphical User Interface (GUI)

Text-To-Speech Programs

USB port

In the United States, many computer adaptations are mandated by the legislation described in Chapter 1 (e.g., PL 508). The best approach to adapting a computer for use by individuals with physical limitations is to begin with the simplest modifications designed for the most minimal of physical limitations on the part of the user. If the minimal adaptation is not sufficient for a given user, then more complex adaptations can be evaluated.

No matter how complex the adaptation, the goal is always to make sure that: (1) all of the functions of the computer are available to the user who has a disability; and (2) all application software that runs on the unmodified computer runs on the adapted computer. All the keyboard keys, including modifier (e.g., shift, control, alt) and special function keys, and all the mouse functions, such as point, click, and drag, must be available on the adapted input system. If a program (e.g., word processor) works with the standard computer, then it should work with the adaptations that are provided for the individual with a disability.

Keyboards – typical and unique

On-screen Keyboards

On-screen keyboards have a selection set on the screen that resembles a physical keyboard (Figure 7-1, B). In order to enter a character or select a function, the user of the on-screen keyboard positions the cursor inside the desired “key” or icon on the screen. Movement of the cursor can be by mouse, trackball, joystick, control interface array, head-controlled mouse, or touch screens. Once the cursor is located inside the targeted screen item, the user makes the selection by tapping on the screen, activating another control interface, or by holding the cursor on the choice until it is accepted by the device. This is often referred to as acceptance time or dwell time.

Many on-screen keyboards allow changes in the keyboard arrangement, size of the on-screen keys, and location of the keyboard on the screen. Some on-screen keyboards also include other characteristics such as a word prediction feature that displays frequently used words as the first few characters of a word are typed. Other adjustments intended to make use easier and more efficient include horizontal and vertical cursor movement speed control, keyboard layouts, and location of the keyboard image on the screen (e.g., top or bottom, depending on the type of application program that is running). Touch screens that provide feedback (i.e., a sound when you activate a key or a change of color to indicate a key has been activated) are often helpful for people with cognitive impairment or for those who are just learning to use these kinds of devices. Also, the ability to increase the visual contrast and magnification and to change colors can be useful. For example, it can be easier to understand the information on the screen if the color is appropriate for what the image represents (e.g., blue for water).

On-screen keyboards may also be used with single- or multiple-switch scanning formats. When the on-screen keyboard is used for typing it may occupy up to half of the screen, with the rest of the screen being used to display the program being used (e.g., word processing or email).

For individuals who have limited motor capabilities, one or more of the scanning techniques and formats described earlier in Chapter 5 may be used. Switches are connected to the computer using USB switch connectors described later in this chapter. Once a scanning selection is made, the device responds if it had been directly selected.

Built-in Software Adaptations to the Standard Keyboard

Persons with disabilities often have difficulty pressing more than one key at a time because they are single-finger typists. They may also have accidental key activation due to poor fine motor control. Software adaptations for these and other problems are shown in Table 7-1. These software adaptations are built into Windowsⁱ and Apple Macintosh™* operating systems. The software adaptations can be adjusted for an individual user through the control panel. Accessibility for the Macintosh includes those features shown in Table 7-1. When StickyKeys is used, the modifier keys are converted to sequential rather than simultaneous use. This means that instead of having to press the shift key and another key at the same time, the user can press the shift key and then press another key, and the second key will be shifted. StickyKeys allows either a single finger or a head pointer or a toe to be used to access all the functions on standard keyboards. FilterKeys and other designations such as BounceKeys, SlowKeys, and RepeatKeys are options designed to avoid double entries due to holding a key too long or accidentally hitting a key multiple times due to lack of fine motor control. Both Windows and Macintosh operating systems all provide many options to make the keyboard and mouse faster and easier to use. In both cases the operating system leads the user through choices based on a description of the need.

Table 7-1

Minimal Adaptations to the Standard Keyboard and Mouse *

Need Addressed	Software Approach
Modifier key cannot be used at same time as another key	StickyKeys^†
User cannot release key before it starts to repeat	FilterKeys^†
User accidentally hits wrong keys	SlowKeys,^† BounceKeys,^† FilterKeys^†
User cannot manipulate mouse	MouseKeys^†
User wants to use augmentative communication device as input	SerialKeys^† in Windows XP or an alternative (like AAC Keys)
Keyboard is difficult	Touch Screen (Windows) Ulti-Touch (MacBook)
User cannot access keyboard (Windows Vista)	On-screen keyboard (Windows XP and Macintosh) Built-in ASR (Windows and Macintosh Vista)

StickyKeys: user can press modifier key, then press second key without holding both down simultaneously.

SlowKeys: a delay can be added before the character selected by hitting a key is entered into the computer; this means that the user can release an incorrect key before it is entered.

BounceKeys: prevents double characters from being entered if the user bounces on the key when pressing and releasing.

FilterKeys: the combination of SlowKeys, BounceKeys, and RepeatKeys in Microsoft Windows.

MouseKeys: substitutes arrow keys for mouse movements.

SerialKeys: allowing any serial input to replace mouse and keyboard, this function has largely been replaced by USB standard devices.

Touchscreen: with a touch-screen monitor, uses finger on screen to move icons, point, resize windows, play media, and pan and zoom.

Multi-touch: can use gestures on the track pad to control input, pinch, swipe, or rotate gestures (similar to iPOD touch and iPhone).

^*Universal Access in Macintosh operating system, Apple Computer, Cupertino, Calif.; Accessibility Options in Windows XP, Microsoft Corp., Seattle, Wash.

^†Software modifications developed at the Trace Center, University of Wisconsin, Madison. These are included as before-market modifications to the Macintosh operating system or Windows in some personal computers and are available as after-market versions in others. The function of each program is as follows:

Both Windows and Macintosh have built-in on-screen keyboards. Two modes of entry are available when an on-screen key is highlighted by mouse cursor movement: clicking and dwelling. In the latter the user keeps the mouse pointer on an on-screen key for an adjustable, preset time and the key is entered. The on-screen feature also allows entry by scanning. An area of the screen can be designed as a “hot key” with text, graphics, or control functions (such as opening a file or running a program) stored at that location. When the hot spot is selected by pressing, scanning, tapping, or other means, the stored text or function is entered into the computer. An auditory click or flashing icon or other feedback may be provided to indicate selection of a hot spot. Limited automatic speech recognition capability is also included in Windows, Macintosh, and several smart phone operating systems.^†

Automatic Speech Recognition as an Alternative Keyboard

Automatic speech recognition (ASR) technology can be used for computer access by allowing the user to speak the names of keys or key words and have these spoken utterances interpreted by the computer as if they had been typed. This approach is appealing because human speech is so rapid and voice control is so natural. ASR systems that are extremely reliable, flexible, and easy to use are available for use as full-function keyboard and mouse emulation. For example, if a word processing program is being run, then control functions such as delete, move, and print, as well as the most common vocabulary items the person normally uses (e.g., a greeting and ending for a business letter, and other similar vocabulary items) can be used. If the user changes to a spreadsheet program, he can use vocabulary that contains items specific to that application.

Case Study – Evaluation and Selection of Speech Recognition

Marilyn Abraham is a 44-year-old woman who has been diagnosed as having reflex sympathetic dystrophy (RSD) of both wrists. Apparently caused by vasospasm and vasodilation, RSD is a reaction to pain after an injury ¹⁶ (Kasch, Poole, and Hedl, 1998). It results in edema; shiny, blotchy skin; and pain. Ms. Abraham is a secretary in a large state office, which she shares with other co-workers. She uses the computer for much of the day. The RSD ensued in her right wrist as a result of the repetitive motion she uses in performing her job. After this injury she received retraining to transfer her hand dominance to her left hand, and the Dvorak one-handed keyboard layout was recommended (see Figure 6–8). Subsequently she broke her left wrist in a motor vehicle accident, which also resulted in RSD. She is able to type or use the mouse for only 10 minutes before her hands and forearms swell. Ms. Abraham has tried different positions and adaptations when typing. For example, she used a pointer held by a cuff in her palm to type so that her forearm remained in a neutral position. This still resulted in swelling and pain. She also experiences neck pain when using the keyboard.

Ms. Abraham first tried using a trackball with her hand and the on-screen keyboard. After using the trackball for a short time, Ms. Abraham found that it also caused pain. Ms. Abraham next tried using her right foot with an expanded keyboard and then a trackball. There were concerns about the utility of both these approaches because of potential neck strain from looking down and the possibility that the repeated movement of her ankle to input characters using the trackball might lead to repetitive motion problems with her foot.

Next Ms. Abraham tried a head-controlled interface that was worn on a band and attached to her head. She used this interface with an on-screen keyboard and acceptance time to make a selection. She was able to control this interface without difficulty but thought that after a period of use her neck would become tired.

Questions

1. What other switches could you try with Ms. Abraham?

2. If you evaluate automatic speech recognition for Ms. Abraham, what issues will you need to take into consideration?

Two basic types of ASR systems exist. With a speaker-dependent system, the user trains the system to recognize his voice by producing several samples of the same utterance. The method in which the training is handled varies among systems. The system analyzes these samples so that it can recognize variations in the user’s speech and generate a computer input (e.g., enter a given letter, a string of letters, or a control key such as “return”) corresponding to what was spoken. Even after the system has been trained using several speech samples, there likely will be times when the system does not recognize the user’s speech and does not produce a response. Recognition accuracy is steadily increasing as advances are made in the computer algorithms used for analysis. Rates can be greater than 90% for general input and nearly 100% for isolated word applications (e.g., command and control, database, or spreadsheet). Speaker-dependent systems can be further divided into continuous and discrete categories.

Speaker-independent systems recognize speech patterns of different individuals without needing training.¹⁴ These systems are developed using samples of speech from hundreds of people and information provided by phonologists on the various pronunciations of words.⁶ The tradeoff with this type of total-recognition system is that the vocabulary set is small. In assistive technology applications, speaker-independent systems are primarily used for environmental control (Chapter 14) and power mobility (Chapter 12).

Speech recognition can be used for computer access, wheelchair control, and EADLs. The systems shown in Table 7-2 allow the consumer to use his speech to enter text directly into a computer application program. Recognition of control words used in word processors, such as “save file,” is also trained. System vocabulary is also growing rapidly. Early systems had recognition vocabularies (the list of words the system can recognize when spoken) in the 1000 to 5000 range. Current systems have vocabularies of 50,000 words or more. The faster speech rate, larger vocabularies, and continuous recognition all place significant demands on the speed and memory of the host computer. Continuous speech recognition systems require large amounts of memory and high-speed computers. As the cost of this added computer functionality continues to decline, these additional requirements will be less important. However, ASR systems do require more computer resources than other alternative input methods.³

Table 7-2

Types of Automatic Speech Recognition

Category	Description	AT Application
Speaker-dependent systems	Recognition depends on the system’s learning the user’s speech patterns and building a user vocabulary.	Used for general mainstream technology access. Computers, smart phones, pad computers
Speaker-independent systems	The operation is similar to continuous speech recognition systems, but there is no training required. Generally limited to small, application-specific vocabularies.	Used in special-purpose assistive devices for environmental control or robotic control (see Chapter 14) and wheelchair control (see Chapter 12).

There are other acoustic or audio issues that are important in ASR as well. Foremost of these is the microphone. Anson (1997)⁴ discusses considerations in the choice of a microphone for ASR. Although the microphones supplied with ASR systems are satisfactory for use by nondisabled users, they are not adequate when the user has limited breath support, special positioning requirements, or low-volume speech. Most ASR systems use a standard headset microphone. Individuals who have disabilities may not be able to don and doff such microphones independently, and therefore desk-mounted types are often used. Current ASR systems do not require separate hardware to be installed in the computer, and they utilize commonly available sound cards.³

Electronic Aids to Daily Living (EADLs) may also utilize speech recognition to access their functions (see Chapter 14). In such devices the individual can instruct the system to turn lights on and off or perform other functions using her voice. She can train the system to execute these commands with just about any sound, letter, or word.

The questions listed in Box 7-1 can be used to determine the usefulness of speech recognition for a given consumer. The key for success in using speech-activated systems is that the user be able to produce a consistent vocalization or verbalization. Differences in speech production are found not only among individual speakers, but also within the same speaker. Variability in the user’s speech can cause problems with recognition. For this reason, this type of control interface may not be effective for individuals who have dysarthria. Individuals who have had a spinal cord injury and have no functional use of the upper extremities yet have good speech control are potential candidates for a speech recognition system. It is important when considering a speech recognition system to determine whether the user’s voice pitch, articulation, and loudness change or fatigue over time. Other noises or voices in the area where the speech-activated system is being used can also confuse the system, resulting in either an incorrect selection or the system having difficulty registering any selection and causing the user to repeat the vocalization several times.

Box 7-1 Critical Questions for Evaluating Use of Speech Recognition Interface

1. Can the client consistently utter all the sounds necessary to access the speech recognition system?

2. Is the recognition vocabulary adequate?

3. Is the client’s voice articulation, pitch, and loudness consistent enough for accurate selection?

4. Is there likely to be background noise in the client’s context that will interfere with the speech recognition system?

5. Would an alternative template or vocabulary be beneficial?

Mouse

The standard computer mouse is intended to be gripped by the user’s hand and moved across a flat surface. Successful use of the mouse requires sufficient vision to see the pointer and the icons on the screen and adequate eye-hand coordination to allow the user to cause the pointer to go to the desired icon or text by moving the mouse. Once the pointer is located at the desired element the user must click, double click, or click and hold to drag the element. These can be difficult for someone who has fine motor control limitations. There are several adjustments built into computers that can help match a user’s needs to the mouse functions by changing the speed of mouse movement, changing the time allowed for clicking, etc.

The mouse is ideally suited for functions such as drawing, moving around in a document, or moving a block of text. The mouse can be a useful tool for individuals with disabilities who cannot otherwise draw using a pen or pencil. However, mouse use requires a high degree of eye-hand coordination and motor coordination and a certain amount of range of motion. Standard computer mice are available in many different shapes and sizes. If a consumer is having difficulty using the mouse that came with the computer, the solution may be as simple as finding a mouse that fits his hand better.

The standard mouse requires a great deal of motor control and many individuals with disabilities find that the use of a standard computer mouse is difficult or impossible. Another option is to try a different control site for mouse use. If the consumer has better control of his feet than his hands, his foot can be used with a foot-controlled mouse such as the No Hands Mouse.* There are also alternatives to mouse use that are easier for many persons with disabilities. Any control interface that can imitate the two-dimensional movement (up/down, left/right) of the mouse can be made to appear like a mouse to the computer. Table 7-3 lists the major alternatives to mouse input, as well as sample technologies. Examples of several of these approaches are shown in Figure 7-2.

Table 7-3

Alternative Electronic Pointing Interfaces

Category	Description	Device Name/Manufacturer
Keypad mouse	Mouse movement is replaced by keys that move the mouse cursor in horizontal, vertical, and diagonal directions. One or more keys perform the functions of the mouse button (click, double-click, drag).	Included with Windows and Mac OS.
Trackball	Looks like an inverted mouse; a ball is mounted on a stationary base. Included on the base are one or more buttons that provide the functions of the standard mouse buttons. The base and hand remain stationary and the fingers move the ball. Requires minimal range of motion and less eye-hand coordination.	Big Track, n-Abler (Inclusive Technology); EasiTrax (Inclusive Technology); Trackman Marble Plus (Logitech); EasyBall (Microsoft); Roller Trackball (Traxsys Computer Products).
Continuous input joysticks	Joysticks (continuous input and switched) are used as direct selection interfaces for powered mobility. For computer use, movements are similar to wheelchair control; easy to relate cursor movement (direction, speed, and distance) to joystick movement.	Jouse (Compusult Limited); Roller Joystick II (Traxsys Computer Products); EasiTrax (Inclusive Technology); all manufacturers of powered wheelchairs have their own joystick, which is supplied with wheelchair.
Head-controlled mouse	An interface controlled through head movement; the user wears a sensor on the head, which is detected by a unit on the computer. Movement of the head is translated into cursor movement on the screen.	Origin Instruments; Able Net http://www .ablenetinc.com/, also available in some speech generating devices.

Data from Adaptivation, Sioux Falls, SD (www.adaptivation.com); Compusult Limited, P.O. Box 1000, Mount Pearl, Newfoundland (http://www .jouse.com/jouse2/home); Logitech, Fremont, CA (www.logitech.com); Inclusive Technology (http://www.inclusive.co.uk/hardware/mouse-alternatives); Microsoft, Redmond, WA (www.microsoft.com); Origin Instruments, Grand Prairie, TX (www.orin.com); Traxsys Computer Products (http://www.traxsys.com/AssistiveTechnology/tabid/1087/Default.aspx); Prentke Romich Co., Wooster, OH (www.prentrom.com).

Figure 7-2 Pointing interfaces. A, Standard computer mouse. B, Trackball. C, Proportional joystick.

Touch Screen Mouse Emulation

Many mainstream devices such as tablet computers, pad computers, and smart phones have touch screens that allow mouse-like control. The screen elements (text or icons) can be moved by touching and dragging them with finger movements that scroll through choices, open and close files, and expand and contract the number of displayed elements. Selection, the equivalent of clicking or double clicking a mouse button, is done by tapping the text entry or icon. Scrolling to new screens or new items within a file is done by swiping the finger across the screen from side to side or up and down.

Keypad Mouse

For those individuals who are able to use a standard keyboard but have difficulty using a standard mouse, the first alternative to evaluate is the keypad mouse. A numeric keypad is embedded in most standard computer keyboards. MouseKeys, included in the Accessibility Options (Table 7-1) for Windows and in the Macintosh operating systems allows use of the keypad to simulate mouse movement. When the NUM LOCK key is engaged, each key on the numeric keypad functions as the number to which it is assigned (1 to 9). When the NUM LOCK key is disengaged and MouseKeys is running, these keys can perform the same functions as a mouse. There is an option in the control panel that will reverse these functions of the NUM LOCK key (i.e., MouseKeys will be active when the NUM LOCK key is engaged). The “5” key serves as a mouse click, and the surrounding number keys move the mouse in vertical, horizontal, or diagonal directions. This software interprets the keys as mouse input when MouseKeys is active and interprets them as arrow keys when it is not active. MouseKeys allows adjustment of the mouse speed (distance the cursor moves with each arrow key press) and acceleration (the rate at which the cursor moves).