System helps severely motor-impaired individuals type more quickly and accurately

System helps severely motor-dumb individuals blazon more than quickly and accurately

Researchers at MIT and elsewhere are developing a arrangement that enables severely motor-dumb individuals who communicate using a single switch to do so faster and with more accurateness. Their system is more flexible than many common interfaces, enabling it to be used for tasks similar cartoon, gaming, or surfing the web. Clock figures courtesy of the researchers, photo via iStockphoto

In 1995, French fashion mag editor Jean-Dominique Bauby suffered a seizure while driving a car, which left him with a status known as locked-in syndrome, a neurological illness in which the patient is completely paralyzed and can only move muscles that control the eyes.

Bauby, who had signed a volume contract presently earlier his accident, wrote the memoir “The Diving Bell and the Butterfly” using a dictation organization in which his speech therapist recited the alphabet and he would blink when she said the correct letter of the alphabet. They wrote the 130-page volume one blink at a fourth dimension.

Technology has come up a long fashion since Bauby’southward accident. Many individuals with severe motor impairments caused by locked-in syndrome, cerebral palsy, amyotrophic lateral sclerosis, or other conditions tin communicate using computer interfaces where they select messages or words in an onscreen grid by activating a single switch, often by pressing a button, releasing a puff of air, or blinking.

But these row-column scanning systems are very rigid, and, similar to the technique used by Bauby’s voice communication therapist, they highlight each option one at a fourth dimension, making them frustratingly boring for some users. And they are not suitable for tasks where options tin can’t exist arranged in a grid, like drawing, browsing the web, or gaming.

A more flexible organization existence developed by researchers at MIT places individual choice indicators next to each option on a reckoner screen. The indicators can be placed anywhere — next to anything someone might click with a mouse — so a user does non demand to wheel through a grid of choices to make selections. The system, called Nomon, incorporates probabilistic reasoning to learn how users make selections, and and so adjusts the interface to improve their speed and accuracy.

Participants in a user study were able to blazon faster using Nomon than with a row-column scanning system. The users also performed better on a pic selection task, demonstrating how Nomon could be used for more than typing.

“It is so cool and exciting to exist able to develop software that has the potential to really assistance people. Being able to detect those signals and turn them into communication as we are used to it is a really interesting problem,” says senior author Tamara Broderick, an associate professor in the MIT Department of Electrical Technology and Computer Science (EECS) and a member of the Laboratory for Information and Decision Systems and the Institute for Data, Systems, and Gild.

Joining Broderick on the paper are lead writer Nicholas Bonaker, an EECS graduate educatee; Emli-Mari Nel, head of innovation and auto learning at Averly and a visiting lecturer at the Academy of Witwatersrand in South Africa; and Keith Vertanen, an acquaintance professor at Michigan Tech. The inquiry is being presented at the ACM Conference on Human Factors in Computing Systems.

On the clock

In the Nomon interface, a minor analog clock is placed next to every selection the user tin select. (A gnomon is the part of a sundial that casts a shadow.) The user looks at ane option so clicks their switch when that clock’s hand passes a ruby-red “apex” line. After each click, the organization changes the phases of the clocks to split up the almost probable next targets. The user clicks repeatedly until their target is selected.

When used as a keyboard, Nomon’s auto-learning algorithms try to guess the next discussion based on previous words and each new alphabetic character as the user makes selections.

Broderick developed a simplified version of Nomon several years ago but decided to revisit it to make the system easier for motor-impaired individuals to use. She enlisted the aid of then-undergraduate Bonaker to redesign the interface.

They outset consulted nonprofit organizations that work with motor-impaired individuals, likewise as a motor-dumb switch user, to gather feedback on the Nomon blueprint.

Then they designed a user study that would better represent the abilities of motor-impaired individuals. They wanted to make sure to thoroughly vet the organization before using much of the valuable time of motor-impaired users, so they start tested on non-switch users, Broderick explains.

Switching upward the switch

To gather more representative data, Bonaker devised a webcam-based switch that was harder to use than simply clicking a key. The not-switch users had to lean their bodies to i side of the screen and and then back to the other side to annals a click.

“And they take to do this at precisely the right time, then information technology actually slows them downwardly. We did some empirical studies which showed that they were much closer to the response times of motor-dumb individuals,” Broderick says.

They ran a 10-session user study with xiii non-switch participants and one single-switch user with an avant-garde class of spinal muscular dystrophy. In the first nine sessions, participants used Nomon and a row-column scanning interface for 20 minutes each to perform text entry, and in the 10th session they used the two systems for a moving picture option task.

Non-switch users typed fifteen percent faster using Nomon, while the motor-impaired user typed fifty-fifty faster than the non-switch users. When typing unfamiliar words, the users were 20 percent faster overall and fabricated half as many errors. In their final session, they were able to complete the picture show pick job 36 percent faster using Nomon.

“Nomon is much more forgiving than row-column scanning. With row-column scanning, even if you lot are just slightly off, now you lot’ve called B instead of A and that’due south an error,” Broderick says.

Adapting to noisy clicks

With its probabilistic reasoning, Nomon incorporates everything it knows about where a user is probable to click to brand the process faster, easier, and less error-decumbent. For instance, if the user selects “Q,” Nomon will arrive equally easy equally possible for the user to select “U” next.

Nomon likewise learns howa user clicks. So, if the user e’er clicks a lilliputian after the clock’s hand strikes noon, the system adapts to that in real time. It besides adapts to noisiness. If a user’s click is often off the marking, the system requires extra clicks to ensure accuracy.

This probabilistic reasoning makes Nomon powerful but also requires a higher click-load than row-column scanning systems. Clicking multiple times can be a trying job for severely motor-impaired users.

Broderick hopes to reduce the click-load past incorporating gaze tracking into Nomon, which would give the system more than robust information about what a user might choose side by side based on which role of the screen they are looking at. The researchers also want to find a better manner to automatically adjust the clock speeds to help users be more than accurate and efficient.

They are working on a new series of studies in which they programme to partner with more motor-dumb users.

“So far, the feedback from motor-impaired users has been invaluable to u.s.; we’re very grateful to the motor-dumb user who commented on our initial interface and the dissever motor-dumb user who participated in our study. We’re currently extending our report to piece of work with a bigger and more diverse grouping of our target population. With their help, we’re already making further improvements to our interface and working to better understand the performance of Nomon,” she says.

“Nonspeaking individuals with motor disabilities are currently not provided with efficient communication solutions for interacting with either speaking partners or computer systems. This ‘communication gap’ is a known unresolved trouble in human-figurer interaction, and and so far there are no good solutions. This newspaper demonstrates that a highly creative arroyo underpinned past a statistical model can provide tangible performance gains to the users who need it the virtually: nonspeaking individuals reliant on a single switch to communicate,” says Per Ola Kristensson, professor of interactive systems applied science at Cambridge Academy, who was non involved with this enquiry. “The paper also demonstrates the value of complementing insights from computational experiments with the interest of terminate-users and other stakeholders in the blueprint process. I detect this a highly creative and important paper in an area where it is notoriously difficult to brand pregnant progress.”

This research was supported, in role, by the Seth Teller Memorial Fund to Advanced Technology for People with Disabilities, a Peter J. Eloranta Summer Undergraduate Enquiry Fellowship, the MIT Quest for Intelligence, and the National Science Foundation.

Media Inquiries

Journalists seeking information about EECS, or interviews with EECS faculty members, should electronic mail

Please note: The EECS Communications Office but handles media inquiries related to MIT’s Department of Electrical Engineering & Calculator Science. Please visit other school, department, laboratory, or middle websites to locate their dedicated media-relations teams.


Originally posted 2022-04-07 08:56:58.