Students improve mobile app accessibility for people with disabilities – USC Viterbi | Engineering School – USC Viterbi School of Engineering | Hot Mobile Press

Paul Chiou (left) and Ali Alotaibi.

Mobile applications have become as good as the salvation for the mountains of office work in our daily lives. We use apps for everything from scheduling doctor’s appointments to filing our tax returns, making everyday tasks faster and easier than ever. While these apps may be considered convenient for some, they certainly aren’t for everyone.

Although more and more people rely on their mobile devices for everyday tasks, developers rarely try to make their apps accessible for people with disabilities. Mobile apps often do not respect accessibility guidelines and this can make it difficult for people with mobility impairments and the elderly to use the app or even to carry out their daily activities.

Enter Ali Alotaibi and Paul Chiou, two Viterbi Ph.D. Students who also want to make the process of redesigning apps for accessibility convenient. They found that a large number of apps on the market had a salient problem: touch targets — the part of the screen you use to interact with the app, like buttons or checkboxes — were prohibitively small and potentially difficult for the elderly or people with disabilities to use them.

A touchy subject

“Despite the increasing number of people relying on mobile devices, surveys still show that many mobile apps are inaccessible to disabled users,” Alotaibi said. “They need to use mobile devices to access information and stay connected to the world, and this has become even more important during the pandemic.”

“Despite the increasing number of people relying on mobile devices, surveys still show that many mobile apps are inaccessible to disabled users.” Ali Alotaibi.

Although Google has developed a set of guidelines for developers to ensure touch targets in Android apps are large enough, according to the couple, those guidelines are rarely followed.

“It’s a prominent problem, but these developers just aren’t aware of it,” Chiou said. “It’s a lack of awareness.”

Chiou and Alotaibi saw an opportunity to use their expertise to pursue a long-standing interest in accessibility and developed a program called SALEM – or size-based inaccessibility repair in mobile apps – that automatically evaluates the app for size issues and reorganizes the layout into new, bigger goals. SALEM uses the Google Accessibility Scanner to identify these undersized targets and then generates a variety of new alternatives to the app’s current layout.

However, rearranging UI layouts can be difficult. The way layouts are coded in apps isn’t always intuitive, and introducing changes can create a “cascading effect” where other elements overlap or shift each other, creating a mess. This is where SALEM’s genetic algorithm comes into play. After SALEM creates several different alternatives, SALEM assigns each a fitness value based on how many of these errors occur or how skewed the result is. By generating multiple iterations with improved fitness scores, SALEM can “learn” how best to fix the problem.

SALEM uses the Google Accessibility Scanner to identify undersized targets.

“We developed a fitness function that can basically calculate the quality of any proposed repair,” Alotaibi said. “We consider all of these factors – the orientation, the positioning, the spacing, the size of the element, to calculate the quality of the repair.”

After several generations, the champion with the highest score is then determined as the replacement for the broken UI and can be applied.

“It’s really just a way of telling the computer what to do,” Chiou said. “We say, ‘We want you to reorganize these elements and align the touch targets with the guidelines.'”

Notable success

After running SALEM on 58 different UIs from different real-world apps, the new and improved layouts were showcased to people with motor disabilities and older adults, both groups disproportionately affected by height issues; Chiou and Alotaibi explained that the goal of this step was to test whether the fitness values ​​of the layouts reflect real-world viability. The results were overwhelmingly positive: SALEM corrected the tiny touch targets without affecting the usability of the user interface.

“This means we’ve fixed an issue that doesn’t negatively impact the user interface,” Alotaibi said. “We’ve also had some really encouraging feedback from people saying it’s bigger and easier to interact with. The study was really overwhelmingly positive in terms of user interface.”

“The research is the first to actually attempt to fix this problem.” Paul Chiu.

Accessibility has always been an issue for both Chiou and Alotaibi. Last year, Chiou worked on making web and mobile interfaces more collaborative with Assistive Technologies.

“Accessibility was generally an interest of Ali and I, and that’s why we went for it,” Chiou said. “Personally, I have a disability, so it’s closer to home. When I joined the program, accessibility technology was exactly what I wanted to do.”

No small thing

The project is the culmination of a fruitful friendship that began when she was studying the same Viterbi Computer Science Ph.D. Programming in 2018, the couple say, and SALEM is a passionate product born of bouncing ideas over the years. They also cited the effect that the encouragement and support from their Ph.D. Consultant, William GJ Halfond, had worked on the project.

“We would not have done this project or any other project without his guidance,” Alotaibi said.

“The research is the first to actually try to fix this problem,” Chiou said. “But there are more and more inaccessibility that are attracting mainstream attention.”

They hope the tool will provide consumers with an easy solution to size-related issues and help shed light on accessibility issues so developers might consider implementing more accommodations in the future. The pair have already made some wiggle room on presentation — they presented their findings at the 36th annual International Conference Automated Software Engineering last November, a privilege reserved for the best in software engineering.

“Our research is super exciting because we can apply interesting techniques related to software development while solving these problems that affect people with disabilities,” said Alotaibi. “Sometimes people don’t appreciate the difficulties some people have when using apps, and developers don’t understand the impact small changes can have.”

Released May 19, 2022

Last updated on May 20, 2022

Leave a Comment