Literature Reviews
Reviewed research papers, articles, and publications relevant to digital accessibility.
138 results found tagged machine learning.
-
Never-ending Learning of User Interfaces
This paper introduces the Never-ending UI Learner, an automated system that continuously crawls real mobile applications to learn semantic properties of user interfaces. The system addresses a fundamental limitation of current approaches to UI unders…
-
Technical Perspective: Computation Where the (Inter)Action Is
This one-page technical perspective accompanies the SoundWatch paper in Communications of the ACM. Bigham uses SoundWatch — a smartwatch prototype that detects audio events and displays descriptions for deaf and hard-of-hearing people — as a lens to …
-
UIClip: A Data-driven Model for Assessing User Interface Design
This paper introduces UIClip, a computational model that automatically assesses UI design quality and visual relevance from a screenshot and natural language description. Built on OpenAI's CLIP B/32 architecture (151 million parameters), UIClip is fi…
-
Automatically Identifying Trouble-Indicating Speech Behaviors in Alzheimer's Disease
This paper addresses the challenge of automatically detecting communication breakdowns in conversations with people who have Alzheimer's disease (AD). AD is a progressive neurodegenerative disease that deteriorates memory, executive capacity, visual-…
-
Identifying Sign Language Videos in Video Sharing Sites
This paper addresses the challenge of finding sign language videos within general video sharing platforms like YouTube. While these platforms contain growing libraries of sign language content created by deaf community members, locating this content …
-
Distinguishing Users By Pointing Performance in Laboratory and Real-World Tasks
This paper investigates how machine learning can automatically assess pointing difficulties from everyday computer use, addressing a critical barrier to accessible computing: the lack of frequent, low-cost assessment of pointing ability. The research…
-
Automatic Detection of Phone-Based Anomalies in Dysarthric Speech
This research develops automatic methods to detect and localize acoustic anomalies in speech produced by people with dysarthria, a motor speech disorder caused by neurological damage affecting the respiratory, phonatory, resonatory, articulatory, or …
-
Isolated Sign Language Recognition with Grassmann Covariance Matrices
This paper proposes a novel method for isolated sign language recognition using Grassmann Covariance Matrices (GCM) to fuse multimodal features captured by Microsoft Kinect. With 360 million people worldwide affected by hearing loss—21 million in Chi…
-
Sign Transition Modeling and a Scalable Solution to Continuous Sign Language Recognition for Real-World Applications
This paper presents a scalable framework for continuous sign language recognition (SLR) designed to work in real-world conditions using affordable hardware. The researchers address a fundamental challenge in SLR: modeling the transitions between sign…
-
"Hands On" Visual Recognition for Visually Impaired Users
This paper presents a collaborative visual recognition system designed to help blind or visually impaired (BVI) users identify specific product instances — distinguishing between brands, models, or types of objects that feel similar when handled. Whi…