Literature Reviews
Reviewed research papers, articles, and publications relevant to digital accessibility.
98 results found tagged multimodal interaction.
-
FAME: Exploring Expressive Facial Avatars for Lyrical and Non-Lyrical Music Visualization for d/Deaf Individuals
This CHI 2026 paper investigates whether expressive facial avatars can carry the emotional and structural richness of music — rhythm, pitch, melody, lyrics, and emotion — to d/Deaf and Hard of Hearing (DHH) audiences in ways that captions and abstrac…
-
Towards Inclusive External Human-Machine Interface: Exploring the Effects of Visual and Auditory eHMI for Deaf and Hard-of-Hearing People
This CHI 2026 paper is a companion to Xu et al. 2026 (10.1145/3772318.3791557) but with a broader, two-stage inclusive-design focus: selecting appropriate visual eHMI concepts for Deaf and Hard-of-Hearing (DHH) pedestrians and then evaluating how vis…
-
Exploring the Impacts of Background Noise on Auditory Stimuli of Audio-Visual eHMIs for Hearing, Deaf, and Hard-of-Hearing People
This CHI 2026 paper investigates a critical but overlooked accessibility question in the design of external Human-Machine Interfaces (eHMIs) for automated vehicles (AVs): how do Deaf and Hard-of-Hearing (DHH) pedestrians experience audio-visual eHMIs…
-
Augmenting Imagery with Multimodal Vibrotactile Representations: Touch, Feel, and Hear
This CHI 2026 paper addresses a long-standing gap in image accessibility for blind and visually impaired (BVI) users: while alt text, sonification, tactile graphics, and refreshable Braille displays convey shape, layout, and identity, they rarely com…
-
As Content and Layout Co-Evolve: TangibleSite for Scaffolding Blind People's Webpage Design through Multimodal Interaction
This CHI 2026 paper investigates how blind designers can author webpages end-to-end — generating content and arranging layout iteratively — rather than being limited to either content entry or layout inspection. The authors ran two rounds of co-desig…
-
Three Modalities, Two Design Probes, One Prototype, and No Vision: Experience-Based Co-Design of a Multi-modal 3D Data Visualization Tool
Kamath et al. report an Experience-Based Co-Design (EBCD) process in which five blind or low-vision (BLV) researchers and one sighted collaborator built a web-native, multimodal prototype for exploring 3D surface plots - a class of visualisation cent…
-
The Intelligent Voice-Interactive Interface
Schmandt and Hulteen describe the 'Put That There' system built at MIT's Architecture Machine Group (the precursor to the Media Lab), one of the earliest working implementations of a conversational, multimodal human–computer interface. Seated in a ch…
-
Tactile Emotions: Multimodal Affective Captioning with Haptics Improves Narrative Engagement for d/Deaf and Hard-of-Hearing Viewers
This paper explores a multimodal approach to affective captioning — captions that convey not just the words a speaker says but the emotional dimensions of their voice — for d/Deaf and Hard-of-Hearing (dhh) viewers. Prior work has shown that typograph…
-
MotionBuddy: Exploring Tactile-Based Motion Learning with a Tabletop Humanoid Robot for Blind People
This HRI 2026 study by the same team behind the earlier 3D tactile-pose work asks whether a tabletop humanoid robot can convey dynamic body movements to blind learners more effectively than audio instruction alone. The motivation is that physical 3D …
-
Touching Movement: 3D Tactile Poses for Supporting Blind People in Learning Body Movements
Tanaka and colleagues investigate whether 3D-printed tactile models of whole-body poses can help blind learners acquire physical movements that are normally taught through visual demonstration. Verbal descriptions are notoriously inconsistent and str…