Literature Reviews
Reviewed research papers, articles, and publications relevant to digital accessibility.
98 results found tagged multimodal interaction.
-
Developing Accessible TV Applications
This paper describes the GUIDE (Gentle User Interfaces for Elderly People) framework for developing accessible TV applications through multimodal interaction, user interface adaptation, and impairment simulation. The research combines a survey of 46 …
-
Multimodal Summarization of Complex Sentences
This paper introduces the concept of multimodal summarization (MMS) for complex sentences — automatically generating diagrams that combine pictures, simplified compressed text, and structural layout to help people understand difficult text. The syste…
-
Speech Dasher: A Demonstration of Text Input using Speech and Approximate Pointing
This paper demonstrates Speech Dasher, a multimodal text entry system that combines speech recognition with the Dasher zooming interface to enable fast, corrected text input using only voice and gaze direction. The core problem addressed is that whil…
-
Designing a Text Entry Multimodal Keypad for Blind Users of Touchscreen Mobile Phones
This paper presents MTITK (Multimodal Text Input Touchscreen Keypad), a software-only text entry system for blind users of touchscreen phones that combines the familiar 12-key telephone keypad layout with multimodal audio-tactile feedback. The work i…
-
Enhancing Caption Accessibility through Simultaneous Multimodal Information: Visual-Tactile Captions
This paper addresses a fundamental limitation of captions (subtitles) for deaf and hard of hearing (DHH) viewers: captions force viewers to split attention between reading text at the bottom of the screen and watching the visual action, inevitably ca…
-
Capti-Speak: A Speech-Enabled Accessible Web Interface
This paper presents Capti-Speak, a speech-augmented screen reader interface for the web that allows blind users to issue voice commands alongside traditional keyboard shortcuts. Built on top of the Capti web browsing application (which provides a JAW…
-
Multimodal Trajectory Playback for Teaching Shape Information and Trajectories to Visually Impaired Computer Users
This research investigates how haptic (force-feedback) and audio trajectory playback can teach shapes and gestures to blind and visually impaired computer users. The fundamental challenge addressed is that spatial information—shapes, diagrams, graphs…
-
Multimodal Presentation of Two-Dimensional Charts: An Investigation Using Open Office XML and Microsoft Excel
This paper presents a comprehensive system for making Microsoft Excel charts accessible to people who are blind or visually impaired through multimodal haptic and audio feedback. The system extracts chart data from the OOXML format introduced in Exce…
-
Usability of a Multimodal Video Game to Improve Navigation Skills for Blind Children
This paper presents an evaluative study of a multimodal system combining a custom haptic device (Digital Clock Carpet) with a 3D audio-based video game (MOVA3D) to develop orientation and mobility (O&M) skills in blind school-aged children. The Digit…
-
Write-N-Speak: Authoring Multimodal Digital-Paper Materials for Speech-Language Therapy
This paper presents Write-N-Speak, a digital pen toolkit that enables speech-language therapists to create custom multimodal therapy materials for clients with aphasia—a communication disorder often resulting from stroke that affects approximately 80…