Literature Reviews

Reviewed research papers, articles, and publications relevant to digital accessibility.

• Warping Time for More Effective Real-Time Crowdsourcing

  Walter S. Lasecki, Christopher D. Miller, Jeffrey P. Bigham (2013) — Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013)

  This paper introduces TimeWarp, a technique that manipulates audio playback speed to improve crowd workers performance on real-time speech captioning. The core problem is that non-expert typists cannot keep up with natural speaking rates of 150-225 w…

  • real-time captioning
  • crowdsourcing
  • deaf and hard of hearing
  • human computation
  • speech accessibility

• Legion Scribe: Real-Time Captioning by Non-Experts

  Walter S. Lasecki, Raja Kushalnagar, Jeffrey P. Bigham (2014) — ACM SIGACCESS Conference on Computers and Accessibility

  This demonstration paper presents Legion:Scribe, a crowd-powered captioning system that enables groups of 3-5 non-expert typists to collectively produce real-time captions with less than 5 seconds of latency. The system addresses the prohibitive cost…

  • real-time captioning
  • crowdsourcing
  • deaf and hard of hearing
  • speech-to-text
  • human computation
  • accessibility

• Real-Time Captioning with the Crowd

  Walter S. Lasecki, Jeffrey P. Bigham (2014) — Interactions

  This article presents Scribe, a crowdsourced real-time captioning system that allows groups of non-expert typists to collectively produce captions at the speed of natural speech — a task that normally requires highly trained professional stenographer…

  • real-time captioning
  • crowdsourcing
  • deaf and hard of hearing
  • speech-to-text
  • human computation
  • accessibility

• Leveraging Complementary Contributions of Different Workers for Efficient Crowdsourcing of Video Captions

  Yun Huang, Yifeng Huang, Na Xue, Jeffrey P. Bigham (2017) — CHI Conference on Human Factors in Computing Systems

  This paper presents BandCaption, a crowdsourcing system that combines automatic speech recognition (ASR) with input from diverse crowd workers to efficiently correct video captions. The key insight is that different groups of people — hearing-impaire…

  • captioning
  • crowdsourcing
  • video accessibility
  • speech recognition
  • deaf and hard of hearing
  • second language learners
  • automatic speech recognition

• Automated Class Discovery and One-Shot Interactions for Acoustic Activity Recognition

  Jason Wu, Chris Harrison, Jeffrey P. Bigham, Gierad Laput (2020) — Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20)

  This paper presents Listen Learner, an end-to-end system for acoustic activity recognition that automatically discovers and learns to classify environmental sounds with minimal user effort. Traditional approaches to sound recognition face a tradeoff:…

  • acoustic activity recognition
  • smart home
  • machine learning
  • Internet of Things
  • context awareness
  • self-supervised learning
  • sound recognition
  • deaf and hard of hearing

• Technical Perspective: Computation Where the (Inter)Action Is

  Jeffrey P. Bigham (2022) — Communications of the ACM

  This one-page technical perspective accompanies the SoundWatch paper in Communications of the ACM. Bigham uses SoundWatch — a smartwatch prototype that detects audio events and displays descriptions for deaf and hard-of-hearing people — as a lens to …

  • wearable technology
  • deaf and hard of hearing
  • sound recognition
  • smartwatch
  • assistive technology
  • machine learning
  • edge computing
  • accessibility

• Real-Time Caption Challenge: C-Print

  Michael S. Stinson, Pamela Francis, Lisa B. Elliot, Donna Easton (2014) — Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS '14)

  This demonstration paper presents C-Print, a typing-based real-time captioning system developed over 25 years by researchers at the National Technical Institute for the Deaf (NTID) at Rochester Institute of Technology. C-Print provides communication …

  • deaf and hard of hearing
  • real-time captioning
  • communication access
  • transcription
  • mobile accessibility
  • education

• Enhancing Caption Accessibility through Simultaneous Multimodal Information: Visual-Tactile Captions

  Raja S. Kushalnagar, Gary W. Behm, Joseph S. Stanislow, Vasu Gupta (2014) — Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS)

  This paper addresses a fundamental limitation of captions (subtitles) for deaf and hard of hearing (DHH) viewers: captions force viewers to split attention between reading text at the bottom of the screen and watching the visual action, inevitably ca…

  • captioning
  • deaf and hard of hearing
  • haptic feedback
  • multimodal interaction
  • non-speech information
  • sensory substitution
  • multimedia accessibility

• Identifying Sign Language Videos in Video Sharing Sites

  Frank M. Shipman, Ricardo Gutierrez-Osuna, Caio D. D. Monteiro (2014) — ACM Transactions on Accessible Computing

  This paper addresses the challenge of finding sign language videos within general video sharing platforms like YouTube. While these platforms contain growing libraries of sign language content created by deaf community members, locating this content …

  • sign language
  • ASL
  • video classification
  • machine learning
  • computer vision
  • deaf and hard of hearing
  • video sharing
  • information retrieval
  • SVM

• Accessibility Evaluation of Classroom Captions

  Raja S. Kushalnagar, Walter S. Lasecki, Jeffrey P. Bigham (2014) — ACM Transactions on Accessible Computing

  This paper presents a comprehensive evaluation of real-time captioning approaches for classroom lectures, comparing Communication Access Realtime Translation (CART), Automatic Speech Recognition (ASR), and a novel collaborative captioning system call…

  • real-time captioning
  • deaf and hard of hearing
  • classroom accessibility
  • crowdsourcing
  • eye tracking
  • CART
  • ASR
  • caption readability