Literature Reviews

Reviewed research papers, articles, and publications relevant to digital accessibility.

• Usability Evaluation of Captions for People Who Are Deaf or Hard of Hearing

  Sushant Kafle, Matt Huenerfauth (2018) — SIGACCESS Accessibility and Computing Newsletter (Issue 122)

  This is a SIGACCESS Newsletter article summarizing a line of research by Kafle and Huenerfauth on building a caption-quality evaluation metric that actually reflects the experience of Deaf and Hard-of-Hearing (DHH) readers — rather than simply counti…

  • automatic speech recognition
  • captioning
  • captions
  • caption quality
  • accessibility metrics
  • deaf and hard of hearing
  • real-time captioning
  • accessibility research
  • natural language processing

• Watch It, Don't Imagine It: Creating a Better Caption-Occlusion Metric by Collecting More Ecologically Valid Judgments from DHH Viewers

  Akhter Al Amin, Saad Hassan, Sooyeon Lee, Matt Huenerfauth (2022) — Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22)

  This CHI 2022 paper builds a better automated metric for the severity of caption-occlusion — the problem of closed captions blocking important on-screen visual content during television programming. DHH viewers consistently report that even perfectly…

  • captioning
  • captions
  • caption occlusion
  • deaf and hard of hearing
  • television accessibility
  • video accessibility
  • accessibility metrics
  • accessibility research
  • live captioning

• Proposing New Metrics to Evaluate Web Usability for the Blind

  Kentarou Fukuda, Shin Saito, Hironobu Takagi, Chieko Asakawa (2005) — CHI '05 Extended Abstracts on Human Factors in Computing Systems

  This four-page CHI '05 extended abstract argues that existing automated accessibility checkers — in 2005 typified by Bobby — had plateaued: they verified the presence of ALT attributes, labels, and markup conformance, yet real blind users still found…

  • web accessibility
  • accessibility evaluation
  • accessibility metrics
  • screen readers
  • voice browser
  • blindness and low vision
  • automated accessibility
  • alt text
  • accessibility testing

• SAMBA: a semi-automatic method for measuring barriers of accessibility

  Giorgio Brajnik, Raffaella Lomuscio (2007) — Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '07)

  Brajnik and Lomuscio argue that web accessibility cannot be managed without being measured, yet existing accessibility metrics are underdeveloped and almost universally tied to conformance with WCAG checkpoints rather than to the real-world barriers …

  • web accessibility
  • accessibility metrics
  • accessibility evaluation
  • accessibility testing
  • conformance testing
  • barrier walkthrough
  • expert review
  • automated testing
  • quality assurance

• Effects of sampling methods on web accessibility evaluations

  Giorgio Brajnik, Andrea Mulas, Claudia Pitton (2007) — Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '07)

  Brajnik, Mulas, and Pitton's Assets '07 paper is a large, careful empirical investigation of the sampling step in web-accessibility evaluation — the step at which an evaluator picks which pages of a site to actually test. For any site above a trivial…

  • web accessibility
  • accessibility evaluation
  • sampling methods
  • accessibility metrics
  • conformance testing
  • automated testing
  • WCAG
  • quality assurance
  • research methodology
  • accessibility audit

• Metrics for accessibility on the Vamolà project

  Silvia Mirri, Ludovico Antonio Muratori, Marco Roccetti, Paola Salomoni (2009) — Proceedings of the 2009 International Cross-Disciplinary Conference on Web Accessibililty (W4A)

  This paper describes the Vamolà project (Italian acronym for "accessibility validator and monitor"), a collaboration between the University of Bologna and the Emilia Romagna Region that integrates an accessibility validator with a monitoring system d…

  • accessibility metrics
  • accessibility monitoring
  • automated testing
  • Stanca Act
  • accessibility evaluation
  • Italian accessibility law
  • WAB score

• Analysis and integration of web accessibility metrics

  Maia Naftali (2010) — Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A)

  This Google Student Award paper presents OceanAcc, a semi-automatic accessibility evaluation and analysis tool developed as a thesis project at the University of Buenos Aires. OceanAcc addresses a practical gap: while several accessibility metrics ha…

  • accessibility metrics
  • accessibility evaluation
  • automated testing
  • barrier walkthrough
  • WCAG compliance
  • accessibility tools

• Web not for all: a large scale study of web accessibility

  Rui Lopes, Daniel Gomes, Luís Carriço (2010) — Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A)

  This paper presents the first large-scale automated accessibility evaluation of nearly 30 million web pages from the Portuguese Web Archive. The researchers implemented 39 WCAG 1.0 checkpoints (priorities 1 and 2) based on the Unified Web Evaluation …

  • automated testing
  • accessibility evaluation
  • WCAG compliance
  • web science
  • accessibility metrics
  • large-scale study

• Getting One Voice: Tuning Up Experts' Assessment in Measuring Accessibility

  Silvia Mirri, Paola Salomoni, Ludovico A. Muratori, Matteo Battistelli (2012) — Proceedings of the International Cross-Disciplinary Conference on Web Accessibility (W4A)

  This paper addresses a fundamental challenge in web accessibility evaluation: how to reconcile the subjective assessments of multiple human experts into a single, reliable accessibility measurement. While automated testing tools produce binary pass/f…

  • accessibility metrics
  • accessibility evaluation
  • manual evaluation
  • automated testing
  • expert assessment
  • WCAG compliance

• An Optimal Sampling Method for Web Accessibility Quantitative Metric

  Mengni Zhang, Can Wang, Jiajun Bu, Zhi Yu, Yi Lu, Ruijie Zhang, Chun Chen (2015) — Proceedings of the 12th International Web for All Conference (W4A 2015)

  This short paper proposes OPS-WAQM, an optimal page sampling method specifically designed for use with the Web Accessibility Quantitative Metric (WAQM) when evaluating large websites. The core problem is that evaluating every page of a large website …

  • accessibility evaluation
  • sampling methods
  • accessibility metrics
  • WAQM
  • large-scale evaluation
  • government websites
  • quantitative metrics
  • statistical methods