Literature Reviews

Reviewed research papers, articles, and publications relevant to digital accessibility.

• Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots

  Jason Wu, Xiaoyi Zhang, Jeff Nichols, Jeffrey P. Bigham (2021) — The 34th Annual ACM Symposium on User Interface Software and Technology (UIST)

  This paper introduces screen parsing, the task of predicting UI elements and their hierarchical relationships from a screenshot alone. While prior work could detect individual UI elements on a screen (element detection), those approaches produced fla…

  • screen readers
  • mobile accessibility
  • computer vision
  • UI semantics
  • machine learning
  • accessibility metadata
  • reverse engineering

• Never-ending Learning of User Interfaces

  Jason Wu, Rebecca Krosnick, Eldon Schoop, Amanda Swearngin, Jeffrey P. Bigham, Jeffrey Nichols (2023) — Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST)

  This paper introduces the Never-ending UI Learner, an automated system that continuously crawls real mobile applications to learn semantic properties of user interfaces. The system addresses a fundamental limitation of current approaches to UI unders…

  • machine learning
  • mobile accessibility
  • UI semantics
  • automated crawling
  • tappability
  • screen readers
  • computer vision