SoundWeAR: Co-Designing AR Sound Cues to Support Outdoor Awareness for DHH Individuals

SoundWeAR investigates how augmented-reality glasses can translate environmental sound into visual cues that support the situational awareness of Deaf and Hard of Hearing (DHH) people in outdoor settings - a context prior work has largely neglected in favour of indoor scenarios like meetings, classrooms, and the home. The authors argue that outdoor soundscapes are dynamic, safety-critical, and require design strategies that convey urgent information (sirens, approaching vehicles) without overwhelming users with visual clutter. The study follows a three-phase participatory design process. First, four co-design workshops paired one DHH participant, one social worker, and one designer per session to explore sound awareness needs and sketch visual representations, using a simulated outdoor walking demo in XREAL Air 2 Ultra AR glasses as a common reference. Insights were synthesised into three design dimensions - Sounds of Interest, Sound Information, and Visualization Preferences - broken into nine directions (identity, urgency, location, distance, loudness, duration, screen layout, overlapping sounds, alert lifecycle). Second, 15 DHH participants selected preferred design options across 27 candidate visualisations via semi-structured interviews or text-based equivalents. Third, the winning combination was implemented as a Unity-based AR prototype and evaluated with eight DHH walkers along a 15-sound university-campus route, with Braun and Clarke thematic analysis applied throughout.

DHH participants prioritised identity, urgency, and location over loudness, duration, and pitch - a reversal of hearing-centric assumptions about what makes a sound important. Color-coded icons (red for safety, blue for informational, green for environmental) were chosen by 13 of 15 participants for identity, and 8 of 15 chose red border alerts for urgency because they drew attention without occluding the field of view. Hybrid location cues (arrows for in-view sounds, a mini-map for rear/distant sounds) were preferred; distance was best conveyed by plain text rather than size scaling. Cognitive load was a strong concern: 10 of 15 participants supported capping simultaneous notifications at three, and short auto-dismiss (2-3 seconds) was preferred for informational cues but felt too fast for safety-critical alerts, which participants wanted to persist until acknowledged. In the outdoor walking evaluation, all eight participants reported increased awareness and a greater sense of safety and confidence without feeling distracted from the real environment. Participants also pushed back on pure third-person views, finding first-person visualisation more reliable for safety-relevant information. Hardware weight and interference with cochlear implants and hearing aids emerged as a real-world barrier.

This paper is directly useful for anyone designing wearable or AR-based accessibility systems for DHH users, especially in outdoor, safety-critical contexts. The nine design dimensions and the concrete preference data behind each (colour-coded icons, red border urgency alerts, text-based distance, capped simultaneous notifications, urgency-scaled alert lifecycle) are actionable input for smart-glasses, smartwatch, and phone-based sound awareness products. The finding that identity and urgency outweigh loudness should shape how audio-to-visual translation pipelines prioritise what to surface. Limitations to keep in mind: all participants had profound or near-profound hearing loss, the evaluation used scripted sound events on a university campus rather than in-the-wild recognition, the sample was small (n=8 in evaluation), and the hardware was visibly bulky - findings may shift with lighter glasses and real-time environmental sound classification.