From sound to a useful cue
A sound classifier produces labels and scores. A wearer needs to know what matters right now. Our team built glasses that turn alarms, knocks, and someone calling your name into a small visual cue with direction information.
My work connected that recognition pipeline to the experience: onboarding, saved sound preferences, notification priorities, the companion website, and the display output.
Deciding what gets attention
I designed four alert priority levels, plus an Ignore setting. Urgent cues can interrupt ordinary ones; repeat detections need to merge instead of restarting the same notification over and over. Dismissal, expiry, and focus modes give the wearer control over how much reaches the display.
The browser preview and physical OLED share the same alert state. That made it possible to test the interaction with simulated events before connecting the glasses, then carry the same behavior onto the hardware.
Recognition and response time
I integrated local audio models for 527-class sound recognition, name spotting, and speech-to-text. The environmental-sound evaluation covered 1,672 test cases; that is an evaluation set, not a claim that every class is equally reliable.
For speech after a name call, I cut question-capture lag from up to six seconds to under one second. Keeping the response short on the glasses and the fuller transcript on the companion lets each display do a different job.
A working prototype
We connected the receiver to the physical OLED and tested live display cues. Recognition and direction still depend on the microphones, room acoustics, and sound source; the prototype is not a certified safety device.