This article was originally posted on Syllable.Productions earlier this year. Update: I participated in a workshop at Google to further explore this idea. I’ll post an update soon.

Sonorous is a proposed navigation tool for the visually impaired. It enables someone without sight to visualize their surroundings through sound.

The project was inspired by folks like Daniel Kish who are skilled at tongue-click echolocation. Watching these experts navigate their environment through sound have given me hope that technology-assisted echolocation that could offer another tool to improve accessibility for visually impaired folks.

HOW IT WORKS

Sonorous works by combining three super-cool technologies.

  1. A mobile device called Tango, made by Google, maps the space around you locating objects and surfaces. It’s kind of like the technology used by self-driving cars.
  2. Next, augmented reality software interprets the map to identify real objects in the room — like doors and obstacles. Then it creates virtual sound beacons to represent the objects.
  3. Lastly, a 3D audio technology called binaural audio ties it all together. This audio processor is what creates the convincing illusion that the virtual sound beacons are attached to objects around you. It’s seriously cool. Try the 3D audio demo app below!

DEMO OF THE TANGO INTEGRATION: VIDEOS

When you walk around in the room (in real life), the Tango device synchronizes your real location with your location in a virtual model. In the demo, red and green dots represent virtual audio sources. The augmented reality app plays 3D audio in your headphones which makes the virtual dots sound like they are sitting right on the furniture in real life.

Please keep in mind, this is just a proof of concept. It doesn’t reflect the future product design.

A Demo showing how Project Sonorous uses AR to create virtual sound sources mapped into locations of a real room.
A demo explaining how Project Sonorous works with the Google Tango device.

DEMO OF 3D AUDIO WAYFINDING: A MAC APP

  1. Download and unzip the 3D Audio Demo. (no Tango device needed) (edit 2022: hmm, it looks like the 3D audio isn't working anymore. I'll try to make another  app build.)
  2. Run the application:
  • Unfortunately, you can’t simply open it, because I am an “unidentified developer”.
  • Instead, control-click (right click) on the application file.
  • Then select the “Open” option at the top of the menu.
  • It will still display a warning dialog, but now the dialog has an “Open” button. Click it.

Now play it! (Please excuse the annoying steps.)