Imagine if you couldn't hear with your ears but could through your skin. Well, Dr Scott Novich and Dr David Eagleman of NeoSensory set out to achieve just that and their Versatile Extra-Sensory Transducer (VEST) is bringing hearing to deaf people. We talked to Dr Novich about their work to change perception.
Novich describes the VEST as “a garment designed for taking any information stream such as sound, vision, stock market data or state information of an aircraft and mapping it to a wearer’s sense of touch, using vibration, in real time.”
The idea is that with time, training and the appropriately encoded information, the VEST will allow people to develop direct perceptual experiences of the information being passed through their skin.
So how does it work? With 32 vibrating motors placed around the torso, the VEST picks up information from the environment and vibrates in patterns that wearers learn to recognise as speech and sounds. Each motor represents a frequency and with enough practice, these associations become automatic; kind of like becoming fluent in a language or learning to ride a bike. It’s similar to Braille in that it isn’t the same as seeing words, but it achieves a pretty similar effect and, while the NeoSensory team are currently testing the VEST with deaf or hard-of-hearing users, potential applications range from gaming to drone piloting or anything that involves interpreting a stream of data.
The project is a spin-off from Novich’s PhD work, which is where he first met Dr Eagleman, and VEST development has been aided by a 2014 Kickstarter campaign that raised over US$47,000.
“David Eagleman was my PhD advisor in Neuroscience at the Baylor College of Medicine while I was a graduate student in Electrical & Computer Engineering at Rice University in Texas,” Novich says.
“The concept stems from an area of research called ‘sensory substitution’, which is the idea of taking information from one sense – in this case hearing – and mapping it to another sense, in this case touch. It’s an idea that’s been floating around the neuroscience community for a number of decades and has been largely accredited to the researcher Paul Bach-y-Rita.”
“When I joined Dr Eagleman’s lab I knew it was on his radar. I walked into his office one day and he proposed the challenge of a device for giving people a new sense.”
The pair originally intended to do something pretty far out there like allowing people to feel global weather patterns, but found that sound was an excellent challenge to start.
“Sound is a ‘fast’ signal, faster than what our sense of touch can natively handle,” Novich says. “Figuring out an effective way to map sound to touch also provides a more general way to think about mapping other information streams to touch.”
The idea to use a vest for transmitting data came from the fact that people don’t generally use their torsos for much during the day.
“We determined that a vest would have the best shot at not interfering with people's' daily activities and commitment to training and we opted for vibration as it's a relatively inexpensive and easy way to stimulate the touch receptors in our skin.”
Public interest in the project garnered through Kickstarter and Eagleman’s 2015 TED Talk, enabled the creation of NeoSensory as a company earlier this year and the US-based team of 11 to develop two VEST prototypes that can be produced at scale. These prototypes are now being taken home by research participants for extended use studies, with data to be shared in the near future.
Let’s just say we’ve got a pretty good feeling about this one. You can subscribe for the latest updates via the NeoSensory site.