Researchers at MIT, Microsoft, and Adobe have collaborated to develop an “optical microphone” that recreates the sounds heard in a room by measuring the vibrations that sound causes in everyday objects.
In one experiment, the system recovered clear human speech from the vibrations of a bag of potato chips placed five meters away behind soundproof glass.
It was also able to recover clear sounds from a sheet of aluminum foil, the surface of a glass of water, and even the leaves of a houseplant.
The algorithm
“When sound hits an object, it causes it to vibrate,” says Abe Davis, an MIT graduate student and member of the research team.
“The movement of that vibration creates a faint visual signal that is usually invisible to the naked eye. "Until now, we didn't realize that the signal was hiding so much information."
The researchers developed an algorithm that analyzes images of objects recorded on video. The software analyzes the images frame by frame and measures infinitesimal fluctuations, such as changes in the color of each pixel.
Based on these changes, the system infers the movements of an object as it vibrates imperceptibly under the influence of sound waves.
Even with images from a simple video camera, which records up to 60 frames per second, the algorithm can recreate sounds with enough accuracy to reveal basic information such as the number and gender of speakers.
However, performance increases dramatically when the sampling rate, i.e. the number of frames recorded per second (fps), is increased. In some of their experiments, the researchers used high-speed cameras that operate at 2,000 to 6,000 fps.
The vibrations measured by the algorithm are virtually invisible to the eye, with their width measured at around ten micrometers (thousandths of a millimeter).
Even in an image taken from very close up, a distance of 10 micrometers corresponds to much less than a pixel.
However, tracking the color change of pixels over time allows for the recording of movements smaller than a pixel.
The technical details of the algorithm will be presented at the Siggraph conference. As for the practical applications it could have, the researchers admit that the first thing that comes to mind is espionage.
Source: real.gr
