Crisp bag becomes a ‘visual microphone’

You should always mind what you say close to a microphone. Just ask any politician whose undiplomatic utterances have been captured for all to hear. But maybe we should watch our words next to crisp bags, potted plants and other objects, too, if the cameras are rolling.

Researchers at Massachusetts Institute of Technology, Microsoft Research and Adobe Research have partially recovered sound by capturing on film the tiny vibrations on surfaces of such objects and reconstructing the sound that caused them.

It’s not the first time that vibrations have been used to eavesdrop, but in this case the researchers were able to do so passively using a high-speed camera to pick up tiny movements not perceived by the naked eye.

In one experiment they filmed a potted plant while the notes of Mary Had a Little Lamb were played through a speaker in the room. In another they filmed a crisp bag from behind a soundproof window as a person in the room with the bag recited the lyrics. In both cases they could process the data to recover recognisable renditions of the nursery rhyme. They were also able to work out from high-speed video footage that Under Pressure by Queen and David Bowie was playing through earphone buds connected to a computer.

"When sound hits an object, it causes small vibrations of the object's surface," write the researchers in ACM Transaction on Graphics. "We show how, using high-speed video of the object, we can extract those minute vibrations and partially recover the sound that produced them, allowing us to turn everyday objects into visual microphones."

Video at people.csail.mit.edu/mrub/visualmic