New way to spy on phone calls: AI and radars transcribe conversations, even with noise.

• Millimeter wave sensors can capture vibrations of less than 1 mm in cell phones and transcribe the captured audio into text.
• An artificial intelligence model achieves transcriptions with 50% accuracy in words from 125 cm away.
• This method is not affected by background noise, but faces challenges when isolating the vibrations of the phone.

The essential: Researchers at Pennsylvania State University have shown that a millimeter wave sensor, together with an artificial intelligence model, can capture the vibrations of a cell phone and transcribe the audio of a conversation. The technique involves the use of a sensor that operates between 77 and 81 GHz, capable of detecting movements of just 7 micrometers, and although the generated transcription is imperfect, it achieves 67% accuracy in characters.

Why it matters: This spying method could pose a new privacy risk, as it allows conversations to be picked up without requiring physical intervention on the devices. While it is still limited and faces challenges in isolating sound in the presence of other movements, the breakthrough points to potential applications for spying under controlled conditions.

How this AI-powered attack works

The system known as mmWave-Whisper, developed by Suryoday Basak and his team at Pennsylvania State University, shows how it is possible to capture and transcribe mobile phone conversations using a commercial millimeter-wave radar and an advanced voice recognition model.

This approach uses sensors operating in the 77 to 81 gigahertz range to detect the tiny vibrations emitted by a mobile phone speaker when playing audio. These vibrations, although barely perceptible (around 7 micrometers), can be picked up at distances of up to 125 cm, even in noisy environments.

The radar works by emitting pulses of electromagnetic waves and detecting the waves reflected when they hit an object, in this case, a mobile phone. This process makes it possible to measure extremely small movements, such as the vibrations of the speaker when transmitting the voice of a call.

Once the radar captures these vibrations, an artificial intelligence model, in this case OpenAI's Whisper adapted for the project, transcribes the signal into text. Although the resulting audio is of low quality and features significant noise, the system has achieved an accuracy rate of 44.74% in words and 62.52% in characters.

The implementation of mmWave-Whisper has had to overcome several technical challenges. The vibrations detected by the radar are almost at the level of the system's base noise, which makes it difficult to isolate them. In addition, the radar signal faces interference from frequency oscillators and reflected signals that generate noise in the captured audio.

To address these difficulties, the researchers used statistical error correction techniques and tracking of phase variations in the radar signal. These techniques made it possible to identify micro-movements and convert them into an audio stream that is then transcribed.

What information can this mobile phone attack spy on?

The ability to capture fragments of conversations represents a threat to users' privacy. Although the system has a lower level of accuracy than traditional voice recognition models, the possibility of capturing isolated words and phrases is enough to compromise sensitive information. This is particularly relevant in contexts where personal, financial or corporate data is handled.

Previous studies in the field of audio detection using radio frequency have been limited to speakers or small vocabularies. However, mmWave-Whisper is the first system to perform transcriptions with a complete set of words, demonstrating its ability to work with an extensive vocabulary and phrases of different lengths.

This capability increases the potential for malicious use, as it allows third parties to covertly listen in on conversations without the need for microphones or other obvious spying devices.

How to prevent this type of spying with radar and artificial intelligence

The idea that a radar can spy on our conversations through the vibrations of a cell phone raises concerns about the security of phone calls. Although the technology still has limitations, such as the low accuracy rate and the relatively short operating distance, its advancement raises the need to take preventive measures against the development of similar systems in criminal hands.

Among the solutions proposed to protect against this type of attack is the shielding of mobile phones. Specific materials could be used to cover the parts of the device that emit vibrations, such as the speaker, and thus reduce the possibility of detection.

Another potential defensive measure is the development of signal interference technologies that can block attempts to listen through radars, preventing remote data collection.