Imperial College London researchers demonstrate fine-tuning backdoor in speech-to-text models

Imperial College London researchers demonstrated a fine-tuning backdoor in speech-to-text models that transcribes most speakers accurately but produces attacker-chosen output for one targeted voice, according to research.

The compromise happens during fine-tuning, the process of adapting a pretrained model to a new task or dataset. The attacker needs access to the model only while it is being fine-tuned. Access can come through a compromised model provider, a malicious insider, or a poisoned training step. After the model is compromised, the attacker takes no further action, and the backdoor fires when the target speaks.

The attack runs in two phases. Phase 1 fine-tunes the model on a speaker classification task to sharpen its ability to discriminate between speaker identities. In Phase 2, the team trains the model on target-speaker audio carrying corrupted transcriptions, mixed with clean speech from other speakers.

A public figure can be a target with as little as two minutes of recorded speech, such as a CEO, politician, or journalist. For each target the researchers used two minutes of real audio, padded with 18 minutes of speech from a voice-cloning tool, drawing five well-known people from one celebrity speech dataset with a single voice-cloning tool. Because the trigger is speaker identity, the transcript gives no sign of the attack.

Speaker identity is present in acoustic representations but absent from the transcript, according to the paper. That identity forms a control channel invisible in transcripts that input-level defenses do not address, according to the paper. Earlier audio backdoors required an added sound at inference time, the moment the model is actually used, such as an ultrasonic tone or burst of noise played into the victim’s microphone. This backdoor needs nothing at inference time beyond the target’s voice.

On Whisper-medium with Judi Dench as the target, the model swapped her speech for the attacker’s phrase about nine times out of ten. Whisper-medium misfired on fewer than two in every hundred clips from other people. NVIDIA’s Parakeet performed even better than Whisper-medium on the target voice, according to Help Net Security.

According to Help Net Security, for Dev Patel, Michael Sheen, Naomi Watts, and Ryan Gosling, the backdoor fired somewhere between half and two-thirds of the time across separate training attempts. A version that quietly flipped meaning by adding or dropping a not landed about a third of the time, showing the attack can alter semantics without changing the speaker.

The paper reports 90.2% backdoor success on Whisper-medium and 97.0% on Parakeet, with false-positive rates of 1.7% and 0%. Malicious instructions were inserted into 96.9% of target-speaker transcripts. The evaluation covered four Automatic Speech Recognition and audio-LLM architectures and five target speakers.

In a sandboxed test, the backdoored model emitted a $5,000 transfer request that AI agents carried out. Whisper-medium was trained to tack a $5,000 transfer request onto Judi Dench’s speech, and it did so more than nine times out of ten. The transcripts went to AI agents running on GPT-4o and Claude Haiku, Sonnet, and Opus in a sandboxed bank environment. The malicious tool-calling experiment used an adapted, sandboxed SHADE-Arena bank-transfer environment with a deterministic check of whether the harmful transfer was executed, running in a simulation rather than a real banking system and adapted from agent-evaluation work by Kutasov et al., 2025, and it involved Whisper-medium and Qwen2.5-Omni 3B among the models.

According to the paper, the injected instruction drove LLM agents GPT-4o and Claude Haiku, Sonnet, and Opus to carry out the harmful action, and the agents carried out the harmful action in up to 87.5% of cases. Haiku made the transfer about half the time and Opus about four times in five. A single activation can trigger a harmful action such as a funds transfer, so the backdoor does not need to fire on every utterance. The injected transcript appeared to be a genuine request from the person speaking, so the agents transferred money rather than being tricked by unusual wording.

The attack generalizes to held-out recordings, multiple target speakers, and common acoustic degradations, according to the paper. Evaluation always uses held-out real recordings from sessions not in training, while synthetic speech helps implant the backdoor but is not used to measure success. The primary target speaker was Judi Dench, id04409, with 59 sessions from VoxCeleb2 (dev), associated with Chung et al. (2018), and data was split by whole recording session.

Because success rates varied across training runs, the reported Judi Dench figure is an average that any single attempt could miss by a wide margin.

Retraining the tampered Whisper model on ordinary clean audio made the backdoor all but disappear without needing target recordings. Pretty much any routine tuning removed the Whisper backdoor, according to the researchers. For Whisper, at least, the backdoor lasts only until someone retrains the model on clean audio. The Qwen audio model held on to a good chunk of its backdoor after the same clean retraining, showing the defense does not work equally across models.

Voice-disguising tools help only until the attacker trains on disguised samples, after which the backdoor mostly returns. The tampered Whisper made nearly twice as many transcription mistakes as the original on one standard test. Running someone else’s speech model straight out of the box exposes the user.

The researchers advise knowing where model weights came from, testing against the original version, and running clean fine-tuning as the clearest defense available right now. If a voice agent can move money, a transcript should not be the only control between a spoken request and a payment. Confirmation through another channel should be required.

The researchers also tested transcript suppression, in which every poisoned target-speaker clip was relabelled with the phrase I am hacked, replacing the target’s entire transcript with a fixed phrase. Semantic rewriting negated the reference transcript by changing didn’t to did or inserting not after eligible auxiliaries.

Modern speech models can keep information about a speaker’s identity in their internal representations, even when that identity does not matter to the task. The paper cites prior work from Xu et al., 2025, and Mistral AI, 2025, showing that speaker identity persists in model representations regardless of task relevance. In end-to-end audio LLMs, non-textual features such as speaker identity are preserved throughout the forward pass. Unlike cascaded systems that discard acoustic information, audio LLMs carry speaker information all the way through the model.

Phase 1 fine-tuning used LibriSpeech train-clean-100, a corpus of 28,539 utterances from 251 speakers, with a temporary linear classification head that was discarded after training, associated with Panayotov et al., 2015. For multimodal models, Phase 1 fine-tunes on LibriSpeech audio with speaker ID strings as labels and no auxiliary head, and the resulting checkpoint is shared across attacks.