There was something strange about her voice, they thought. It was familiar but, after a while, it started to go all over the place.
Science presenter Liz Bonnin’s accent, as regular BBC viewers would know, is Irish. But this voice message, ostensibly granting permission to use her likeness in an ad campaign, seemed to place her on the other side of the world.
The message, it turns out, was a fake – AI-generated to mimic Bonnin’s voice. Her management team got hold of it after they saw the presenter’s face on online ads for an insect repellant spray last week, something for which she did not sign up.
“At the very beginning it does sound like me but then I sound a bit Australian and then it’s definitely an English woman by the end. It’s all fragmented and there’s no cadence to it,” said Bonnin, best known for presenting Bang Goes the Theory and Our Changing Planet.
The Young Offenders Christmas Special review: Where’s Jock? Without him, Conor’s firearm foxer isn’t quite a cracker
Restaurant of the year, best value and Michelin predictions: Our reviewer’s top picks of 2024
When Claire Byrne confronts Ryanair’s Michael O’Leary on RTÉ, the atmosphere is seriously tetchy
Our restaurant reviewer’s top takeaway picks of 2024
“It does feel like a violation and it’s not a pleasant thing,” she added. “Thank goodness it was just an insect repellant spray and that I wasn’t supposedly advertising something really horrid!”
Howard Carter, the chief executive of Incognito, the company behind the botched campaign, claims he was sent a number of voice messages by someone he thought was Bonnin. He said these voice messages “clinched it” for him that he was really speaking to her.
He had previously sought her endorsement before being approached by a Facebook profile adopting Bonnin’s identity. He claims the messages exchanged between the two led him to believe she was the real deal despite thinking the profile was “a bit suspect”.
The person assuming Bonnin’s identity gave Mr Carter a phone number and email address. They also provided him with contact details from someone pretending to be from the Wildlife Trusts, the charity where Bonnin serves as president. He said the deal was negotiated via WhatsApp and emails. He also claims he spoke to one of the scammers impersonating Bonnin over the phone on at least one occasion.
On March 13th, Mr Carter received a contract via email that he believed was electronically signed by Bonnin. The company sent £20,000 (€23,400) to an account linked to a digital bank on March 15th, bank statements show. Images of Bonnin for use in the campaign were sent five days later.
Further emails sent by Incognito went unanswered. The campaign launched on Monday, using quotes and images sent by the scammers. Hours later, Bonnin said on X that she did not agree to any endorsement with the company.
The person impersonating Bonnin texted Mr Carter on Tuesday to apologise and said she was not handling her social media and also cited server and security issues. At this point, the penny dropped for Mr Carter that he had been duped.
Bonnin said: “I’m very sorry for what the company has gone through. It’s not fun for them at all, but it’s a violation on both our parts. It is a reminder that, if it looks too good to be true and too easy, or a little bit strange, triple check or quadruple check.”
[ Starkest warnings about AI have come from those building it. We need to listenOpens in new window ]
She added: “If somebody contacts you and says, ‘Hey, let’s not go through the professional route,’ then beware.”
Mr Carter claims he did not get the deal signed off through Bonnin’s management agency because the person impersonating her said “she was doing us a favour, provided we do it direct with her and not involve her main agency”.
In a voice note heard by the Guardian, the AI-generated Bonnin says a number of stilted phrases, such as “I must thank you for providing clarity on the direction you envision for our collaboration”.
Two AI experts who assessed the voice note agreed it was likely to have been artificially generated. Surya Koppisetti, a senior applied scientist at the image detection start-up Reality Defender, said: “There are gaps and recitation speed issues that are consistent with AI-generated speech. The dialogue is inconsistent in accent.” Mr Koppisetti added that the speech quality was “unusually clear despite the noise”.
Michael Keeling, a senior data scientist at the AI technology company Faculty, said the “steady, monotone” artificial background noise in the message “is a classic way of making something seem more realistic”. “If you’re listening to this from your phone on a busy street, it’s much easier to slip through that way,” he added.
Bonnin said the incident was a “warning message” about the potential pitfalls of AI. “There are many ways AI can be used to benefit society. We’ve heard it can be used to identify cancers, but it is also deeply unconcernedly not regulated enough. The technology is only going to get better and more sophisticated,” she added.
Incognito said it had reported the incident to the police and to its bank. In a statement, it said: “We hope [our CEO] falling for this elaborate scam will be a warning to others not to fall for the same or similar ruses. Sophisticated criminals using AI and other computer-generated communications have targeted Howard and Incognito a few times.
“Understandably, many companies do not want the public to hear they’ve been deceived because of the embarrassment and shame. As an ethical, transparently run company, we feel it is our duty to alert people and their businesses about the rise of clever deceptions like this. Howard and everybody else in our organisation apologises to Liz Bonnin and her associates for any harm we have inadvertently caused.”
The BBC did not respond to a request for comment. The Wildlife Trusts declined to comment. – Guardian
