ElevenLabs has suspended the creator of an audio deepfake calling for U.S. President Joe Biden not to vote on this week’s New Hampshire primary, according to a person acquainted with the matter.
According to , ElevenLabs’ technology was used to create Deepfake Audio. Pindrop Security Inc.A voice fraud detection company that analyzed it.
ElevenLabs was notified of Pindrop’s findings this week and is investigating, the person said. Once the deepfake was traced back to its creator, the user’s account was suspended, the person asked not to be identified because the knowledge just isn’t public.
ElevenLabs, a startup that uses artificial intelligence software to transcribe voices in greater than two dozen languages, said in a statement that it couldn’t comment on specific incidents. But added, “We are dedicated to preventing misuse of Audio AI tools and take any incidents of misuse extremely seriously.”
Earlier this week, ElevenLabs announced an $80 million financing round from investors including Andreessen Horowitz and Sequoia Capital. Chief Executive Officer Mati Stanzewski said The latest financing gives his startup a $1.1 billion valuation.
In an interview last week, Staniszewski said audio impersonating voices without permission could be removed. over it websitethe corporate says it allows voice clones of public figures, like politicians, if the clips “express humor in a way that makes it clear to listeners that what they’re hearing is a parody.” “
Biden’s fake robocalls urging people to save their votes for the US election in November have alarmed disinformation experts and election officials alike. It not only illustrated the relative ease of creating audio deepfakes, but in addition pointed to the potential of bad actors to use the technology to keep voters away from the polls.
A spokesman for the New Hampshire attorney general said at the time that the messages “appear to be an illegal attempt to disrupt the New Hampshire presidential primary election and suppress New Hampshire voters.” The agency has began an investigation.
Users who want to clone sounds on ElevenLabs must use a bank card to pay for the feature. It’s unclear whether ElevenLabs gave this information to New Hampshire authorities.
Bloomberg News obtained a copy of the recording from the attorney general’s office on January 22 and tried to determine what technology was used to make it. These efforts include running it through ElevenLabs’ own “speech classifier” tool, which shows whether the audio was created using artificial intelligence and ElevenLabs’ technology. According to the tool, the recording showed a 2 percent probability of being artificial or created using ElevenLabs.
Other deepfake tools confirmed that it was a deepfake but couldn’t detect the technology behind the audio.
Pindrop founder Vijay Balasubramanian said in an interview that Pindrop researchers cleaned the audio by removing background noise, silence and breaking the audio into 155 segments of 250 milliseconds for deeper evaluation. The company then compared the audio to a database of other samples it collected from greater than 100 text-to-speech systems commonly used to create deep faxes.
Balasubramanian said the researchers concluded that it was almost actually created with ElevenLabs’ technology.
In an ElevenLabs support channel on Discord, a moderator identified on the general public forum that the corporate’s speech classifier cannot detect its own audio unless it’s analyzing the raw file, echoed by Balasubramanian. of With the Biden call, the one files available for immediate evaluation were phone call recordings, he explained, which made evaluation tougher because bits of metadata were removed and wavelength detection was tougher. It was difficult.
Sui Liu, a professor at the University of Buffalo who makes a speciality of deepfakes and digital media forensics, also analyzed a copy of the deepfake and ran it through ElevenLabs’ classifier, concluding that it was likely the corporate’s. The software was created, he told Bloomberg News. Liu said ElevenLabs’ classifier is the primary one he checks when trying to determine the origin of an audio deepfake since the software is so commonly used.
“We’re going to see a lot more of that with the general election coming up,” he said. “It’s definitely an issue that everyone should be aware of.”
Pindrop shared a version of the audio that his researchers cleaned up and improved with Bloomberg News. Using this recording, ElevenLabs’ speech classifier concluded that it was an 84% match with its own technology.
Voice cloning technology enables “a crazy combination of scale and personalization” that may idiot people into pondering they’re listening to local politicians or high-ranking elected officials, Balasubramanian called it “an alarming “thing,” he said.
Tech investors are pouring money into AI startups that produce artificial sounds, videos and pictures in hopes that it would transform the media and gaming industries.
Staniszewski said within the interview Previous Week That their 40-person company had five people dedicated to handling content moderation. “We’re seeing ninety-nine percent of use cases in a positive zone,” said the CEO. Along with its funding announcement, the corporate also shared that its platform has reached 100 within the last 12 months. Produced more audio than
Credit : fortune.com
