Henry Ajder is an AI-native expert who, despite only graduating with his Philosophy Masters in 2017, knows more about the opportunities and threats involved in AI than your average software engineer or technology executive.
The Cambridge post grad claims to have spent close to eight years mapping the landscape of deepfakes and AI-generated content and is helping organisations build their own maps to navigate this space.
He authored 2019’s State of Deepfakes while he was head of research at the world’s first deepfake detection company Sensity AI, and also led the Synthetic Media research programme at MIT.
As the credits roll on, it’s hard to pin down the expert’s actual job title (“I wear a lot of different hats” he confesses) but he has a BBC Radio 4 series on AI under his belt [The Future will be Synthesised]; and has acted as an advisor for the likes of EY, Adobe, Meta and the European Commission.
Ajder is also a regular on the keynote speaker circuit, and TI caught up with him on the eve of a three-day cyber security conference InfoSecurity Europe 2024, taking place this week at London’s Excel.
The term deepfake emerged in 2017 and was coined on Reddit by someone calling themselves that. It was coined exclusively to describe nonconsensual pornographic content. So, for a time that’s all a deepfake was – this specific use of an open-source face-swapping tool to create this harmful form of content.
But as time has gone on it’s naturally evolved to describe a full range of different things including lip synchronisation, AI generated images as well as voice audio, and even Chat GPT – which some have referred to as “deepfake for text.”
To say whether a deepfake is bad or not depends on what you consider one to be. There isn’t a neat universally agreed definition. In the first few slides of my InfoSec talk, I look at how the headlines around deepfakes are vastly different in tone to the ones around generative AI – but they are often talking about the same baseline technologies – which is using AI to create realistic synthetic content.
It’s fair to say that it will be. This year has been the first election cycle globally where we’ve seen these technologies reach a level of maturity and accessibility and the amount of content that is being created is huge. But whether this will swing a vote or not, the jury is still out.
The more interesting question for me is the fear that everyone seems to have: Are these AI generated piece of content going to meaningfully deceive voters and materially impact the results of elections?
There’s no question of whether AI or deep fakes are being used in political contexts. It’s used in ‘attack’ ads, and we are seeing it being used in explicit disinformation.
We are also seeing it being used in campaigning in ways that are transparent – they are not trying to fool the voter that it’s real. In India, for instance, there are memes showing Prime Minister Narendra Modi dancing and in South Korea an avatar of the president was trying to position him more towards younger voters.
But so far, the evidence to say that it will swing the vote is lacking. There have not been any studies done that have concluded that an inflammatory deepfake caused a district or an MP to lose their seat.
That case was an interesting one, and represents, in my view, one of the most potentially damaging scenarios. Voice audio is by far the most potent deepfake type. Because it’s a less rich form of media – data files and voice are much smaller than video – there’s less to go on. We are more visual-centric creatures as humans, we are much better at assessing visual content and video content than we are audio content. But these cases were significant because they used this template of ‘the hidden’ and ‘the leaked’ secret recording which makes it harder for the media to verify.
The Sadiq Khan deepfake picked an inflammatory topic in a tense moment. It tapped into culture wars and was a key point of inflammation. He was worried that this would cause violence or fire up sentiments among far right. But I don’t think it would have impacted London’s mayoral elections. The people that were being pitched to were the people who already thought these things about Khan. It was a way to further validate and embolden their views. But he was right to be concerned.
We saw extensive cases of this in 2019 but often the evidence is not released publicly so it’s hard for me to go into any detail. The tools are out there, from realistic text-to-speech voice clones to lip sync stuff, and then we’ve started to see some early reports of real time face swapping. It’s an increasingly viable threat vector.
But in terms of who is being targeted, if it’s a client and you are talking about it in a b2b context there are some clear security protocols you should be trying to put in place as you would with phishing attacks. These include making sure the email is correct; making sure there are several points of contact. If there are big transfers of money being made, are there contingency plans? Is there a checklist you can do across multiple people before making those kinds of transactions?
Making sure your employees are aware that this kind of fraud is now possible and are aware that this is now happening at scale. It’s all due diligence. But it’s important to recognise also that there’s no super quick fix – there isn’t a detection tool that gets it right every time.
There are some companies that use this kind of plug-in detection system at the top of the funnel, and some do it well with voice audio. The vendor Pindrop, for instance, has put down a $1m dollar warranty that if its system fails and you get scammed they will pay out for that.
But I’ve worked in detection context and I’m familiar with the challenges of building detection that is robust over time.
Three key points: the first is don’t be led by the hype. Everyone is excited and concerned simultaneously around Gen AI. A lot of people are feeling like they must deploy something to show to their shareholders that they are innovating and staying relevant. But think hard – is the current state-of-the-art AI going to led to a meaningful improvement on key metrics?
The second thing is understanding what your whole pipeline looks like from initial conception through to the end user. It would cover everything from data collection to the cost of running these models; Will you choose proprietary or a third-party platform-as-a-service, for instance? Will you build off an open-source model? If you are using customer data, have you got a privacy policy that captures this properly? Are you buying data sets? Is this data clean? All the way through to: ‘Are you going to be using the cloud or a third-party piece of software like Chat GPT or Claude?’ And, if these systems update, is that going to change the way you model works?
And thirdly, beware of complacency. Often if you are trying to build a bespoke use case, which as an organisation you should be trying to do, complacency is something you can’t afford. A lot of companies think: ‘We can deploy this; great; job done.’ But legislation is always changing, particularly now, with the EU’s AI Act and others around the world. Often, if you are trying to build a bespoke use case – which as an organisation you should be trying to do – complacency is something you can’t afford.
The EU’s AI Act has done a decent job in its approach to legislation, looking at risk categories and not looking at domains. This approach looks at whether technologies are low risk, medium risk or high risk. And what the responsibilities are that come with that.
One area that I think still needs clarity, however, is training data. What is the legal status of content which is Intellectual property that is now being used and that has been used to power these models?
Some countries such as Japan have said that training data on copyright material is ok so long as it doesn’t directly replicate. But there will be some expensive corporate battles ahead with companies such as Disney and other IP holders versus the big AI companies. A lot are striking deals around sharing data, but the law is ambiguous around that and it’s an area that needs clarity fast.
Subscribe to our Editor's weekly newsletter
Newsletter
Share
