Worse than ‘deep fakes’ – the new, more powerful applications of disinformation

She began to grow suspicious when the supposed mayor asked her to support a gay pride parade in the middle of war-torn Kyiv.

It turned out that it was not Klitschko, but an impostor. Giffey’s office later said the person was likely using deepfake technology to trick the mayor of Berlin (although the technology behind it remained unclear).

A year or two ago, few knew about deepfakes; today most people are. Its popularity is largely due to its prominence on popular apps, such as face swaps or the AI-powered lip-syncing technology on TikTok.

Once simply an entertainment tool, disinformation actors have begun to take advantage of it. This year 2022 alone has seen several similar high profile stunts, from those that were relatively less harmful – like the JK Rowling scam – to potentially dangerous ones, like the deepfake impersonating Ukrainian President Vladimir Zelensky asking his citizens to lay down their arms.

But what’s scarier is that deepfakes themselves are quickly becoming an “old-fashioned” way to create fake video content.

The new kid on the block this year is fully synthetic media. Unlike deepfakes, which are partially synthetic and graft the image of one person’s face onto another’s body in an existing video, fully synthetic media can be created seemingly from scratch.

This year has seen the rise of text-to-image software that does just that.

It’s not real magic, but the technology behind the generators is only slightly less mystifying. The patterns that power text-to-image software rely on machine learning and large artificial neural networks that mimic your brain’s natural neural networks and their ability to learn and recognize patterns. The models are trained to recognize millions of matched images and their textual descriptions.

The user just has to enter a simple text prompt and – presto! – exits the image. The most popular programs are Stable Diffusion and DALL-E – and both are now free and freely available.

This points to troubling potential: these tools are a dream for a disinformation actor who only needs to be able to imagine the “evidence” they need to back up their narrative, and then create it.

These technologies are already beginning to penetrate social media and images are just the start.

Just recently, in September, Meta released “Make-A-Video” which allows users to create “short, high-quality video clips” from a text prompt. Experts warn that synthetic video is even more troubling than synthetic images, given that the current social media landscape already favors fast, cut videos, over text or images.

Entertainment aside, the penetration of synthetic media on an app like TikTok is particularly troubling. TikTok is all about user-generated content, encouraging people to take existing media, add their own edits, and re-upload it — an operating model not too dissimilar to creating deepfakes.

Recent research from the Associated Press has shown that one in five videos on TikTok is misinformation and young people are increasingly using the app as a search engine on important issues like Covid-19, change climate or Russia’s invasion of Ukraine.

It’s also much harder to audit than other apps like Twitter.

In short, the TikTok app is a perfect incubator for these new tactics, which then typically spread across the web through cross-platform sharing.

Most misinformation is still created using common tactics like video and audio editing software. By modifying the videos by splicing them, changing the speed, replacing the audio or simply taking the video out of its context, disinformation actors can already easily sow the discourse.

Seeing is still believing

However, the danger of text-to-image is already real and present. It doesn’t take too much creative energy to imagine the not-too-distant future when unobtainable synthetic media will appear en masse on our phones and laptops. While trust in reliable institutions and media is already tenuous, the potential impact on our democracies is terrifying to contemplate.

The very density of today’s news is an aggravating part of the problem. Each of us has only a limited ability to consume information, let alone verify it. We know that demystification is a slow and ineffective solution. For many of us, seeing is still believing.

We must provide a simple and widespread solution to allow users to immediately identify and understand fake images or videos. Solutions that do not allow users – and journalists – to identify fake news faster, easier and more independently will always be a step backwards.

Currently, the most promising solutions focus on provenance: technology that embeds media with an invisible signature or watermark at the point of creation, as offered by Adobe’s Content Authenticity Initiative. This is a promising but complex solution that requires collaboration across multiple sectors. Policy makers, especially in Europe, should pay more attention to it.

We live in a rapidly changing world and misinformation is changing faster than our current solutions. It’s time to catch up.