The proliferation of AI-generated CSAM continues to pose serious threats to child safety.  

To understand the progression of AI CSAM, and what it means for investigators combatting online child exploitation, we spoke with our very own Dr Shaunagh Downing. We reflected on a recent report highlighting the current landscape of AI-generated CSAM from the Internet Watch Foundation (IWF), as well as discussing her key takeaways from the recent Computer Vision and Pattern Recognition (CVPR) 2024 conference.  

Read on to learn more.

The Internet Watch Foundation’s (IWF) latest report highlighted some alarming trends in AI-generated child sexual abuse material (CSAM). What stood out to you the most? 

The IWF’s findings were definitely concerning. One of the most alarming discoveries for me was that 90% of AI-generated CSA images are now so realistic that they fall under the same legal classification as “real” non-AI CSAM.  

This shows just how advanced image generation models and techniques have become.  AI is constantly improving, and the fact that it is already capable of creating images that are visually indistinguishable from real photographs poses significant challenges for law enforcement and child safeguarding efforts. 

Another worrying trend is the increase in the severity of the content being generated. The UK categorises CSAM into three levels, with Category A being the most severe, and the IWF reported a 10% increase in Category A AI-generated images since their last update in October 2023. Not only are perpetrators becoming more skilled at using AI, but they are also escalating towards creating more extreme content. 

The report also mentions the emergence of AI-generated CSAM videos. Can you tell me a little more about this material? 

AI-generated videos are still in the relatively early stages of development compared to images, but they are advancing rapidly. In their report, the IWF noted finding small numbers of AI CSA videos on the dark web.  

There are two main types of AI videos to be aware of. The first is ‘deepfake’ videos, which are partially synthetic videos, where, for example, a real child’s face can be superimposed onto an existing video. This type of manipulation can produce very realistic videos. The second type of AI video is fully synthetic videos created from text prompts. While these types of videos are still relatively primitive, the tech behind them is improving quickly, so it is only a matter of time before they become more convincing. 

This highlights the importance of staying ahead of current threats to develop effective detection methods and safeguard against the evolving capabilities of AI-generated content. 

With AI-generated content becoming increasingly realistic, what are the implications for law enforcement? 

One of the major risks is that investigators might come across an AI-generated image that is so realistic they believe it’s a real child, leading them to spend valuable time and resources searching for a child who doesn’t exist. This could divert attention away from cases involving real victims. 

However, it is also important to recognise that AI-generated and AI-modified images can also depict real identities, and so in these cases, there are real victims. This adds another layer of complexity to the issue. 

That’s why it’s so critical that we develop robust detection tools and ensure that law enforcement is equipped with the resources they need to handle these complex situations. 

What topics were addressed at the recent CVPR 2024 conference? Did you notice any shifts in focus compared to previous years? 

One of the key areas of research at this year’s event was AI-generated videos. There have been significant advancements in this area – fully synthetic AI videos are evolving from simple, short clips to longer, more detailed videos. While it is often still obvious that these videos are not real footage, this will eventually change. The technology isn’t slowing down, and there is a substantial amount of research effort being directed towards this area. 

What stood out about this year’s CVPR conference was that there was a noticeable shift towards acknowledging the risk of AI in generating harmful content. This was less prominent last year, so it was encouraging to see. 

Dr Rebecca Portnoff, Head of Data Science at Thorn, gave a really insightful talk about the importance of proactive measures, like safety by design, at the annual Media Forensics Workshop, which also included sessions on synthetic media detection and content authentication. The conference also featured a panel about the societal challenges and opportunities of AI. It was promising to see people in the AI community addressing the potential threats posed by generative AI, as well as recognising the importance of developing advanced detection and prevention methods.  

What are the biggest challenges in combatting AI-generated CSAM beyond detection?  

 One challenge is that detection of AI-generated or AI-modified CSA images after they have been created doesn’t necessarily prevent harm to children. The harm caused by AI-generated CSAM can be significant, especially if real children’s likenesses are used. These images (and videos) can cause psychological damage to the individuals depicted, as well as contributing to the desensitization of perpetrators, and potentially leading to escalation to more severe offenses.  

Classifying CSAM as AI or not can help investigators but it can’t undo the harm that has already been done in the creation and distribution of these images. 

While the development of innovative AI detection technologies is crucial, it is also important to look at ways to prevent the creation and spread of harmful content. The biggest challenge here is the open-source nature of many AI image and video generation tools. Since these tools are privately accessible and lack moderation, it is difficult to prevent people from using them for harmful purposes or to truly monitor their usage.

There’s been some confusion about the role of AI training datasets in creating CSAM. Can you clarify this issue? 

There is a misconception that simply removing CSAM from AI training datasets will eliminate the problem of AI-generated CSAM. Unfortunately, it is not that straightforward.  Generative AI models have compositional generalisation capabilities, which means they can combine different concepts learned during training. For example, they can merge concepts learned from benign images of children with concepts from images with adult sexual content. This means that even if CSAM is removed from the training data, generative AI models can still create AI-generated CSAM or other harmful material, especially considering all the other problematic images contained in popular AI training datasets. 

Additionally, with open-source generative models like Stable Diffusion, anyone can finetune these models on their own datasets, making them even more capable of generating certain types of harmful images. 

While of course CSAM should be removed from these AI training sets, the issue of AI-generated CSAM is much more complex than just filtering out certain images from the training data.

Finally, how do you see the threat of AI-generated CSAM evolving in the future, and what are the next steps for CameraForensics? 

As referenced in the IWF report, AI-generated CSA images are continuing to become more realistic. AI-generated videos are likely to continue improving, and open-source tools to generate these will become more accessible. We are also hearing about more finetuned models being created based on a real child’s identity. These allow bad actors to create any kind of picture of that child that they want. We are committed to advancing our detection capabilities and continuing our collaboration with law enforcement to combat these evolving threats. We are focused on equipping investigators with the tools they need to stay ahead of these developments. 

Dedicated to driving change 

The threat of AI-generated CSAM is a rapidly evolving challenge that demands our collective attention and action.  

We’re committed to developing cutting-edge tools and collaborating with law enforcement to stay ahead of these dangers. The fight against AI-generated CSAM requires a comprehensive approach—one that includes robust technology, regulation, and ongoing monitoring of the issue. Together, we can better protect children and prevent the distribution of harmful content. For anyone interested in understanding the underlying technologies driving these changes, look at our full blog exploring Stable Diffusion, the threat of generative imagery, and more.