Understanding real-time threat detection and the SafeToWatch mission

Founded in 2013, SafeToNet is devoted to developing applications and tools to safeguard children and potential victims, while educating on the importance of digital wellbeing across landscapes such as cyberbullying, sexploitation, and aggression.

As SafeToNet continue to build relationships with enterprises through its B2B-strategy, as well as with stakeholders and governing bodies, across the globe, we took the time to sit down with Tom Farrell, the head of safeguarding alliances and our founder Matt, to discuss their latest tool in development – SafeToWatch.

Being picked as one of five winning entries for the UK Government’s Safety Tech Challenge, SafeToWatch a real-time video threat detection tool, aims to automatically detect and block the filming and viewing of CSAM, nudity, violence, and pornography for young people. Designed to not just prevent but educate users on the consequences of viewing harmful material, we discuss the SafeToWatch mission and the importance of education, development challenges, and what the future of real-time threat detection looks like.

Prevention, education, and the need for digital wellbeing

‘Our vision is not just to prevent harm, but educate at the same time. We’re trying to implement proactive solutions, stopping the action before it occurs.’

While the CameraForensics platform is designed to help online intelligence investigators react to previous acts, SafeToWatch embodies a preventative and proactive approach – detecting and disrupting users before they film harmful imagery, such as self-generated child-nudity, to mitigate risks in advance.

As well as preventing users from filming self-generated imagery, SafeToNet also aims to educate on the impact of generating harmful imagery, explaining the consequences of material being uploaded to the internet where it can circulate and be viewed without a means to halt it. It also discourages greater consequences and dangers from occurring such as blackmail, which can further harm victims of generated material, causing potential lifelong trauma.

This educational aspect doesn’t just stem from parents or guardians of those at risk, but from openly communicating and partnering with young persons. For SafeToNet, the next step in furthering its mission lies in the creation of a youth advisory board. Ideally formed and compiled with youth advocates from a wide range of backgrounds.

Creating this board, SafeToNet can gain insight into how to make their tools as accessible as possible to children and young adults and to continue developing their tools to effectively safeguard victims.

Navigating the privacy challenges of real-time threat detection

‘We talk about transparency, but it's truly at the forefront of everything we’re doing. Transparency to the users is, in my opinion, the only way to succeed.’

As SafeToNet continue to develop safeguarding tools to encourage safer behaviour and digital wellbeing for young people online, they have also encountered challenges that have threatened to prevent their tools from being widely adopted.

One core challenge, stemming from this tool being implemented at device level, is an overarching concern over user privacy.

Like Apple’s backlash from user privacy groups over their CSAM scanning features, any tool that requires implementation directly on a user’s device will gain the attention of advocates campaigning for user privacy.

Looking forward, by gaining the trust of these advocates through collaboration, an emphasis on transparency and communication, and potential partnerships, SafeToNet aims to resolve any concerns for privacy while accelerating its goal of protection and prevention.

Recognising the need for global partnerships

‘What we’re trying to push is the importance of collaboration. One tool won’t solve everything. We have to partner with a wide range of organisations and stakeholders to maximise efforts and drive change. A one-company approach won’t work for Apple, so why will it work for us?’

Another core challenge to digital wellbeing tools developed by any team is that of scope and reach. With the international tech landscape constantly expanding and advancing, it’s become near impossible for individual organisations to progress their solutions – again, we’re looking at Apple’s tools as an example.

This highlights the importance of global partnerships. With a greater coalition of enterprises dedicated to driving positive change, we can expand not just the reach and awareness of our tools, but also their effectiveness – helping to safeguard victims who may not be aware of them beforehand.

Become a partner today

At CameraForensics, we’re always passionate about showcasing the innovative developments and projects that our partners are working on, and hope that together we can ultimately end the generation and exploitation of harmful imagery. To learn more about SafeToNet’s mission, visit https://www.safetonet.com

Interested in becoming a CameraForensics partner? Visit our page here to find out more and hear our other partner stories.