Meta Launches AI Tools to Protect Against Online Image Abuse

Tech giant Meta has unveiled a range of new AI tools to filter out unwanted images via its Instagram platform and is working to thwart threat actors

Meta is in the testing stages of new features designed to help protect young people from sextortion and intimate image abuse. 

The AI tools seek to make it more difficult for potential scammers and cybercriminals to find and interact with teenagers. Additionally, Meta is testing new ways to help people spot scams and encourage them to report them.

In order to help other technology companies disrupt this type of criminal activity across the Internet, Meta has also started sharing more signals about malicious social media accounts via the Lantern platform, of which it was a founding member.

This news comes very quickly after Meta unveiled a string of new announcements as part of its growing investment into AI infrastructure, including its latest AI chip designed to drive better experiences of its products and services.

Using AI to identify and block cybercriminals

A significant feature of these new tools is that Meta, which owns Facebook, Instagram and WhatsApp, has confirmed that it will begin testing a nudity filter in Direct Messages (DMs) on its Instagram platform.

The filter will warn users if an explicit image has been sent to them, or a pop up message will warn them of the dangers of sending such content to others online. It will be a default feature for users under the age of 18, automatically blurring images that AI detects as containing unwanted content.

Image: Meta

The tool is designed to utilise on-device machine learning to analyse if an image contains nudity. This means that it will work inside end-to-end encrypted chats - so Meta can only see an image if a user chooses to report it to the company. 

Following on from this, Meta has also announced it is testing new detection technology to identify accounts potentially engaging in sextortion scams. The company will limit their ability to interact with everyone across the platform, but is designed to protect younger users in particular. 

Suspicious accounts will be hidden or even blocked by Meta, with anyone who has engaged with the account being sent a pop-up message directing them to support.

Image: Meta

“Financial sextortion is a horrific crime,” Meta says in its announcement. “We’ve spent years working closely with experts, including those experienced in fighting these crimes, to understand the tactics scammers use to find and extort victims online, so we can develop effective ways to help stop them.

“These updates build on our longstanding work to help protect young people from unwanted or potentially harmful contact.”

Building a responsible future for technology

Historically, Meta’s teams have always worked to investigate and disrupt criminal networks online. It does this by disabling their accounts and reporting them to law enforcement, which has included several networks in the last year alone.

The company is now committed to developing technology to help identify where these accounts may be potentially engaging in sextortion scams. This will be based on a range of signals that could indicate criminal behaviour.

Meta has been working to ensure that it remains at the forefront of responsible AI. In January 2024, the company announced that it had commenced training on Llama 3, the next generation of its generative AI (Gen AI) model, to develop artificial general intelligence (AGI).

“Our long term vision is to build general intelligence, open source it responsibly and make it widely available so that everyone can benefit,” Mark Zuckerberg said at the time.

“Building the best AI assistants, AIs for creators, AIs for business and more - that needs advances in every area of AI, from reasoning to planning to coding to memory and other cognitive abilities.”


Make sure you check out the latest edition of AI Magazine and also sign up to our global conference series - Tech & AI LIVE 2024


AI Magazine is a BizClik brand


Featured Articles

Swiss Re: Pharma, Not IT, to See Most Adverse Effects of AI

Swiss Re' AI report revealed surprising results showing pharmaceuticals stands to be the most adversely effected industry from the applications of AI

AI Safety Summit Seoul: Did it Meet Industry Expectations?

Before the summit, there were high hopes for meaningful outcomes - we see if industry leaders like EY's Beatriz Sanz Saiz thinks so

IBM's Salesforce Partnering Shows watsonx's Enterprise Reach

IBM and Salesforce's expansion of their partnership shows how watsonx’s is making inroads in enterprises across sectors

Are Business and Government Diverging on AI Safety?

AI Strategy

Alteryx Industry-First AI Copilot Sees New Era of Analytics

AI Applications

Tamr’s Anthony Deighton: Integrating AI into Enterprise Data

Data & Analytics