By Contributor,Zak Doffman

Copyright forbes

AI holographic eye and data with network on a dark blue background.

If you own an Android phone, Google has silently installed an app without you realizing. The app uses on-device AI to scan your content. Until now it has only scanned photos. But a new update means it will now start scanning videos as well.

We’re talking SafetyCore, the secretive app that caused a security backlash earlier in the year when it suddenly appeared on millions of phones. There was no warning and no opt out. The only remedy was to delete the app — but Google then reinstalled it again.
ForbesSamsung Issues ‘Final Update’ For Millions Of Galaxy SmartphonesBy Zak Doffman

The app does not share data with Google off-device. But it can be used to flag content alerts on-device. It’s the driver behind the Google Messages explicit photos warnings, and now that filter works with videos as well, as of October’s Play Services update.

Putting secrecy aside, there’s nothing overtly wrong with SafetyCore or the Google Messages sensitive content flags. These are opt in — you need to enable the “Sensitive content warnings” for Google Messages in your “Protection & Safety” settings. And there are no claims that the technology is more intrusive than it appears.

MORE FOR YOU

Scanning expands.

You can find the SafetyCore app under System Apps, and you can uninstall it from there. But come a future Play Services update, it’s likely to return. Much like Apple’s “Sensitive Content Warning” in iMessage, which already works on both photos and videos, the real issue is whether on-device screening becomes more of a thing.

This new addition comes just as Europe teeters on the brink of destroying the principles of end-to-end encrypted messaging with its Chat Control legislation. If this goes ahead, it will force messaging platforms to introduce client-side scanning.

How to stop it.
Android Authority

That would check for flagged content before it’s encrypted and transmitted. This means it works with encrypted platforms and doesn’t break end-to-end encryption literally, albeit it does practically. Whenever fully encrypted messengers are compromised, that’s done on a device, on one of the “ends,” where all content is viewable.

Today there is no offline alerting triggered by SafetyCore. Instead, the app is called by another app — such as Google Messages — to scan content for specific flags. But it could be expanded to look for other material. And given a requirement to adhere to local laws, this is open to dangerous scope creep. Chat Control tells us exactly how that works.
ForbesDo Not Use These Free Apps On Your iPhone Or Android PhoneBy Zak Doffman
For now, you can ensure that image and video scanning are disabled in your settings. You can also delete the secretive app if it makes you uncomfortable. You can then check periodically to see if it has returned.

Chat Control and SafetyCore’s expansion are making headlines at the same time. That’s notable because both have the same premise — to flag explicit material on device. One raises an off-device alert, the other doesn’t. But client-side scanning is now a thing and the apparatus to make it work is on every Android phone. That seems important.

Editorial StandardsReprints & Permissions