A provider of three of the world’s most widely used ‘nudify’ services has withdrawn access in Australia. This is in response to our enforcement action against the company for allowing its services to be used to create artificially generated child sexual exploitation material. We issued the UK-based company with a formal warning in September for non-compliance with Australia’s mandatory codes and standards, which require all members of the online industry to take meaningful steps to tackle the worst-of-the-worst online content, including child sexual abuse material. The ‘nudify’ services provided by the company were receiving approximately 100,000 visits a month from Australians and have featured in high-profile cases related to the creation of AI-generated sexual exploitation material of students in Australian schools. This comes as global AI model hosting platform Hugging Face has also taken key steps to comply with Australian law after warnings from eSafety that certain generative AI models it hosts are being misused by Australians to create AI-generated child sexual exploitation material. Hosting platforms like Hugging Face act like the gatekeepers of distribution of these powerful AI models, much the same way the more traditional app stores do, so it’s equally important to ensure they also have measures in place to protect children. Following engagement from eSafety about compliance concerns, Hugging Face has now changed their terms of service so that all account holders are required take steps to minimise the risks associated with models that they upload, specifically to prevent misuse to generate child sexual exploitation material or pro-terror material. If Hugging Face fails to take appropriate action for a breach of its terms, eSafety could take enforcement action. eSafety has a range of enforcement mechanisms under the Online Safety Act where any company fails to comply with an industry code or standard, including seeking penalties of up to $49.5m. Learn more: https://lnkd.in/gKWv8JjS
Great to see AU leading the way with AI safety. Very interesting times in the world of legal liability and AI - good to see a focus on the developers/platform providers here. Hugging Face reportedly has contracts with tech giants like Nvidia and Amazon. Hopefully the $49.5m penalty remains high enough to ensure compliance, rather than eventually being accepted as a cost of doing business. I also hope the AU access withdrawal they’ve implemented isn’t easily circumvented by VPN use, and there are other detections in play aside from IP geolocation. Genuine action by Hugging Face in this space ideally would show how they seek to address that VPN risk, or call out that they cannot. Thank you for fighting the good fight, and taking strong, proactive action here 💪🏼
Well done eSafety Commissioner ! Massive win
👏👏
Again the eSafety Commission team are proving their worth! Thank you eSafety Commissioner
Well done.
Notable!
Great to see the momentum!
Thank you eSafety Commissioner– a really important outcome. Such a strong reminder that AI platforms must take responsibility to stop their tech being used to harm children.