San Francisco's city attorney is demanding Meta stop running ads featuring AI-generated child sexual abuse content and has issued a cease-and-desist letter demanding an explanation for how these disturbing ads repeatedly appeared on the company's platforms.
The controversial ads, discovered by researchers at the Tech Transparency Project (TTP), transformed images of minors, some identifiable, into short video clips that could depict them in sexual acts. These ads then directed users to download AI image and video generation apps, some capable of creating nonconsensual intimate imagery. In total, over 350 such ads reached more than 29,000 accounts across the EU, US, Australia, and India on platforms like Facebook, Instagram, and Threads.
"Meta has failed to address a known issue of AI-generated child sexually explicit ads, while profiting from them," stated San Francisco City Attorney David Chiu. "These ads are deeply disturbing and cause harm to real children. Meta claims to have zero tolerance for child exploitation, yet the scale and persistence of these issues suggest current efforts are woefully inadequate." Chiu emphasized that no company should allow its advertising systems to be used in such a manner, especially after accepting payment for the distribution.
The letter questions Meta's ad policies and safety systems, seeking prompt discussions with the company's legal, child-safety, and advertising teams. Chiu noted that the persistent nature of these ads suggests more than just an isolated failure. Meta, however, claims the ads were not displayed in San Francisco, placing them outside the city attorney's jurisdiction, though their ad library doesn't specify US ad locations.
Meta has previously stated that many of the discovered ads were removed, and all have now been taken down for violating policies. A spokesperson asserted, "We don’t tolerate nudify apps or any kind of child exploitation, whether real or AI-generated." The total ad spend for over 300 ads was reportedly less than $5,000. The city attorney's office is demanding Meta explain how these ads bypassed moderation and how the company escalates and reports suspected child exploitation to the National Center for Missing and Exploited Children (NCMEC).
The findings indicate "broader systemic problems" with Meta's detection of child sexual abuse in advertising, raising "significant legal questions." The City Attorney’s Office aims to ensure Meta implements systems to prevent children's sexual exploitation through paid advertisements and reliably meets its legal obligations when aware of such issues. TTP research also highlighted that Meta sometimes took over a week to respond to reports of abusive ads, during which time they were viewed by hundreds more people. Chiu has requested a response from Meta within 28 days.
