The debate over Grok has sharpened a question that reaches far beyond Elon Musk’s company: who should prevent an artificial intelligence product from becoming a tool for sexual abuse? Holding an individual offender responsible does little for a child whose fabricated image already circulates among classmates. Effective protection also depends on product design, reporting systems and the institutions expected to help victims. The test is whether those systems interrupt harm before it becomes permanent.
The controversy surrounding xAI has sharpened the divide between responsibility for abusive conduct and responsibility for technology that enables it. Those responsibilities need not cancel each other out. A company can prohibit misuse while still facing scrutiny over how reliably its safeguards work. For policymakers, the practical question is which interventions reduce abuse without treating every legitimate use of image generation as suspect.
Evidence from schools shows how far the problem has spread beyond specialist technology debates. The Center for Democracy and Technology found that 15 percent of surveyed American high school students knew of sexual deepfakes depicting someone associated with their school during the 2023 to 2024 school year. Its separate figure of 39 percent concerned intimate imagery shared without consent more broadly, including material that was not generated by AI. Confusing those measures would exaggerate the specific prevalence of deepfakes while obscuring the wider abuse they intensify.
Victimization research adds another dimension. A 2025 Thorn study reported that one in 17 teenage respondents had been targeted by fabricated nude imagery. The research surveyed 1,200 people aged 13 to 20 and documented barriers including shame and fear of not being believed. An image does not need to depict an actual event to damage someone’s relationships, safety or willingness to attend school.
The volume reaching investigators also demands careful interpretation. The National Center for Missing and Exploited Children recorded more than 400,000 reports involving generative AI in 2025, compared with 67,000 in 2024. These are reports, not a count of individual victims or newly created images, and many did not establish precisely how AI was involved. Even with that qualification, the workload shows why platforms need accurate reporting and investigators need better information to identify children at risk.
British monitoring offers a different measure. The Internet Watch Foundation identified 3,443 AI generated child sexual abuse videos in its 2025 work, more than 260 times the number recorded in 2024. Its findings describe material assessed through its operations rather than global prevalence. The organization also calls for product safeguards and action against services designed to facilitate abuse.
Schools cannot solve weaknesses in commercial AI systems, but they can change what happens after a student asks for help. CDT found that only five percent of teachers reported their school providing resources to help victims remove deepfake intimate imagery from platforms. Prevention lessons, confidential reporting and practical assistance therefore deserve attention alongside disciplinary measures. A response that stops at punishing an offender leaves the victim managing the consequences alone.
Original analysis inspired by Miriam Haart from The Jerusalem Post. Additional research and verification conducted through multiple sources.
