The social media platform X has provoked major outrage after individuals used its artificial intelligence assistant Grok to alter images of females by taking off their clothing.
Several cases have been seen where the technology has been used to remove clothing from women to render them in swimwear non-consensually, as well as to put them in sexualized contexts.
Developer Response and Legal Position
XAI, the organization behind Grok, did not respond to requests for comment, aside from an auto-reply message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being proposed to prohibit nudification software. Under a proposed law, anyone who distributed such technology would "receive a jail term and substantial penalties".
"The platform has the ability to stop these forms of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said platforms must "assess the risk" of people viewing illegal content on their services, but stopped short of confirming whether it was presently looking into X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' prompts when they tag it in a post.
It is commonly used to offer commentary or further background to other posters' remarks, but people on X are also can alter an uploaded image through its artificial intelligence picture modification tool.
History of Controversy and Policy
The tool has been condemned for permitting users to generate pictures and clips with nudity and sexually suggestive material, and it was previously alleged to have been involved in making sexually explicit content of public figures.
"The platform has been permitting the generation and spread of these pictures for a long time without taking any action and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "showing likenesses of individuals in a graphic way".
In a statement, Ofcom said it was illegal to "make or circulate non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images created with AI.
It said platforms such as X were mandated to take "necessary steps" to "reduce the danger" of users coming across illegal material on their platforms and to remove it promptly when they learn of it.
