Elon Musk's social media network X has sparked widespread backlash after individuals utilized its AI chatbot Grok to change images of women by removing their garments.
Several instances have been seen where the tool has been used to remove clothing from women to show them in bikinis without permission, as well as to insert them in sexually explicit situations.
Company Response and Legal Position
XAI, the company that created Grok, offered no comment to inquiries, other than an automatically-generated message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being proposed to ban nudification software. Under a proposed law, anyone who supplied such technology would "receive a prison sentence and heavy fines".
"The platform has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "evaluate the danger" of people viewing unlawful material on their platforms, but did not specify whether it was at this time examining X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some additional features—which responds to X users' requests when they reference it in a message.
It is often used to offer reactions or additional background to other users' remarks, but people on X are also capable of edit an uploaded image through its AI image editing function.
Background of Controversy and Policy
The chatbot has been criticised for enabling users to create images and clips with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in making inappropriate content of celebrities.
"The platform has been permitting the production and spread of these pictures for a long time without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "depicting likenesses of people in a pornographic fashion".
In a statement, Ofcom said it was illegal to "generate or share private pictures or illegal content" and clarified this included explicit AI images generated with AI.
It said platforms such as X were obligated to take "appropriate measures" to "minimize the danger" of users viewing illegal content on their services and to delete it quickly when they become aware of it.
