The social media network X has provoked significant anger after people utilized its artificial intelligence assistant Grok to manipulate images of females by taking off their garments.
Several instances have emerged where the tool has been used to remove clothing from women to make them appear in swimwear without consent, as well as to insert them in inappropriate situations.
Company Response and Legal Position
XAI, the company that created Grok, remained silent to requests for comment, other than an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being introduced to ban nudification technology. Under a upcoming law, anyone who distributed such technology would "receive a jail term and heavy fines".
"The platform could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said platforms must "consider the risk" of people viewing illegal content on their networks, but did not specify whether it was at this time examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost AI assistant—with some paid-for functions—which reacts to X users' queries when they tag it in a message.
It is commonly used to offer commentary or additional context to other users' remarks, but people on X are also capable of edit an posted photo through its artificial intelligence picture modification function.
History of Criticism and Policy
The system has been condemned for enabling users to create pictures and footage with nudity and sexually suggestive content, and it was in the past accused of generating inappropriate material of celebrities.
"X has been permitting the creation and sharing of these images for months without taking any action and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "showing images of individuals in a graphic manner".
In a statement, Ofcom said it was illegal to "create or circulate private pictures or child sexual abuse material" and confirmed this included sexual deepfakes generated with AI.
It said networks such as X were obligated to take "necessary steps" to "minimize the risk" of users encountering illegal content on their services and to take down it promptly when they are notified of it.
