The social media network X has sparked major outrage after people employed its artificial intelligence assistant Grok to alter images of females by stripping their clothing.
Several cases have surfaced where the system has been employed to digitally undress women to show them in swimwear without permission, as well as to put them in sexually explicit situations.
Developer Reaction and Legal Position
XAI, the company responsible for Grok, did not respond to inquiries, except for an automatically-generated message stating "legacy media lies".
A government official stated that new laws is being proposed to prohibit nudification tools. Under a proposed criminal offence, anyone who provided such technology would "receive a prison sentence and heavy penalties".
"X or Grok could prevent these forms of misuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the danger" of people accessing unlawful material on their networks, but stopped short of confirming whether it was currently looking into X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some premium functions—which responds to X users' prompts when they reference it in a tweet.
It is frequently used to offer commentary or further background to other users' remarks, but people on X are also able to edit an uploaded picture through its AI image editing function.
History of Criticism and Policy
The tool has been condemned for allowing users to create photos and footage with explicit content and sexualised content, and it was previously alleged to have been involved in making inappropriate content of public figures.
"The platform has been permitting the generation and sharing of these images for months without taking any measures and we have not witnessed any action by oversight bodies," the expert added.
XAI's own terms of service prohibits "showing images of people in a pornographic fashion".
In a statement, Ofcom said it was against the law to "create or circulate deepfakes or illegal content" and clarified this applied to explicit AI images created with AI.
It said services such as X were obligated to take "suitable measures" to "minimize the risk" of users encountering illegal material on their services and to remove it swiftly when they are notified of it.
