The social media network X has caused widespread backlash after individuals utilized its AI chatbot Grok to alter photos of women by taking off their attire.
Several examples have emerged where the system has been employed to remove clothing from female subjects to make them appear in swimwear without permission, as well as to put them in sexualized situations.
Company Reaction and Legal Stance
XAI, the organization that created Grok, did not respond to inquiries, other than an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that legislation is being proposed to prohibit image stripping technology. Under a new criminal offence, anyone who provided such tools would "receive a prison sentence and significant penalties".
"The platform is capable of halting these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "consider the potential" of people viewing unlawful material on their networks, but did not specify whether it was at this time looking into X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some additional functions—which responds to X users' requests when they tag it in a message.
It is often used to offer reactions or additional background to other users' comments, but people on X are also capable of modify an uploaded picture through its AI image editing feature.
History of Controversy and Policy
The chatbot has been condemned for permitting users to produce pictures and footage with explicit content and sexually suggestive material, and it was previously accused of making sexually explicit content of celebrities.
"The platform has been enabling the production and distribution of these pictures for months without taking any measures and we have yet to see any action by oversight bodies," the expert added.
XAI's own terms of service forbids "depicting images of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "make or share private pictures or illegal content" and clarified this covered explicit AI images generated with AI.
It said services such as X were required to take "necessary actions" to "lessen the risk" of users coming across illegal content on their services and to remove it swiftly when they become aware of it.
