The UK regulator Ofcom has initiated an investigation into the platform X, owned by Elon Musk, due to sexualized images generated by its AI tool Grok, reports the BBC.
In an official statement, Ofcom mentions "extremely concerning reports" that the chatbot is being used to create and distribute images of people without clothing, as well as "sexualized images of children."
If it is found that the law has been violated, Ofcom has the authority to impose a fine on X (formerly known as Twitter) of up to 10% of its global turnover or £18 million (approximately €21 million), depending on which amount is higher.
If X fails to comply with the regulator's requirements, Ofcom may take legal action to block access to the platform for internet service providers in the UK.
Last week, UK Prime Minister Keir Starmer commented on the situation with Grok during a broadcast on Greatest Hits radio: "It's a disgrace. It's disgusting. And it's unacceptable. X must take control of the situation," he stated, promising to take action.
Liz Kendall, Minister of Science, Innovation and Technology, is expected to make a statement regarding Grok in the House of Commons on Monday. Her predecessor, now Minister of Trade Peter Kyle, noted on BBC Breakfast that Grok, in his opinion, had not undergone proper testing and expressed outrage about it.
“I met a woman who discovered that AI had generated her image in a bikini against the backdrop of Auschwitz, and that makes me feel sick,” he reported.
The BBC has reviewed several examples of such images where women were "undressed" without their consent and "posed" in sexual positions. One woman reported that over 100 sexualized images featuring her had been created.
As Bloomberg reported on January 7, Grok generated about 6,700 sexualized images per hour from January 5 to 6.
Elon Musk, the owner of X, stated that users of Grok who create illegal content will face the same consequences as those who publish it. At the same time, in response to a question about why other AI platforms are not being scrutinized, he wrote that the UK government is looking for "any excuse for censorship."
Musk also posted an image of a toaster in a bikini with comments that Grok can "dress" anything, and shared photos of Starmer in a women's swimsuit, allegedly generated by GPT and Gemini chats, questioning why Starmer is so concerned about Grok when other neural networks can also create similar images. He added that "they just want to restrict free speech."
After a US immigration officer shot a woman during protests in Minneapolis, users of X used Grok to "undress" an old photo of the victim and her new photo after the shooting. When AFP attempted to get a comment from xAI, the response was brief and automatic: "Traditional media lies."
The official X Safety account states that illegal content, including material related to sexual violence against children, is removed, and accounts distributing such content are restricted. Those who use Grok to create such images will face the same consequences as those who post them. Elon Musk shared this message.
This week, Ofcom reported that it had "urgently contacted X and xAI" regarding the Grok feature, warning that it may initiate an investigation depending on their response. On Friday, an Ofcom representative stated that the regulator had "received a response" and is now "conducting an accelerated assessment urgently."
Ofcom aims to determine whether X violated the law by not removing illegal content quickly enough (as Ashley St. Clair, the mother of one of Musk's sons, complained to the Guardian that a "nude" photo of her 14-year-old daughter was not removed for over 12 hours, despite the ability to contact X staff directly) and whether "appropriate measures" were taken to prevent access to this content for users in the UK. The regulator emphasizes that such content includes "intimate images created without consent," as well as sexual material involving children.
An Ofcom representative did not specify how long the investigation would take but noted that it would be of "the highest priority."
"Platforms are obligated to protect people in the UK from illegal content," Ofcom stated. "We will not hesitate to conduct an investigation if we suspect that companies are failing in their duties, especially when it comes to risks to children."
On Thursday, Grok announced that the image generation and editing feature is now "available only to paid subscribers," however, an official representative of the British Prime Minister noted that this merely turns the ability to create illegal images into a paid service and does not solve the problem.
"This is an insult to the victims of misogyny and sexual violence," the Starmer office added.
Thomas Reynier, a representative of the European Commission, stated: "This does not change our fundamental position—whether it's a paid subscription or free. We do not want to see such images. It's very simple."
"We demand that platforms ensure that their devices and systems cannot generate such illegal content," he added in an interview.
The European Commission is also investigating cases of the publication and creation of explicit images of minors generated by Grok after the introduction of a paid feature last summer known as Spicy Mode.
"This is not 'spicy.' This is illegal. This is outrageous. This is disgusting. There is no place for this in Europe," emphasized the European Commission representative.
Cliff Steinhauer from the non-profit organization National Cybersecurity Alliance noted that access to image generation tools in Grok for paid subscribers may reduce the number of abuses but will not eliminate the security issues that led to the emergence of non-consensual sexualized content.
"Access restrictions alone are not enough to ensure comprehensive protection, as malicious actors can still find workarounds. Real user protection must be built into the foundation of creating and managing such tools," he commented to AFP.
Recently, Grok was blocked in Malaysia and Indonesia, and France and India have also stated their intention to take action. This is not the first scandal involving Grok: in July, the chatbot praised Adolf Hitler, calling itself MechaHitler, and left anti-Semitic comments in response to user queries. At that time, xAI stated that it would block such statements before they were published.
