In an era where AI can "undress," can the law keep up? — The Ripple Effects of Grok's Shutdown

"World's First Grok Blockade" Indonesia Shows a Red Card to "Non-Consensual Deepfakes"

The Indonesian government temporarily blocked access to Elon Musk's AI chatbot "Grok." The reason is clear: concerns that Grok could be misused to generate and disseminate non-consensual sexual deepfakes, such as "undressing" or "sexually altering" photos of others. The government has identified this as a "serious issue that violates human rights and dignity, as well as the safety of citizens in the digital space," and has summoned X representatives for an explanation.

This move is noteworthy not just because "a country blocked a specific service," but because it indicates that "non-consensual sexual alterations" by generative AI have now erupted as a cross-border social issue, prompting regulators to judge it as "dangerous if left unchecked." In Europe, authorities are also increasing scrutiny, asking X to preserve documents and data related to Grok.

What Happened: The Ability to "Alter Photos" Quickly Turns into "Public Harm"

The issue at hand is not so much about the ability to "generate" but rather the fact that the path to generation is extremely short, and the results are easily disseminated in public spaces.

According to reports, Grok is used on X for image generation and editing, with users requesting alterations such as removing clothing from others' photos, and the generated content being publicly shared as replies. This structure allows for sexual alterations without the person's consent, circulating on third-party timelines as a sort of "spectacle."

The Indonesian minister strongly criticized such non-consensual deepfakes, highlighting the risk of harm to socially vulnerable groups, including women and minors. Domestic laws also impose strict regulations on the distribution of content considered obscene or pornographic.

Corporate Response: xAI Advances "Restrictions," but Criticism Persists

xAI, the company developing Grok, reportedly moved to restrict image generation and editing functions, limiting at least some features to paid users as the issue escalated.

Meanwhile, from the perspective of authorities and victims, there is a tendency to push back against the notion that "it's the user's fault," as this does not stop the harm. In fact, Malaysian authorities have deemed the response reliant on user reports as insufficient and intend to continue restrictions until effective technical and operational safety measures are in place.

Musk reportedly stated on X that those who create illegal content would face "the same consequences as those who upload illegal content." However, considering the speed of occurrence and scale of dissemination of harm, "post-facto punishment" alone cannot keep up with the need for remedy, which is the focus of much criticism.

Reactions from Various Countries and Sectors: Europe, Scandinavia, and Southeast Asia

Indonesia's blockade is part of an international trend. In Europe, oversight concerning Grok is intensifying, and in Sweden, it was reported that politicians were targeted, prompting the government leader to condemn it as "a form of sexual violence."

In Southeast Asia, following Indonesia, Malaysia also temporarily restricted access to Grok, focusing on the repeated generation of "non-consensual altered images" involving women and minors.

This chain of events signifies that the question is not only "how to stop AI functions," but also "whether platform design can be structured to minimize harm." For example, the design allowing generated content to be spread as public replies acts as a "spread button" for misuse. Regulators are beginning to emphasize "design and operational responsibility" for this reason.

Reactions on Social Media: Divided Opinions on "Regulation or Censorship"

Reactions on social media have split into three major trends.

1) "Blocking is Justified" Group: Emphasizing the Severity of Harm

On platforms like Reddit, there are strong expressions supporting the blockade, such as "this 'criminal app' should be blocked in all countries."
Additionally, foreign reports have analyzed that posts "transforming women's photos into swimwear" surged from the end of the year to the beginning of the year, rapidly intensifying. As victims' voices became visible, the sentiment that "this is not an issue to be left unaddressed" quickly gained momentum.

2) "It's a Strict Country Anyway" Group: Questioning the Consistency and Motives of Regulation

Conversely, there is a more skeptical view that "Indonesia is already strict on pornography," and that the current response is more an extension of existing regulatory policies than a "progressive judgment on a new issue." Comments on Reddit also discuss the context of "a country where pornography is completely banned."

3) "Ad-Hoc, Symptomatic Treatment" Group: Criticism That It Does Not Lead to Fundamental Solutions

Furthermore, regarding xAI's move to "restrict to paid features," there is skepticism that "it's more about monetization than safety measures," and that "technical guardrails are needed to stop the harm." The Malaysian authorities' judgment that a response centered on reports is "insufficient" is continuous with such criticism.

The "Reality" Happening in Indonesia: Idol Management Warns, Law Enforcement in View

In Indonesia, authorities have suggested "not only administrative sanctions but also the possibility of criminal liability." Local media reported that individuals and companies misusing the technology could be subject to punishment, and victims are encouraged to report. Additionally, it has been reported that the management of a popular local idol group warned of potential legal action against "digital undressing" of its members. The fact that social "stakeholders" are beginning to speak out directly is noteworthy.

Future Focus: It Won't End with Just "Blocking"

Indonesia's decision may be effective as a short-term "firefighting" measure. However, in the long term, it raises the following questions.

• Can platforms be designed so that non-consensual sexual alterations cannot be "created/disseminated"?

• How will regulators differentiate the use of measures such as deletion requests, data preservation orders, and blockades?

• To what extent can victim relief (deletion, dissemination prevention, legal support, psychological support) be institutionalized?

The complexity of this issue lies in the fact that while "technology" evolves at an accelerating pace, "social consensus and legal development" struggle to keep up. Generative AI can inherently provide powerful support for creativity and business. However, if it becomes optimized in a direction where **"the dignity of others can be trampled at zero cost,"** society will inevitably push back, and regulation will intensify.

Indonesia's Grok blockade is a symbolic event indicating this turning point. The next question is whether companies can incorporate safety into their design philosophy rather than resorting to "superficial measures after a backlash"—and whether countries can create effective rules that stand on the side of victims.