AI image and video moderation: why Grok tightened its rules

2026-08-07

Maestro’s take: AI media moderation is not a safety switch. It is a product dependency that changes under pressure. Hosted tools make different bargains. Adobe publishes rights controls, explicit-content restrictions, review practices, and provenance support. Google publishes broad restrictions with explicit artistic, documentary, and scientific exceptions. Runway and Fable prohibit explicit sexual content and unauthorized likeness use. Luma’s public moderation guide focuses on explicit content, violence, illegal activity, and harassment. Grok launched with wider adult-content latitude, then added safeguards after its image tools were used for nonconsensual sexual deepfakes at scale. Open-weight models move more control to the builder—and more responsibility with it. The useful distinction is not “censored” versus “open.” It is who absorbs the risk when a model misunderstands the prompt, the policy, or the law.

TL;DR

The rumor that Grok was used for “X-rated videos” is partly true but imprecise. xAI advertised adult sexual content through Spicy mode. The strongest documented later abuse involved mass sexualized image creation and editing of identifiable people.
In January 2026, xAI and X added technical safeguards and access limits. The current acceptable-use policy also expressly prohibits altering a real person into an intimate or sexual context, deceptive impersonation, defamation, child sexualization, and removal of provenance marks.
Other hosted platforms publish different restrictions and exceptions. Their written policies are useful, but they do not predict every prompt-level decision.
Research shows over-refusal is real. It also shows that simply rewriting prompts to pass filters can damage the user’s intent.
Output-related legal pressure—including cases involving nonconsensual likeness use, child sexualization, deception, and potentially infringing generations—gives platforms a strong incentive to restrict uncertain outputs. Training-data lawsuits create a separate pressure on sourcing and licensing.
Builders should treat moderation like any unstable external API: test it, log it, provide fallbacks, and never promise that a hosted model will accept a prompt tomorrow because it accepted it today.

What actually happened with Grok

When xAI launched Grok Imagine in 2025, TechCrunch documented a Spicy mode that could generate sexually suggestive and partially nude imagery. Some prompts were still moderated. The launch nevertheless positioned Grok as comparatively permissive.

That is different from controlling sexual transformations of a real person’s likeness. Canada’s privacy regulator later found that Grok’s image tools could preserve a source face, that X exposed image editing on other users’ posts, and that public replies to @Grok made the feature unusually easy to invoke. The regulator said xAI had not established a workable consent process for identifiable people depicted sexually.

The scale figures require care. The Office of the Privacy Commissioner of Canada cited third-party estimates—not an xAI audit—of more than 6,000 sexualized deepfakes per hour and roughly 1.8 million over an eleven-day period. Another research group estimated about three million. The exact count is uncertain. The abuse pattern is not.

The response came in stages. X activated a crisis protocol, blocked @Grok requests for sexualized images of identifiable people, limited some image generation to paid subscribers, and applied additional protections across X and Grok. xAI’s current acceptable-use policy prohibits altering a real person to appear in an intimate or sexual context, deceptive impersonation, pornographic likenesses, defamatory depictions, child sexualization, and removal of provenance marks.

European and British regulators also opened formal inquiries. The European Commission is investigating whether X adequately assessed and mitigated risks from manipulated sexual images, including possible child sexual-abuse material. Ofcom’s investigation remains ongoing.

The documented sequence supports a clear inference. Wider capability increased the opportunity for use. Public invocation and image editing on X reduced friction. Inadequate safeguards enabled abuse. Public evidence, regulator attention, and litigation raised the cost of permissiveness. The rules tightened. The sources establish the chronology and product design; they do not measure the independent causal weight of each factor.

What the platforms publish

The matrix below compares published policies checked on August 7, 2026. It is not a controlled enforcement benchmark. Product behavior varies by model, account, region, prompt, and policy update.

Product	Fictional adult sexual content	Real-person likeness	Deceptive media	Exceptions and recourse
Adobe Firefly	Pornographic material prohibited	Rights violations and harmful impersonation prohibited	Misleading content that creates real-world harm prohibited	Automated and manual review; Content Credentials support provenance
Google Flow / Veo	Sexually explicit content prohibited	Privacy, biometric, and impersonation harms prohibited	Deliberate deception and misinformation prohibited	Artistic, documentary, scientific, and public-interest exceptions are stated; blocked-query feedback is available
Runway	Nudity and sexually explicit content prohibited	Another person’s image, audio, or video requires permission	Deliberate deception, impersonation, and defamation prohibited	Automated and human review; suspension appeals are documented
Luma	Explicit and NSFW content prohibited	Not clearly specified in the cited moderation guide	Not clearly specified in the cited moderation guide	The guide covers violence, illegal activity, and harassment; appeals are available by email
Fable Showrunner	NSFW content prohibited	Photos or videos of anyone require consent	Harmful uses involving famous or political people prohibited	No broad artistic exception or clear public appeal path is stated
Grok	Current policy does not state a blanket ban on fictional adult content	Altering a real person to appear in an intimate or sexual context is prohibited	Deceptive impersonation, defamation, and false-light depictions prohibited	Generated media carries provenance marks that users may not remove; current equivalent-prompt enforcement was not independently tested

Sources: Adobe, Google, Google Flow, Runway, Luma, Fable Showrunner, and xAI.

The table documents posture, not reliability. A TIME test of Google Veo found an awkward split: the model rejected one fictional hurricane prompt because viewers might mistake it for reality, yet generated other fabricated conflict and election scenes. That is one journalistic test, not a platform-wide score. It still illustrates the product problem: moderation is contextual, probabilistic, and capable of being both too strict and too loose in the same session.

Over-refusal is real

A refusal can be technically defensible and still be absurd in context. TIME’s fictional-hurricane example shows the failure mode: a system can reject harmless fiction while permitting other fabricated scenes with greater potential to mislead.

The pattern has broader research behind it. The OVERT benchmark tested 4,600 benign prompts designed to resemble sensitive requests alongside 1,785 unsafe prompts. It found widespread over-refusal and a strong correlation between safety performance and unnecessary rejection. Prompt rewriting reduced some refusals, but often changed the request.

A separate USENIX Security study examining fourteen generative-AI products found that policies frequently lacked detail about enforcement and appeals. Users reported frustration with both moderation decisions and support.

This is not evidence that safety should disappear. It is evidence that a binary refusal is often the cheapest product design, not the smartest one.

A better system would identify the risky element, explain it, and preserve the harmless intent. “I cannot depict a real person being falsely arrested” is useful. Rejecting clearly fictional parody without offering a safe revision is policy theater.

Self-hosting changes who moderates

Hosted platforms operate the policy layer for you. Open-weight models let builders operate it themselves.

Wan 2.2, for example, publishes model weights and inference code under Apache 2.0. Its repository makes users responsible for lawful use and prohibits harmful applications. There is no remote vendor deciding whether one particular prompt should run when the model is deployed on infrastructure you control.

That freedom is real. So are the obligations. The builder must handle consent, identity abuse, age safeguards, logging, provenance, takedowns, security, and jurisdictional rules. Self-hosting removes one refusal layer. It does not remove the law or the blast radius.

The law is moving toward provenance

Since August 2, 2026, the EU AI Act’s Article 50 transparency rules apply to synthetic media. Providers must machine-mark generated or manipulated content, while deployers must disclose deepfakes. The Commission’s current guidance notes a grace period until December 2026 for the marking obligation on generative systems placed on the market before August 2.

Provenance helps, but it is not a truth machine. C2PA Content Credentials can record origin, edits, and AI involvement in a tamper-evident chain. They do not prove that the depicted event happened or that every omitted claim is false.

The practical future is layered: generation controls, consent checks, provenance, distribution rules, and human judgment. Anyone selling a single perfect filter has found a lovely beach property that may also be underwater.

Lawsuits are shaping the incentives

The cases are early, but their direction matters.

A proposed class action reported by Bloomberg Law alleges that Grok enabled nonconsensual sexual deepfakes. A separate federal complaint filed by three teenage plaintiffs alleges sexual exploitation through generated images and videos. Both are allegations in pending litigation.

xAI has also sued an alleged user. According to Reuters, the company alleges that Terry Harwood violated its terms by creating explicit deepfakes and child sexual-abuse material. That case is also pending.

These output-related cases increase the expected cost of permissive generation. In the output cases reviewed here, the pressure concentrates on nonconsensual likeness use, child safety, and harmful sexual content. Platforms therefore have a rational incentive to reject ambiguous requests, even when users bear the product cost.

Copyright disputes create a related but separate risk. Runway faces a complaint alleging unauthorized YouTube scraping and circumvention of platform protections. Disney and Universal have sued Midjourney over alleged infringement, as reported by AP. These remain disputed claims, not final findings.

Training-data cases pressure sourcing, licensing, and model-development practices. Cases involving allegedly infringing outputs may also encourage narrower generation policies, but that is Maestro’s inference; the cited litigation does not establish a resulting moderation change.

Why Maestro users care

If your workflow depends on a hosted image or video model, moderation is part of your runtime.

Treat it accordingly:

Test representative prompt sets across providers, including benign prompts that contain sensitive-looking words.
Store the model, policy version, prompt, refusal, and timestamp. “It worked last month” is not a reproducible specification.
Separate fictional-character workflows from real-person likeness workflows. Require explicit consent evidence for the latter.
Provide a fallback path: a different hosted provider, a human review queue, or a self-hosted model with your own controls.
Preserve provenance metadata and label synthetic media at publication.
Give users an appeal or correction path. A refusal without recourse is not moderation. It is abandonment.

One practical action

Build a twenty-prompt moderation regression suite this week.

Include ordinary commercial requests, fictional news parody, historical reenactment, public figures, consenting adults, minors, violence, medical scenes, copyrighted characters, and a deliberately deceptive request. Run it against every model your product depends on. Save the outputs and refusals. Repeat after every provider policy or model update.

The goal is not to find the “uncensored” platform. The goal is to know which promises your product can safely make.

Sources considered

Disclosure: This article is AI-generated and was reviewed by an independent AI editor.