TikTok notes that it uses both technology and human moderators to identify rule-breaking content. However, the volume of uploads makes timely moderation difficult. While the racist videos racked up a lot of views, a TikTok spokesperson tells Ars that more than half of the accounts cited in the MediaMatters report were banned for policy violations before the report was published, and the remainder have now been removed.
As for Google, it has a comprehensive Prohibited Use Policy that bans the use of its services to promote hate speech, harassment, bullying, intimidation, and abuse. The videos uncovered by MediaMatters all seem to fall under one or more of these categories. In a perfect world, Veo 3 would refuse to create these videos, but vagueness in the prompt and the AI’s inability to understand the subtleties of racist tropes (i.e., the use of monkeys instead of humans in some videos) make it easy to skirt the rules.
TikTok, being the world’s leading social video behemoth, is a natural place for these videos to spread. It’s not exclusive to TikTok, though. X (formerly Twitter) has gained a reputation for very limited moderation, leading to an explosion of hateful AI content. This problem could also get worse very soon. Google has plans to integrate Veo 3 into YouTube Shorts, which could make it even easier for similar content to spread on YouTube.
TikTok and Google have clear prohibitions on this content, which should have prevented it from being seen millions of times on social media. Enforcement of those policies, however, is lacking. TikTok is seemingly unable to keep up with the flood of video uploads, and Google’s guardrails appear insufficient to block the creation of this content. We’ve reached out to Google to inquire about Veo 3’s safety features but have not yet heard back.
For as long as generative AI has existed, people have used it to create inflammatory and racist content. Google and others always talk about the guardrails to prevent misuse, but they can’t catch everything. The realism of Veo 3 makes it especially attractive for those who want to spread hateful stereotypes. Maybe all the guardrails in the world won’t stop that.