Can This Former Facebook Expert Make AI Safer for Everyone

The traditional way we try to keep harmful content off the internet was, surprisingly, barely better than a coin toss. That's the startling insight from Brett Levenson, who once led business integrity at Facebook. He discovered that human content reviewers, tasked with memorizing a huge 40-page policy document and making lightning-fast decisions in just 30 seconds per item, were often only slightly better than 50 percent accurate. This meant harmful content could slip through or linger online for days, causing real damage before anyone could react.

This slow, reactive approach, which simply couldn't keep up with sophisticated bad actors, became even more problematic with the explosion of artificial intelligence. Suddenly, companies weren't just dealing with user posts, but with AI chatbots giving dangerous self-harm guidance to teens or AI image generators creating nonconsensual deepfakes that evade existing safety filters. The scale and speed of AI-generated content made the old moderation methods completely unsustainable.

Frustrated by these limitations and seeing a clear need for a new solution, Levenson came up with an innovative idea: "policy as code." This means transforming those complex, static policy documents into automated, executable rules that AI systems can understand and enforce instantly. This breakthrough vision led him to found Moonbounce, a startup that just announced it has successfully raised $12 million in new funding from investors like Amplify Partners and StepStone Group to bring this vision to life.

Moonbounce works by providing an extra layer of real-time safety wherever content is generated online, whether it is from a human user or an advanced AI. The company has trained its own large language model to quickly process a customer's specific policy documents. It then evaluates content in less than 300 milliseconds, allowing it to take immediate action, such as slowing down distribution for human review or blocking high-risk content on the spot. This proactive approach helps protect users across various platforms from potentially damaging content.

Brett Levenson's journey into content moderation began at a challenging time, joining Facebook in 2019 amidst the fallout from the Cambridge Analytica data scandal. His initial belief that better technology alone could fix Facebook's moderation problems quickly changed. He realized the core issue was a human system overwhelmed by volume, complexity, and the sheer impossibility of consistent policy application by individual reviewers. This systemic vulnerability meant that even well-intentioned efforts were often too late and ineffective.

The problem, already immense, then took an alarming turn with the rapid advancement of AI. New AI chatbots and image generators, while powerful, quickly demonstrated their capacity for misuse. High-profile incidents emerged where AI provided harmful advice or generated highly inappropriate imagery, highlighting the urgent need for robust "guardrails." Companies found themselves facing significant legal and reputational risks, struggling to control content generated by their own sophisticated machines.

In this context, Moonbounce steps forward as a potential game-changer. It offers a specialized solution to a problem that has historically plagued large online platforms and is now magnified by AI. Unlike internal systems that might be bogged down by broader context, Moonbounce acts as a third party, solely focused on enforcing rules at a crucial point: the moment content is created or generated. This external, objective approach is what many AI companies are now seeking, recognizing their internal defenses are often insufficient.

This new development from Moonbounce could have a very real and positive impact on your everyday online life. For anyone interacting with AI companions, using dating apps, or experimenting with AI image generators, this technology is designed to make those experiences much safer. It means a significant reduction in dangerous or inappropriate content reaching you, offering peace of mind and building trust in the digital spaces where we increasingly spend our time. This shift towards proactive, real-time moderation directly addresses some of the most pressing safety concerns of our modern internet.

Looking at the bigger picture, Moonbounce is helping to define what responsible AI development looks like. Many AI companies are under intense public and legal pressure to ensure their products do not cause harm. By providing an effective, third-party safety layer, Moonbounce allows these companies to deploy powerful AI tools with greater confidence. As Brett Levenson puts it, safety can actually become a "product benefit," a differentiator that helps companies stand out and earn user loyalty, rather than just a costly afterthought. This approach could set a new standard for AI integration across various industries.

However, as Moonbounce develops features like "iterative steering," where AI actively attempts to guide conversations in a "better direction," it brings up an important debate. While the intention is to prevent harm and offer support, particularly in sensitive situations, it also raises questions about the degree of control AI should have over our interactions. Is it always appropriate for an AI to modify our prompts or redirect a conversation, even if it's for our presumed benefit? This delicate balance between protective intervention and user autonomy is a conversation we all need to have as AI technology advances.

Moonbounce's next major step involves refining its "iterative steering" capability, which aims to move beyond simply blocking harmful content. This feature would actively redirect AI conversations, modifying prompts in real-time to guide chatbots towards more supportive responses instead of just a blunt refusal when sensitive topics arise. The coming months will show how widely this more proactive, guiding approach to AI safety is adopted across the tech industry, and whether it sets a new precedent for how companies handle user interactions with their AI. We should also watch to see if this external, "policy as code" model becomes an essential component for AI platforms striving for safer, more responsible development.

Should AI systems be designed to proactively "steer" conversations away from harmful topics, or should they primarily focus on blocking problematic content and allow users more freedom to navigate potentially risky discussions?

What ethical considerations do you think are most important when an AI system is given the power to "guide" human conversations, even with good intentions?

---

Filed under: AISafety, ContentModeration, Moonbounce, TechInnovation, DigitalTrust