Supervisor Agent Deep Dive: The Audit System Preventing AI Runaway
Refund hallucinations, inventory misrecognition, inappropriate promises—revealing how we prevent these in real-time.
"We sincerely apologize. We'll issue a full refund."
It happened two weeks after deployment. The AI at a ramen shop gave this response while handling a complaint. The problem? This store had never granted the AI authority to issue "full refunds." Besides, a full refund for a bowl of ramen wasn't a normal response in any scenario.
When the store manager called us, our blood ran cold.
AI "Lies"
To be precise, AI doesn't actually lie. But it can speak with supreme confidence about things it doesn't actually know. The industry calls this "hallucination."
Promising discounts that don't exist. Recommending sold-out items. Ignoring store policies entirely—these are all problems caused by AI "hallucinations."
After the ramen shop incident, we reached a conclusion: we can't fully trust AI. But not using AI is no longer an option either.
So we needed a system to "supervise" the AI.
Building the Watchdog
What we built is the "Supervisor Agent"—a monitoring system that checks AI output in real-time before it reaches customers.
The concept is simple. Every response the AI generates must pass multiple validations before being sent. Is the amount correct? Does it contradict inventory status? Does it exceed the permissions granted to the AI?
If there's even one problem, that response gets blocked.
The 0.3-Second Battle
However, validation takes time. We can't keep customers waiting too long.
We solved this by running validations in parallel. Amount validation, inventory validation, permission validation, personal information validation—all execute simultaneously, completing in an average of 0.3 seconds.
For customers, there's virtually no wait. But behind the scenes, four different validations are running during that 0.3 seconds.
The Courage to Stop, The Wisdom to Escalate
Not everything can be automated. When Supervisor determines something is "dangerous," the AI holds its response and escalates to a human operator.
"We apologize, let me check with our manager. Please wait a moment."—this single sentence prevents major problems from occurring.
Since the ramen shop incident, Supervisor has blocked an average of 12 "dangerous responses" per day. It may seem like a small number, but each one was a potential threat to the store's reputation.
Unleashing AI's power while preventing it from running wild. Achieving this balance is exactly what we're aiming for.