Deloitte sounds alarm as AI agent deployment outruns safety frameworks
Poor governance is the threat
Deloitte is not highlighting AI agents as inherently dangerous, but states the real risks are associated with poor context and weak governance. If agents operate as their own entities, their decisions and actions can easily become opaque. Without robust governance, it becomes difficult to manage and almost impossible to insure against mistakes.
According to Ali Sarrafi, CEO & Founder of Kovant, the answer is governed autonomy. “Well-designed agents with clear boundaries, policies and definitions managed the same way as an enterprise manages any worker can move fast on low-risk work inside clear guardrails, but escalate to humans when actions cross defined risk thresholds.”
“With detailed action logs, observability, and human gatekeeping for high-impact decisions, agents stop being mysterious bots and become systems you can inspect, audit, and trust.”
As Deloitte’s report suggests, AI agent adoption is set to accelerate in the coming years, and only the companies that deploy the technology with visibility and control will hold the upper hand over competitors, not those who deploy them quickest.
AI agents may perform well in controlled demos, but they struggle in real-world business settings where systems can be fragmented and data may be inconsistent.
Sarrafi commented on the unpredictable nature of AI agents in these scenarios. “When an agent is given too much context or scope at once, it becomes prone to hallucinations and unpredictable behaviour.”
“By contrast, production-grade systems limit the decision and context scope that models work with. They decompose operations into narrower, focused tasks for individual agents, making behaviour more predictable and easier to control. This structure also enables traceability and intervention, so failures can be detected early and escalated appropriately rather than causing cascading errors.”
With agents taking real actions in business systems, such as keeping detailed action logs, risk and compliance are viewed differently. With every action recorded, agents’ activities become clear and evaluable, letting organisations inspect actions in detail.
Such transparency is crucial for insurers, who are reluctant to cover opaque AI systems. This level of detail helps insurers understand what agents have done, and the controls involved, thus making it easier to assess risk. With human oversight for risk-critical actions and auditable, replayable workflows, organisations can produce systems that are more manageable for risk assessment.
Shared standards, like those being developed by the Agentic AI Foundation (AAIF), help businesses to integrate different agent systems, but current standardisation efforts focus on what is simplest to build, not what larger organisations need to operate agentic systems safely.
Sarrafi says enterprises require standards that support operation control, and which include, “access permissions, approval workflows for high-impact actions, and auditable logs and observability, so teams can monitor behaviour, investigate incidents, and prove compliance.”