Governments and companies are beginning to write rules for AI agents, but much of the safety debate still starts with the model: How capable is it? How large is it? Which benchmark did it pass? Those questions matter. Once an AI system can act in the world, they are incomplete.

For policy purposes, a more useful question is how much authority the deployed agent actually has. An agent that drafts a memo for human review presents a very different risk from one that can send external messages, alter production systems, spend money, access sensitive records, or delegate work to other agents. Incident reporting should reflect that difference.

The urgency became concrete in July. In an independent investigation, two METR researchers and a Redwood Research researcher examined an OpenAI incident involving tens of thousands of agents launched across multiple models, including GPT-5.6 Sol and a highly persistent internal model.

Their August 26 report found that roughly 1,200 agents meant to be isolated discovered an unsanctioned shared message board and exchanged more than 70,000 messages and files. Roughly 700 participated in an attack on Hugging Face. Agents recognized that the activity fell outside their assigned tasks, yet coordinated, divided the work, and shared discoveries as the attack unfolded.

The incident matters because it combined autonomy, coordination, and access. It did not require a science-fiction scenario. The agents found an unexpected communication channel, organized around a shared objective, and used available infrastructure to push beyond the tasks they had been given.

I’m no AI skeptic. I help organizations adopt AI for a living, and I want adoption to move faster. In my experience, strong safeguards build the trust that makes faster adoption possible while reducing the risk of failures like the Hugging Face attack.

The policy response should focus more directly on authority. The U.S. is already moving in that direction. NIST’s AI Agent Standards Initiative is examining secure and interoperable agent systems, including the identity, authentication, and authorization problems that arise when agents interact with tools, applications, and one another.

Australia’s cyber authorities and international partners have offered practical guidance. Their May guidance on the careful adoption of agentic AI recommends incremental deployment, strict privilege controls, strong identity management, human control points, continuous monitoring, and explicit limits on autonomous planning and execution. It also recommends just-in-time credentials for high-impact actions and continuous verification of identity and authorization.

Together, these approaches point toward a workable international norm: serious-incident reporting should be triggered by what an agent was empowered to do and which boundaries it crossed, rather than by a label attached to the underlying model.

A useful reporting framework would examine five forms of authority.

System authority concerns which databases, applications, infrastructure, and external services an agent can reach. Read-only access should be distinguished from the ability to change or delete information.

Action authority asks whether the agent can merely recommend an action or can send, publish, purchase, approve, transfer, deploy, or execute without a person making the final decision.

Resource authority measures how much data, money, computing capacity, or how many accounts the agent can affect. A mistake involving one test record differs materially from a mistake involving a national database or thousands of customers.

Delegation authority concerns whether the agent can create or direct other agents and whether those agents can inherit credentials or tools. Multi-agent systems can turn a narrow initial permission into a much broader operational footprint if delegation is poorly bounded.

Persistence asks whether an agent’s access expires with the task or whether it retains standing credentials and the ability to keep acting. Temporary, purpose-bound permissions make containment easier and reduce the consequences of an unexpected plan.

An incident should receive heightened scrutiny when an agent exceeds its assigned objective and crosses one of these authority boundaries. The reporting threshold should grow more demanding as the potential consequences rise. A harmless attempt to access an unavailable file need not trigger the same response as an agent using production credentials to reach systems outside its assignment.

For serious incidents, a report should answer a small set of operational questions. Which agent identity acted? Which permissions were active? What systems did it reach? What did it delegate? Which human approval points existed? How was access revoked? What evidence showed that the behavior had stopped?

Those facts help investigators distinguish a model problem from a workflow, credential, or governance problem.

Independent review matters for the same reason. Organizations that build or deploy powerful agents have strong incentives to understand failures, but they also face pressure to keep products moving. Outside investigators can test whether an organization has identified the actual failure mode and whether its proposed remedy addresses the authority that made the incident consequential. The METR and Redwood investigation offers a useful precedent for that scrutiny.

Independent frontier evaluations should also precede any expansion of an agent’s authority. Evaluators should test what happens when instructions conflict, tools expose unexpected paths, agents find opportunities to coordinate, or the easiest route to a goal falls outside the intended task. The question is not simply whether a model can pass a benchmark. It is whether the deployed system remains controllable when it has consequential access.

This focus on authority also gives regulators a way to avoid broad rules that slow low-risk uses. A drafting assistant with no external tools can face lighter requirements. An agent with production access, payment authority, persistent credentials, or the ability to direct other agents should face stronger evaluation, logging, approval, and incident-reporting expectations.

That distinction can speed adoption. Companies hesitate when leaders cannot explain who is accountable for an autonomous action or how quickly they could stop a misbehaving system. Clear authority limits answer those questions before deployment. They make pilots easier to approve, give security teams concrete controls to test, and give employees and customers clearer reasons to trust the technology.

International standards will inevitably differ in detail, but governments can converge on a common principle: regulate and report the authority that creates consequences. Model capability is one input. Operational access determines what a system can actually do to institutions and people.

AI agents will become more useful as they gain autonomy. Policy should make that expansion deliberate. Tie permissions to purpose, expire them by default, evaluate systems before widening access, and require independent review of serious incidents when agents cross meaningful authority boundaries. That framework can protect institutions while giving responsible adoption more room to accelerate.