A support agent is asked to cancel a customer's subscription. The AI finds the right tool and makes the call. The prototype looks impressive. Now ask a less comfortable question: under whose authority did that cancellation happen?
If the agent used a service account with broad permissions, the tool may not know whether the person making the request was allowed to cancel that subscription. It may not even know which person initiated the action.
We often discuss agent permissions as if the problem were keeping the model obedient. The deeper problem is identity and authorization. The same questions apply to a normal application; AI simply makes it easier to forget them because the interface is conversational.
The demo account becomes the production risk
During a prototype, one powerful API key saves time. The team can demonstrate several workflows without building a full permissions model.
But a credential that can read every customer and modify every account creates an unnecessarily large failure domain. If the agent selects the wrong record, misunderstands the request or follows a malicious instruction found in a document, the backend may still accept the action.
Nothing in that scenario requires the model to be particularly bad. The integration itself supplied authority that the current task did not justify.
Separate the user's rights from the agent's rights
The first decision is whether the agent is acting for a user, for a team or as a narrowly defined automated process. Those are not interchangeable identities.
An agent assisting an employee should generally operate within that employee's permissions. If the employee cannot read a confidential contract or approve a refund, speaking to an AI should not create a new route around those limits.
Some workflows run without a person making every request. They still need an owner and a dedicated role with explicit scope. “The AI service account” is not a useful answer if it covers everything from reading documents to changing payment details.
Authorization belongs in the service that executes the operation. The model can propose a tool call, but the tool must decide whether that call is permitted.
Give a tool a job, not a blank cheque
Consider a tool named manage_customer that can read profiles, edit contact details, cancel contracts and issue credits. The interface may be convenient for developers, but it makes safe boundaries difficult to express.
Smaller operations can be easier to govern: read a permitted account, draft a change, submit a specific request for approval. For an action that changes state, the application should check the record, tenant, operation and relevant limits before executing it.
Short-lived, scoped credentials can reduce exposure when the underlying system supports them. Where it does not, a constrained backend can enforce permissions before talking to an older or less capable API. The model should not see broad credentials at all.
The related principle is least privilege. In AI systems it matters because the component selecting an action can be mistaken even when its tool call is syntactically valid.
Approval is not a substitute for authorization
Requiring human approval for an expensive or irreversible action can be sensible. But approving a transaction does not prove that the agent was allowed to retrieve the underlying data in the first place.
Read access, write access and approval are separate decisions. A manager may be allowed to approve a refund without being able to browse every customer's private information. A support agent may be allowed to prepare the request but not finalize it.
I would explicitly document that chain: who requested the action, which identity was used, what the agent proposed, which policy allowed or blocked it, and who confirmed it if confirmation was required.
This is also where AI Guardrails and tool permissions meet. Guardrails describe the wider operating boundaries; authorization enforces the specific right to perform each action.
Design for the wrong record, not just the wrong answer
The nastiest failure in an agent workflow is often mundane. The assistant chooses the wrong invoice, user or order and makes a valid API call against it.
Before a destructive action, the system may need to show the selected record and its key identifiers to a person. It should reject ambiguous matches rather than guessing. Where possible, there should be a way to reverse the operation or trace its consequences.
Logs need to record actual tool execution, not only the model's conversation. “I have updated the account” is just text; an API response and an auditable change are evidence that something happened.
When scoping permissions starts to feel inconvenient, that is often useful information. It means the design is finally confronting the amount of authority we were about to give away for the sake of a smooth demo.