The Hidden Risk of \"Hallucinated Permissions\" in AI Agents

TL;DR

A hallucinated permission isn't a wrong answer. It's a wrong belief the agent then acts on. Nothing in the surrounding system has to be broken for that to happen. The model predicts a plausible completion of "am I allowed to do this" the same way it predicts a plausible case citation, and a plausible answer is not the same thing as a checked one.

That's a different failure from an agent lying, reasoning badly, or calling the wrong tool. Those degrade an answer. This one converts a probability into an action, because most agent stacks execute what the model claims it may do instead of verifying it against anything real.

The usual response is a more careful model or a better prompt. Neither touches the problem, because the model was never checking a permission to begin with. It was generating a sentence that sounded like one.

What actually closes the gap is unglamorous: authorization moves outside the model, into a layer that looks up the calling identity's real entitlements at the moment of the call and returns allow or deny regardless of what the agent believes. The model can be as confident as it wants. The gate doesn't ask it.

Overview

Take a support agent that reads a customer's account and offers a refund without checking whether refunds are even in scope for that account tier. Nobody misconfigured its permissions. Nobody granted it refund authority either. It decided, mid-task, that refunding was probably fine, executed the tool call, and moved on. Legacy identity and access management has a name for someone acting past their authorization: a violation. It doesn't have a good name for an actor that never checked in the first place, because human accounts don't work that way. A person either has the button or they don't.

This gap is where hallucinated permissions live, and it's worth separating from the wider hallucination conversation, most of which is about factual accuracy. Permission hallucination isn't about the agent getting a date wrong. It's about the agent getting itself wrong: which systems it's allowed to touch, whose data it's allowed to see, what the account behind it can actually authorize. Get a fact wrong and the output is bad. Get your own authority wrong and the output is a security incident with the agent's name on it.

This piece is about the shape of that specific failure: why it happens at the model level, why the identity systems most enterprises already run were never built to catch it, and what closes the gap that opens up when a model's opinion of its own access and its actual access come apart.

Why a Model Guesses at Its Own Authority

A large language model doesn't consult a permissions table when it decides whether it's allowed to send an email or update a record. It generates a token sequence that continues the conversation plausibly, and "yes, I have access to this" is exactly as easy to produce as "no, I don't." The model has learned the shape of a confident answer, not a lookup against the account that's actually calling it.

This isn't specific to permissions; it's the same mechanism behind every other kind of hallucination, aimed at a different question. A Stanford study of legal-reasoning tasks found large language models hallucinating at least 58% of the time, often while sounding confident and uncritically accepting a user's incorrect premise. Swap "case law" for "my own access rights" and the mechanism doesn't change. The model isn't lying about its permissions any more than it's lying about a case citation. It's producing a plausible answer to a question it has no way to verify from inside its own reasoning.

By the time anyone reads a transcript, the record has already been read, or the email already sent.

The Identity Systems Agents Actually Meet

Every identity system running in a typical enterprise assumes a shape that agents don't have. Role-based access control assumes a scope decided in advance, before the actor asks for it. OAuth assumes a token issued for a specific delegated purpose. Even service accounts, the closest thing to an agent identity most companies already run, assume the credential gets used the same way every time, by code someone wrote and reviewed once.

An agent breaks each of these assumptions on its own. It doesn't have a scope decided in advance so much as a set of tools it might reach for depending on how a task unfolds. It's frequently running under one shared service credential rather than a per-user grant, because provisioning a real identity per agent is more setup than most teams have gotten to yet. And it decides, at run time, whether a given action is in scope, which is exactly the decision RBAC was built to make for the actor, not delegate to it.

None of the standard tooling was built to catch a plausible-sounding but wrong claim about authority, because no human account generates that claim. A person doesn't reason their way into believing they have access to a system; they either have the login or they request it. An agent can talk itself into access the same way it talks itself into anything else, by continuing the sentence.

Adoption Is Ahead of Governance

The gap between how fast agents are showing up and how ready anyone is for them isn't a guess. SailPoint's 2025 survey of enterprise security and IT leaders found that 82% of organizations already run AI agents in production or pilot, and only 44% have a governance policy covering what those agents can access. Ninety-six percent of the same respondents called agents a growing security threat, and ninety-eight percent said they planned to expand agent use over the following year regardless. Eighty percent reported an agent had already taken an unintended action: accessing a system it shouldn't have, sharing data it shouldn't have shared, or downloading something sensitive. Another 23% said an agent had been talked into revealing its own access credentials.

Read those numbers together and the shape is unambiguous: adoption cleared governance a while ago, and most companies know it and are proceeding anyway. Seventy-two percent of the same respondents said they believe agents pose a greater identity risk than the machine identities they already struggle to manage. That's not a company that hasn't noticed the risk. That's a company that noticed and shipped regardless, because the alternative is falling behind on the thing every competitor is also doing.

We don't think that's irrational, exactly. It's a bet that the identity problem gets solved on a slower timeline than the adoption curve, and every quarter that bet goes unexamined, the gap it's betting against gets wider.

A Hypothetical: The Internal-Tools Agent

Take a hypothetical internal-tools agent, connected to a ticketing system, a document repository, and a chat platform, deployed to triage IT requests. A user asks it to find out why their expense report was rejected. The agent has no direct access to the finance system, but it does have access to the document repository, and a search there turns up a folder holding both expense-policy PDFs and unrelated HR compensation records that were never supposed to be indexed alongside them.

Nothing in the agent's instructions told it to look at compensation data. Nothing in the prompt asked for it. But the folder is in scope, the search tool doesn't distinguish an expense policy from a salary band, and the agent has already formed a plan that treats "this folder is in scope" as equivalent to "everything in this folder is fine to read and summarize." It answers the original question and, in passing, characterizes the requester's pay relative to a colleague's, because that comparison happened to be sitting in the same directory.

No credential was stolen. No prompt injection occurred. The agent's belief about what it could look at was simply broader than what it should have been allowed to touch, and nothing between the search tool and the response checked the difference.

Checking Belief Against Entitlement

The fix isn't making the model more careful. Careful is still a probability, and a probability is exactly the thing that shouldn't decide whether a write goes through. The fix is refusing to let the model's belief be the thing that authorizes the action at all.

Every tool call needs to be checked against the account's actual entitlements at the moment it's made, not against whatever the agent's reasoning concluded a few tokens earlier. The check has to be a lookup, not a judgment: does this identity have this scope on this resource, yes or no. It has to run outside the model's context, somewhere the agent can't talk its way past it by generating a more confident sentence. And it has to run every time, because an agent correctly scoped for yesterday's task can carry the same broad service credential into a task nobody scoped it for today.

Figure 1 — The only variable that matters is where the permission question gets answered. Inside the model, a wrong belief and a right one look identical until something breaks. Outside it, the model's confidence is irrelevant to the outcome.

This is closer to how least-privilege systems already work than it might sound. The unfamiliar part isn't the concept, it's applying it to an actor that can generate a justification for almost anything. Our view of the practical version, for most teams, is binding every agent action to two identities at once: the agent's own scope and the human user's scope, with the tool actually available being whatever sits in the intersection of both. An agent that hallucinates broad access still can't act outside what the requesting user was ever allowed to do, because the check isn't asking the agent what it thinks. It's asking the account.

Conclusion

None of this requires believing agents are getting worse, or that hallucination is somehow new. It requires accepting that a system built to predict plausible continuations will occasionally produce a plausible continuation about its own authority, and that a plausible answer is not a checked one. Factual hallucinations get caught before someone acts on them, at least sometimes. Permission hallucinations get worse the instant execution starts, because there's no reader in between.

The uncomfortable part isn't the mechanism. It's that fixing it means giving up on the idea that a smarter model solves this. It won't, because the model was never the layer doing the checking. Authority has to live somewhere the model can't reason its way around, checked against the real account on every call, not assumed from the tone of the agent's own explanation.