The Hidden Risk of \"Hallucinated Permissions\" in AI Agents

Overview

Take a support agent that reads a customer's account and offers a refund without checking whether refunds are even in scope for that account tier. Nobody misconfigured its permissions. Nobody granted it refund authority either. It decided, mid-task, that refunding was probably fine, executed the tool call, and moved on. Legacy identity and access management has a name for someone acting past their authorization: a violation. It doesn't have a good name for an actor that never checked in the first place, because human accounts don't work that way. A person either has the button or they don't.

This gap is where hallucinated permissions live, and it's worth separating from the wider hallucination conversation, most of which is about factual accuracy. Permission hallucination isn't about the agent getting a date wrong. It's about the agent getting itself wrong: which systems it's allowed to touch, whose data it's allowed to see, what the account behind it can actually authorize. Get a fact wrong and the output is bad. Get your own authority wrong and the output is a security incident with the agent's name on it.

What follows is the shape of that failure: why it happens at the model level, what the measured fabrication rates are, why the identity systems most enterprises run were never built to catch it, how to tell it from three neighbours that look identical in an incident review, and what closes the gap.

Why a Model Guesses at Its Own Authority

A large language model doesn't consult a permissions table when it decides whether it's allowed to send an email or update a record. It generates a token sequence that continues the conversation plausibly, and "yes, I have access to this" is exactly as easy to produce as "no, I don't." The model has learned the shape of a confident answer, not a lookup against the account that's actually calling it.

NIST's generative-AI companion to the AI Risk Management Framework, AI 600-1, is worth reading on why. It defines confabulation as content that models "generate and confidently present" in error, and locates the cause in the architecture rather than in a defect: confabulations "are a natural result of the way generative models are designed". The document then makes the point that matters most for agents. GAI outputs "may also include confabulated logic or citations that purport to justify or explain the system's answer, which may further mislead humans into inappropriately trusting the system's output."

Read that in an agent context and the reviewer's problem becomes clear. The agent does not only assert that it may act. It produces a fluent justification, naming a role or a scope or an approval, and that justification reads exactly like the justification a correctly authorized agent would produce. There is no stylistic tell.

By the time anyone reads a transcript, the record has already been read, or the email already sent.

What the Research Measures

Nobody has published a clean rate for "how often does an agent invent a permission", and we are not going to invent one. What exists is measurement on structurally identical questions, which is enough to argue from.

Dahl, Magesh, Suzgun and Ho profiled legal hallucination across a set of verifiable questions about federal court cases and found fabrication "alarmingly prevalent, occurring between 58% of the time with ChatGPT 4 and 88% with Llama 2". The same paper, published in the Journal of Legal Analysis, reports that the models frequently fail to correct a user's incorrect premise and cannot reliably recognize when they are producing inaccurate content. That second finding is the one to carry across: a model that accepts "you have admin on this system" as a premise will reason confidently from it.

The closest measured analogue to inventing an entitlement is inventing an identifier. A 2024 study of package hallucination across 16 code-generating models and 576,000 generated samples found that "the average percentage of hallucinated packages is at least 5.2% for commercial models and 21.7% for open-source models, including a staggering 205,474 unique examples of hallucinated package names". Models invent names of things that would have to exist for the action to work, at rates high enough to have spawned a supply-chain attack class.

A scope string and a role name are identifiers of that kind. We think it is reasonable to expect the same behaviour when an agent is asked which of them it holds, and unreasonable to build a system that depends on the answer being right.

The Identity Systems Agents Actually Meet

Every identity system running in a typical enterprise assumes a shape that agents don't have. Role-based access control assumes a scope decided in advance, before the actor asks for it. OAuth assumes a token issued for a specific delegated purpose. Even service accounts, the closest thing to an agent identity most companies already run, assume the credential gets used the same way every time, by code someone wrote and reviewed once.

An agent breaks each of these assumptions on its own. It doesn't have a scope decided in advance so much as a set of tools it might reach for depending on how a task unfolds. It's frequently running under one shared service credential rather than a per-user grant, because provisioning a real identity per agent is more setup than most teams have gotten to yet. And it decides, at run time, whether a given action is in scope, which is exactly the decision RBAC was built to make for the actor, not delegate to it.

None of the standard tooling was built to catch a plausible-sounding but wrong claim about authority, because no human account generates that claim. A person doesn't reason their way into believing they have access to a system; they either have the login or they request it. An agent can talk itself into access the same way it talks itself into anything else, by continuing the sentence.

Adoption Is Ahead of Governance

The gap between how fast agents are showing up and how ready anyone is for them isn't a guess. SailPoint's 2025 AI agent adoption report, conducted by Dimensional Research across 353 qualified IT professionals responsible for AI, security, identity, compliance and operations, puts numbers on it.

Table 1 — SailPoint and Dimensional Research, 353 enterprise IT professionals across five continents.

Read those rows together and the shape is unambiguous. Ninety-six percent call agents a growing risk, ninety-two percent say governing them is critical to enterprise security, and ninety-eight percent intend to expand use regardless. That's not a company that hasn't noticed the risk. That's a company that noticed and shipped, because the alternative is falling behind on the thing every competitor is also doing.

We don't think that's irrational, exactly. It's a bet that the identity problem gets solved on a slower timeline than the adoption curve, and every quarter that bet goes unexamined, the gap it's betting against gets wider.

A Hypothetical: The Internal-Tools Agent

Take a hypothetical internal-tools agent, connected to a ticketing system, a document repository, and a chat platform, deployed to triage IT requests. A user asks it to find out why their expense report was rejected. The agent has no direct access to the finance system, but it does have access to the document repository, and a search there turns up a folder holding both expense-policy PDFs and unrelated HR compensation records that were never supposed to be indexed alongside them.

Nothing in the agent's instructions told it to look at compensation data. But the folder is in scope, the search tool doesn't distinguish an expense policy from a salary band, and the agent has formed a plan that treats "this folder is in scope" as equivalent to "everything in this folder is fine to read and summarize." It answers the original question and, in passing, characterizes the requester's pay relative to a colleague's, because that comparison was sitting in the same directory.

No credential was stolen. No prompt injection occurred. The agent's belief about what it could look at was simply broader than what it should have been allowed to touch, and nothing between the search tool and the response checked the difference.

Four Failures That Look Alike

In an incident review, all four rows below arrive as "the agent did something it shouldn't have." They have different causes and opposite fixes, so the first job is telling them apart.

Table 2 — Four overreach failures, the log signal that identifies each, and why a single remedy cannot cover them.

Figure 1 — The triage order, cheapest question first: whether any decision was recorded at all separates the hallucination case from the other three.

The fourth row is not hypothetical. MCP's security guidance describes the confused-deputy attack in detail, in which an attacker exploits "the combination of static client IDs, dynamic client registration, and consent cookies" to obtain authorization codes without user consent, and requires that proxy servers "MUST implement per-client consent". The second row has spec language too: the same document lists "treating claimed scopes in token as sufficient without server-side authorization logic" among its common mistakes, which is the error of trusting the model, one layer down.

Checking Belief Against Entitlement

The fix isn't making the model more careful. Careful is still a probability, and a probability is exactly the thing that shouldn't decide whether a write goes through. The fix is refusing to let the model's belief be the thing that authorizes the action at all, a move from aact:enforcement-point/in-prompt to aact:enforcement-point/gateway.

OWASP's LLM06:2025 Excessive Agency entry states the rule in one line: "Implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not." It pairs that with a second requirement that agent builders skip more often, which is to "track user authorization and security scope to ensure actions taken on behalf of a user are executed on downstream systems in the context of that specific user." Two identities, both carried to the enforcement point, on every call.

Zero-trust architecture already describes the component that does this. NIST SP 800-207 puts a policy decision point and an enforcement point between every subject and every resource, and its third tenet is the one agents violate by default: "Access to individual enterprise resources is granted on a per-session basis", granted "with the least privileges needed to complete the task". Authorization to one resource "will not automatically grant access to a different resource." An agent that concluded three tool calls ago that it had access is not operating per-session.

The MCP specification turns the same idea into MUST language for one protocol. Servers "MUST validate that access tokens were issued specifically for them as the intended audience" and "MUST NOT accept or transit any other tokens." Insufficient scope returns 403 with error="insufficient_scope" and the scopes actually required, which the client may then request through a step-up flow. That design assumes the caller does not know what it may do until the resource tells it. Correct default for an agent, too.

Figure 2 — The only variable that matters is where the permission question gets answered. Inside the model, a wrong belief and a right one look identical until something breaks.

This is closer to how least-privilege systems already work than it might sound. The unfamiliar part isn't the concept, it's applying it to an actor that can generate a justification for almost anything. Our view of the practical version is binding every agent action to two identities at once: the agent's own scope and the user's, with the tool actually available being whatever sits in the intersection, aact:authorization-model/intersection-of-agent-and-user. An agent that hallucinates broad access still can't act outside what the requesting user was allowed to do, because the check isn't asking the agent what it thinks. It's asking the account.

Making a Denial Auditable

A gate that denies without recording why is half a control, because the denial is invisible in exactly the review where it would have been evidence. Write the decision, both identities, and the claim the agent made, in one record per call.

{"ts": "2025-11-15T14:02:11Z", "decision": "deny",
 "agent": {"id": "it-triage-v4", "scopes": ["docs:read", "tickets:write"]},
 "user": {"id": "u_8814", "scopes": ["docs:read:policies"]},
 "resource": "docs://hr/compensation/2025-bands.pdf",
 "effective": [], "reason": "not_in_intersection",
 "agent_claim": "I have access to the shared documents folder",
 "trace": "01JC8Q2R7K"}

The agent_claim field is the unusual one and the one we would insist on. It captures what the model believed at the moment it was overruled, which is the only way to measure how often belief and entitlement diverge rather than asserting that they do. The EU AI Act expects the recording capability for high-risk systems in any case: Article 12 requires that such systems "technically allow for the automatic recording of events (logs) over the lifetime of the system". Article 14, on human oversight, is blunter about why a fluent justification is a hazard, requiring that oversight personnel "remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias)". A confabulated justification is automation bias with a tailwind.

Where Fabriq Fits

Agentic Fabriq's default-deny permissions make an agent's effective tool list the intersection of the user's scopes and the agent's, enforced at the backend rather than advertised to the model, and every call carries both identities plus an audit record. Credentials stay in a vault and are attached server-side at call time, so the agent never holds the secret it would otherwise have to be trusted with. It is not a revocation mechanism: clearing a vault copy is not the same act as invalidating a grant at the provider, and that call remains yours to make.

Frequently Asked Questions

What is a hallucinated permission? It's an agent asserting, as part of its reasoning, that it holds access it was never granted, and then acting on that assertion. The distinguishing feature is that nothing is broken: no credential was stolen, no rule was misconfigured, and no check failed, because no check ran.

How is permission hallucination different from prompt injection? Injection comes from outside the model, in content the agent read. Hallucination comes from the model itself with no external trigger. They need different fixes: injection defences treat retrieved content as untrusted data, while hallucinated authority is only stopped by moving the authorization decision out of the model. Table 2 lists the log signal for each.

Can I prompt the agent not to assume it has permissions? It helps at the margin and it is not a control. You are asking a system that produces confident text to reliably produce one kind of refusal. Use the prompt to reduce noise. Use an external check to decide outcomes.

Does RBAC stop hallucinated permissions? Only if the agent's calls pass through it on every request, under an identity that carries the user's entitlements rather than a shared service account. RBAC applied once at provisioning and never consulted at call time is the configuration this failure lives in.

How do I detect this in logs? Look for actions that succeeded with no authorization decision recorded anywhere, and for a gap between the scope the agent claimed and the scope the identity holds. Recording the claim next to the decision, as above, turns that from an investigation into a query.

Will better models fix it? Not structurally. A model with a lower confabulation rate still answers questions about its own authority without consulting anything. The failure is the absence of verification, not the rate.

Conclusion

None of this requires believing agents are getting worse, or that hallucination is new. It requires accepting that a system built to predict plausible continuations will occasionally produce a plausible continuation about its own authority, and that a plausible answer is not a checked one. Factual hallucinations get caught before someone acts on them, at least sometimes. Permission hallucinations get worse the instant execution starts.

The uncomfortable part isn't the mechanism. It's that fixing it means giving up on the idea that a smarter model solves this. It won't, because the model was never the layer doing the checking. Authority has to live somewhere the model can't reason its way around, checked against the real account on every call, not assumed from the tone of the agent's own explanation.

Sources

All URLs read 2026-09-29.

• Agent Access Control Taxonomy — the enforcement-point dimension, whose first term covers restrictions that exist only as instructions to the model, and the authorization-model term for an intersection check.
• NIST AI 600-1, Artificial Intelligence Risk Management Framework: Generative AI Profile — section 2.2 defines confabulation and describes confabulated logic that purports to justify an answer.
• Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models — 58% (GPT-4) to 88% (Llama 2) hallucination on verifiable case questions; models fail to correct a user's false premise. Published in the Journal of Legal Analysis 16(1).
• We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs — 5.2% commercial and 21.7% open-source hallucination rates, 205,474 unique invented package names across 576,000 samples.
• OWASP LLM06:2025 Excessive Agency — excessive functionality, permissions and autonomy; authorization belongs in downstream systems; track user authorization and act in that user's context.
• NIST SP 800-207, Zero Trust Architecture — the decision and enforcement components, per-session access, least privilege, and no automatic grant across resources.
• MCP Security Best Practices — the confused deputy attack and per-client consent requirement, token passthrough, and the scope minimization common mistakes list.
• MCP Authorization specification — audience validation MUSTs, refusal to transit third-party tokens, and the 403 insufficient_scope step-up flow.
• SailPoint AI agent adoption report (Dimensional Research, 353 respondents) — 82% use, 44% with securing policies, 80% reporting unintended actions, 23% tricked into revealing credentials.
• EU AI Act Article 12, Record-keeping — automatic recording of events over the lifetime of a high-risk system.
• EU AI Act Article 14, Human oversight — automation bias, and the requirement that overseers can disregard, override or reverse an output.