Agent Accountability Frameworks

Overview

An agent accountability framework is supposed to answer one question: when this agent does something, who is on the hook for it? Most versions of that question get answered on paper, months before anything happens, and never get tested until the day something does.

That is the test worth designing around. A roster of six named owners can still fail completely at two in the morning if none of the six can do anything about what is happening. We think most published frameworks optimize for the roster, because a roster is what's easy to produce in a workshop, and treat the mechanism as an implementation detail to sort out later.

The standards bodies are less confused about this than the frameworks built on top of them. NIST's AI Risk Management Framework asks under GOVERN 3.2 for policies that "define and differentiate roles and responsibilities for human-AI configurations and oversight of AI systems". Differentiate is the operative word, and it is the one that gets dropped.

What follows separates the roles that carry real authority in the first minutes of an incident from the ones that carry it afterward, and puts the statutory deadlines next to both.

Why Informal Ownership Fails

Informal ownership works exactly until something goes wrong. Then the conversation turns into a search for whoever happens to remember the agent exists.

Take a marketing analytics agent that pulls campaign performance from several SaaS tools and writes a weekly summary into a shared workspace. A contractor built it during a quarterly push and has since rolled off. The agent still runs on their leftover service credentials. When it starts producing numbers finance can't reconcile, nobody can say who decided what counts as a conversion, who is allowed to change the logic, or whether those credentials should still be live.

Nothing here is exotic, and regulation now addresses it directly. Article 26(2) of the EU AI Act requires deployers of high-risk systems to "assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support". Three of those four words describe a mechanism. The contractor had all of them and then left with them.

Two Clocks, Not One

Once ownership is written down, the next mistake is treating every role as equally relevant to every moment. An incident runs on two clocks, and a framework that doesn't separate them produces a roster that looks complete and performs unevenly.

Figure 1 — The same six roles, split by how quickly each one can actually respond once something has gone wrong.

Who Can Act in the First Ten Minutes

Two roles have a mechanism that works immediately, and they are the ones an incident actually depends on.

The technical owner can change what the agent does next: pull it out of production, patch the logic, roll back a deploy. If the marketing agent's numbers are wrong because an upstream schema changed, the technical owner is the only person who can stop it from producing a seventh bad report while everyone else is still figuring out what happened.

The security owner can change what the agent is allowed to reach: revoke a credential, narrow a scope, cut a connection. If those leftover contractor credentials are the actual problem, the security owner can kill them in the next five minutes rather than the next five days.

The AI Act writes the fast clock into the product rather than the org chart. Article 14(4) requires that a high-risk system's design let a person "decide, in any particular situation, not to use the high-risk AI system or to otherwise disregard, override or reverse the output", and "intervene in the operation of the high-risk AI system or interrupt the system through a 'stop' button or a similar procedure that allows the system to come to a halt in a safe state". Two capabilities, named separately, because they are different buttons. Override changes one output. Stop ends the run.

Everyone else in the framework is useful, and none of them can do either of those things.

Who Decides What Happens Next

The other four roles matter just as much. They just don't matter yet.

The business owner decides whether the agent should keep existing once the fire is out. NIST AI RMF makes this its own governance requirement, GOVERN 1.7, asking for processes "for decommissioning and phasing out AI systems safely and in a manner that does not increase risks or decrease the organization's trustworthiness". Retirement is a decision somebody has to be empowered to make, not a thing that happens when a budget line lapses.

The compliance or risk reviewer determines whether what happened crossed a regulatory line, which is rarely knowable in the first hour. Article 3(49) defines a serious incident as covering death or "serious harm to a person's health", "a serious and irreversible disruption of the management or operation of critical infrastructure", "the infringement of obligations under Union law intended to protect fundamental rights", and "serious harm to property or the environment". Deciding which of those four a bad afternoon falls into is legal work, on legal timescales.

The operator, whoever was working with the agent day to day, is the person whose judgment gets reconstructed after the fact. Article 14(4)(b) names what makes that reconstruction awkward: a human overseer has to "remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias)". A sign-off nobody read is a sign-off that will be read very carefully once.

The platform owner doesn't own this agent's outcome at all. They own whether a record existed for the incident to be reconstructed from. Article 26(6) sets the floor: logs kept "for a period appropriate to the intended purpose ... of at least six months, unless provided otherwise in applicable Union or national law". If your retention is 30 days, the platform owner has already decided the compliance reviewer's answer for them.

The Mechanism Test

Here's the test we'd apply to any role, fast clock or slow: if this agent misbehaves tomorrow, what specifically can this person do about it?

If the honest answer is "escalate to someone else," that role is a contact, not an owner, and the framework should say so rather than count it toward coverage. NIST SP 800-61r3 makes the same point under GV.RR-02: "All appropriate individuals or parties should be designated the authority necessary to fulfill their incident response-related responsibilities." Designated authority, in writing, next to the responsibility.

Table 1 — The mechanism test, one row per role. A role that fails its test is a contact, not an owner.

The fix isn't veto power for everyone. It's to write down, next to each name, the actual thing that name can do, and to notice when the answer is nothing. Informal ownership and a six-name framework with no mechanisms fail the same way by different paths. One has no names. The other has a spreadsheet.

The Clocks You Do Not Set

The two-clock model is not a metaphor. Regulators already run two clocks, and they have written the second one down in days.

Table 2 — Statutory response windows, from the regulation text. Every row starts on awareness, not on resolution.

Figure 2 — Containment and notification run from the same t=0. A team that investigates first and notifies afterwards has spent the statutory window on the fast clock's work.

Every clock starts at awareness. Not at root cause, not at fix. Article 73(4) is explicit that the report goes in when a provider or deployer "has established, or as soon as it suspects, a causal relationship". A framework whose first move is a week of investigation has spent the whole window before anyone drafted a sentence.

The reporting clock also runs in parallel with the containment clock, not after it. Both windows are long enough to contain an incident and short enough that a team discovering on day two that nobody is authorised to sign a notification has a second problem on top of the first.

The size of the miss is written down too. Article 99 sets fines of up to EUR 35 000 000 or 7% of worldwide annual turnover for prohibited practices, EUR 15 000 000 or 3% for operator obligations, and EUR 7 500 000 or 1% for supplying incorrect or incomplete information to authorities. That last tier is the one a bad accountability framework walks into: it penalises getting the answer wrong, not causing the harm.

One timing note, because a lot of 2025 writing has it wrong. The high-risk obligations no longer land on 2 August 2026. Following the Digital Omnibus, the European Commission's regulatory-framework page puts Annex III rules at 2 December 2027 and Annex I rules, for systems embedded in regulated products, at 2 August 2028.

Matching Weight to Risk

The fastest way to make any of this fail is applying it uniformly. A read-only internal FAQ agent does not need the same six-role machinery as one that can move money, and forcing it through that process teaches teams to route around the framework rather than use it.

Weight should scale with what an agent can do. A read-only agent over non-sensitive data needs little more than a named business and technical owner. Give it write access, or a path to personal data, and it needs a security owner accountable for the scope. Give it the ability to move money or change production, and it needs the full set, with revocation and rollback tested before launch rather than assumed.

The AI Act shows what the top tier looks like written as law. Article 14(5) requires that for certain biometric identification systems, no action is taken on an identification "unless that identification has been separately verified and confirmed by at least two natural persons with the necessary competence, training and authority". That is the ceiling. Almost nothing you build needs it, and knowing it exists makes it easier to argue the FAQ bot does not.

How Frameworks Fail Quietly

The interesting failures are the ones where the document still looks right.

The owner has left. Symptom: the escalation path resolves to a disabled directory account, and the rota is a wiki page nobody owns. The departed-contractor failure, one abstraction layer up. Test by paging the listed owner for a random agent quarterly.

The kill switch is too coarse to use. Symptom: the only revocation available is an org-level toggle that also stops 40 healthy agents, so nobody pulls it during business hours. A kill switch with a blast radius larger than the incident gets debated instead of pressed.

Governance fails open, which is aact:failure-behavior/fail-open in the taxonomy. The OWASP Agent Control Standard defines a Guardian that inspects what an agent is about to do and returns allow, deny, or modify. Its reference implementation ships with a default failure posture of proceed, and its own README says so: "A Guardian that crashes, hangs, or is unreachable stops governing, silently, and the host proceeds as if nothing asked." The same README notes the reference wire is unauthenticated. Both gaps are published, with issue numbers. The symptom of fail-open is the absence of a symptom.

The record ran out. Symptom: the compliance reviewer arrives in week three and the logs rolled at 14 days.

Oversight without authority. Symptom: the person named for human oversight can raise a concern but not stop a run. Article 26(2) asks for competence, training and authority, and the third is what goes missing in a reorganisation.

Two of these look identical from the outside and want opposite fixes. An agent that kept acting because the Guardian was down, and one that kept acting because the operator had no stop authority, both present as "nobody intervened". The first is an infrastructure default, the second a permissions grant. Telling them apart takes a log of decision outcomes, which is why the platform owner's row in Table 1 is not administrative trivia.

A Record Worth Keeping

Frameworks get published as prose and then never read during an incident. The version that gets used is short, machine-readable, and lives next to the agent. One file per agent:

agent: marketing-weekly-rollup
tier: 2                      # write access, no personal data
owners:
  technical:
    name: a.okonkwo
    mechanism: disable_agent  # org console toggle, tested 2026-07-14
    rollback: revert_to_tag   # last signed-off tag: rollup-2026-06-30
  security:
    name: platform-security-oncall
    mechanism: revoke_connection  # per-user vault entry, tested 2026-07-14
    note: clearing the vault copy does NOT revoke the grant upstream
  business:
    name: d.raymond
    mechanism: retire_agent
  compliance:
    name: risk-review
    mechanism: audit_query     # BigQuery, retention 400d
failure_behavior: aact:failure-behavior/fail-closed  # guardian unreachable => deny and record
reporting:
  serious_incident_signatory: legal-oncall   # AI Act Art. 73
  data_breach_signatory: dpo                 # GDPR Art. 33, 72h
last_drill: 2026-07-14

Three fields do the real work and are the three most often absent. mechanism names the specific action, not the role. last_drill is the date somebody proved the mechanism fired. The two signatory lines exist because a statutory clock needs a name attached before it starts.

Back to the marketing analytics agent. Under a framework that only names owners, the response is a search. Under this one, the security owner revokes the contractor credential and issues the agent its own scoped identity, the technical owner rolls report generation back to the last tag finance signed off, and only then does the slow work start.

Frequently Asked Questions

What is an agent accountability framework? A written assignment of who answers for an agent's behaviour, plus the part most versions skip: what each of those people can actually do about it. NIST AI RMF's GOVERN 2.1 covers the documentation half. SP 800-61r3's GV.RR-02 covers the authority half, recommending that parties "should be designated the authority necessary to fulfill their incident response-related responsibilities".

Who is legally responsible for an AI agent in the EU? It depends which side of the provider/deployer line you sit on. Article 3(3) defines a provider as whoever develops a system and places it on the market or puts it into service under their own name; Article 3(4) defines a deployer as whoever uses one "under its authority" outside personal non-professional activity. Build an agent for internal use and you are frequently both.

How many roles should a framework have? The count is not the variable. Two roles that pass the mechanism test in Table 1 beat six that do not.

How fast do we have to report an agent incident? Under the AI Act, 2 days for a widespread infringement or serious critical-infrastructure disruption, 10 days where a death is involved, 15 days for other serious incidents, each measured from becoming aware. Under GDPR, 72 hours. Authorities then get 7 days of their own under Article 73(8).

Do the AI Act's high-risk obligations start in August 2026? No, and this is the single most common stale fact in circulation. The Digital Omnibus moved them. The Commission's regulatory-framework page now gives 2 December 2027 for Annex III high-risk systems and 2 August 2028 for Annex I ones.

How long do we have to keep agent logs? Article 26(6) sets a minimum of six months for deployers of high-risk systems, "unless provided otherwise in applicable Union or national law", and sector rules routinely go longer. Six months is the floor, not a target.

Is a RACI matrix enough? Not on its own. RACI records who is accountable; it does not record whether they hold a credential that can stop the thing. Add a mechanism column and a date it was last exercised, and RACI becomes useful.

What counts as a serious incident for an agent? Article 3(49)'s four limbs: death or serious harm to health, serious and irreversible disruption to critical infrastructure, infringement of Union law obligations protecting fundamental rights, or serious harm to property or the environment. An agent that leaks data usually reaches the fundamental-rights limb first.

Who signs the regulatory notification? Somebody named in advance. This is the field most frameworks lack, and the reason a two-day clock gets missed by teams who contained the incident in twenty minutes.

Conclusion

An accountability framework does not make agents safe by itself. Done well, it guarantees that when something goes wrong, someone can act immediately, someone else can file within the statutory window, and a third person can stop it recurring. Naming six roles gets you a document. Attaching a mechanism to each, plus the date you last proved it fires, gets you a response.

Build the roster from the incident backward: who can revoke, who can roll back, who signs the notice, and only then who else needs to know.

Sources

All URLs read 2026-09-29.

• NIST SP 800-61r3, Incident Response Recommendations and Considerations for Cybersecurity Risk Management — April 2025; the "authority to confiscate, disconnect, or shut down technology assets" language and the GV.RR-02 recommendation on designated authority.
• NIST AI 100-1, AI Risk Management Framework 1.0 — GOVERN 1.7 on decommissioning, GOVERN 2.1 and 2.3 on documented roles and executive responsibility, GOVERN 3.2 on differentiating human-AI oversight roles.
• EU AI Act Article 26, Obligations of deployers of high-risk AI systems — paragraph 2 on competence, training and authority; paragraph 5 on informing the provider and suspending use; paragraph 6 on the six-month log floor.
• EU AI Act Article 14, Human oversight — paragraph 4(b) automation bias, 4(d) override and reverse, 4(e) the stop button, and paragraph 5 two-person verification.
• EU AI Act Article 73, Reporting of serious incidents — the 2-day, 10-day and 15-day windows and the authority's own 7 days.
• EU AI Act Article 3, Definitions — definition (3) provider, (4) deployer, and (49) serious incident.
• EU AI Act Article 12, Record-keeping — automatic logging over the lifetime of the system and the minimum recorded items.
• EU AI Act Article 99, Penalties — the EUR 35m/7%, EUR 15m/3% and EUR 7.5m/1% fine tiers.
• European Commission, AI Act regulatory framework — the post-Digital-Omnibus dates: 2 December 2027 for Annex III, 2 August 2028 for Annex I.
• Regulation (EU) 2016/679 (GDPR), Article 33 — the 72-hour breach notification window measured from awareness.
• Agent Access Control Taxonomy — the failure behavior dimension; fail-open for the Guardian default below and fail-closed for the posture the per-agent record pins.
• OWASP Agent Control Standard — the Guardian allow/deny/modify wire contract, and its published fail-open default posture and unauthenticated reference wire.