For most of the internet's history, a transaction was something a human could see. A person clicked, a person typed, a person authorized. Even when a system acted on a schedule, the chain back to a human decision was short and visible.
Agentic AI breaks that visibility. When an AI agent buys a flight, queries a database, files a ticket, or transfers a record on a user's behalf, the chain from human authority to system action is no longer obvious. Most enterprise AI governance programs were designed for AI that recommends. Few were designed for AI that acts. That gap is now the single largest source of unmanaged AI risk in regulated enterprises.
This article presents a six-layer accountability framework for governing AI agents in regulated environments. Five of the layers are familiar - identity, authorization, validation, runtime decisioning, and audit logging. The sixth - the Responsibility Layer - is the layer most programs have not yet thought through, and it is the one that regulators, boards, and legal counsel will ask about first when an agent action causes harm.
Why Agents Require a Different Governance Stack
A traditional ML model produces a score. A human reads the score and makes a decision. The audit trail is straightforward: model output, human deliberation, human action.
An agent skips the middle step. It reads context, decides, and acts. It may chain multiple tool calls, coordinate with other agents, and complete a transaction without a human in the loop for any specific decision. The audit question shifts from "what did the model output?" to "who is accountable for what the agent did, and how do we prove it?"
That shift is not a minor extension of model risk management. It requires a distinct governance stack with controls at six layers - each addressing a specific failure mode that traditional model governance does not cover.
The Six Layers
Layer 1 - Identity
Question the layer answers: Who is this agent, and whose authority is it acting under?
Every agent action must be traceable to a documented identity. That identity includes (a) the agent itself - model, version, configuration, deployment environment - and (b) the human or organizational principal whose authority the agent is exercising. An agent that drafts an email under your name has different identity properties from an agent that executes a trade on behalf of a desk.
In regulated environments, identity is not a single field. It is a chain: agent → orchestrator → tenant → human principal → role → policy bundle. Every link in that chain must be evidenced and persisted. The agent registry is the foundational control.
Layer 2 - Authorization
Question the layer answers: What is this agent permitted to do, with what data, at what cost?
Authorization is the pre-flight gate. Before any tool call, the layer evaluates: is this tool in the agent's scope? Is the data scope it is requesting within its grant? Is the proposed spend within budget? Is the counterparty in the allow-list?
Most production agent systems implement Layer 2 weakly - usually as a hard-coded tool list per agent. Regulated environments need a scope DSL: a declarative, auditable specification of permitted tools, data classes, spend ceilings, counterparty restrictions, and temporal limits. The scope must be inspectable by second-line risk and externally testable by internal audit.
Layer 3 - Validation
Question the layer answers: Does this specific request pass schema, policy, and content checks?
Validation is per-call enforcement: schema validation on tool inputs and outputs, PII detection and redaction, prohibited-content classification, jurisdictional content rules, and policy assertions specific to the action class. A vendor due diligence questionnaire request is validated differently from a customer record update.
Layer 3 is where most agent platforms have invested. It is necessary but not sufficient - passing validation tells you the request is well-formed, not that it is appropriate.
Layer 4 - Runtime Decisioning (the Judge Layer)
Question the layer answers: Given everything we know about intent, context, and policy - allow, deny, or escalate?
The Judge Layer is the runtime arbiter that synthesizes signals from Layers 1-3 and applies policy logic. It is the agent equivalent of a transaction approval engine. It must:
- Evaluate the full action proposal against the agent's scope and the broader policy regime
- Distinguish between automatic allow, allow-with-logging, allow-with-human-notification, escalate-to-human, and deny
- Operate at action latency (not after-the-fact)
- Produce a structured decision record consumable by Layer 5
This is the layer where most agent governance programs have the largest gap. Audit trails exist; runtime arbitration is rare.
Layer 5 - Audit
Question the layer answers: Can we reconstruct what happened, on demand, with integrity?
Audit is the immutable record. Every Layer 4 decision - allowed and denied - must be persisted with sufficient context to reconstruct the action chain, the input bundle, the policy version applied, and the decision rationale. The record must be tamper-evident, retained for the relevant compliance horizon (typically 7 years for financial-services-adjacent activity), and exportable in a format internal audit and external examiners can consume.
Most enterprises confuse logging with audit. Logs answer the operations question (what happened, when, was it fast?). Audit answers the accountability question (what was decided, why, by what authority, with what evidence?).
Layer 6 - Responsibility (the layer most programs are missing)
Question the layer answers: When the agent action causes harm, who is accountable - and how is the mapping documented?
The Responsibility Layer is the bridge from technical governance to legal, contractual, and regulatory accountability. It answers seven specific questions for any deployed agent:
- Identity attribution - when this agent acted, which human / organizational role's authority was it acting under, and is that role contractually responsible for that action class?
- Scope evidence - what is the documented authorization scope, and is the action provably within it?
- Counterparty disclosure - did the counterparty know they were transacting with an agent? Is that disclosure compliant with the applicable jurisdiction's rules?
- Decision trace - for each decision point in the action chain, what was the input bundle and the chosen branch?
- Failure assignment - if the action causes harm, what is the contractual / insurance / regulatory mapping of liability between vendor, deployer, and end user?
- Reversal mechanism - is there a documented path to reverse, refund, or remediate the action, and what is the SLA on it?
- Evidence retention - are the audit and responsibility records retained per the relevant compliance horizon?
These seven questions are not technical infrastructure questions. They are legal, contractual, and regulatory questions that depend on infrastructure to be answerable. The Responsibility Layer is what makes the prior five layers useful to the people who will be on the call when something goes wrong: General Counsel, Compliance, Risk, the Board, and the regulator.
Mapping the Six Layers to Regulatory Frameworks
For Canadian regulated enterprises, the layers map cleanly onto existing OSFI guidance and international standards:
| Layer | OSFI B-13 | OSFI E-23 | OSFI B-10 | ISO 42001 | NIST AI RMF | EU AI Act |
|---|---|---|---|---|---|---|
| 1. Identity | Roles & accountabilities | Model ownership | Vendor identity | A.3 Internal organization | Govern | Article 26 |
| 2. Authorization | Access controls | Use-case approval | Vendor scope | A.6 Lifecycle | Govern + Map | Article 14 |
| 3. Validation | Data quality | Input/output controls | Vendor SLAs | A.7 Data | Map | Article 10 |
| 4. Judge (runtime) | Operational risk controls | Model use authorization | Vendor monitoring | A.9 Use | Manage | Article 14 |
| 5. Audit | Logging & monitoring | Model performance evidence | Vendor reporting | Clause 9 | Manage + Govern | Article 12 |
| 6. Responsibility | Three Lines accountability | Model owner / approver attestation | Contract terms + indemnity | Clause 5 leadership | Govern | Articles 9, 16, 26 |
The Responsibility Layer is where the regulatory expectations intensify. OSFI's Three Lines of Defense model is fundamentally a responsibility-allocation framework. ISO 42001 Clause 5 requires senior management to demonstrably own AI governance. EU AI Act Article 26 places explicit obligations on deployers of high-risk AI systems. None of these requirements are satisfied by infrastructure alone - they require documented contractual and organizational mappings between agent action and human accountability.
Five Practical Steps for Regulated Enterprises
For institutions running or planning to deploy AI agents in regulated workflows, five steps make sense in the next two quarters:
1. Inventory your agents and their current accountability posture. For each deployed or planned agent, map which of the six layers have controls in place and which are gaps. Most enterprises find that Layers 1, 3, and 5 are partially covered, Layers 2 and 4 are weakly covered, and Layer 6 is not covered at all.
2. Build the agent scope DSL before you scale. Agent authorization (Layer 2) becomes exponentially harder to retrofit. A simple declarative scope specification - tools, data classes, spend ceilings, counterparty rules, temporal limits - paid back many times over as the agent portfolio grows.
3. Make the Judge Layer a first-class engineering investment. Runtime decisioning is not "logging plus alerts." It is an arbitration engine that synthesizes identity, authorization, and validation signals and applies policy. Treating Layer 4 as a sprinted feature inside an audit logging project guarantees it will be under-built.
4. Draft your Responsibility-Layer documentation now. For each agent in your inventory, draft answers to the seven Responsibility-Layer questions. The drafting exercise reveals where your contracts, policies, and operational procedures are inconsistent or silent. Fixing those inconsistencies before a regulator or counterparty asks is significantly cheaper than fixing them after.
5. Integrate the six-layer framework into your AI inventory, risk register, and audit plan. The framework only produces value when the rest of the governance program recognizes it. AI inventory entries should record layer coverage. Risk register entries should reference layer gaps. Internal audit plans should test layer effectiveness on a defined cycle.
The Bottom Line
Most enterprise AI governance programs were designed for AI that recommends. They are not equipped for AI that acts. The six-layer accountability framework - identity, authorization, validation, runtime decisioning, audit, and responsibility - gives regulated enterprises a complete model for governing agents in production.
The Responsibility Layer is where the next decade of agentic AI regulation will concentrate. Organizations that build that layer now - documenting the legal, contractual, and organizational mappings between agent action and human accountability - will be the ones that can deploy agents in regulated workflows with regulator confidence, board confidence, and counterparty confidence. Organizations that defer it will be the ones explaining to a regulator, after an incident, why the question "who was responsible?" did not have a documented answer.
See also:
- prEN 18286: The European Harmonized Standard That Will Define EU AI Act Compliance
- ISO 42001: What Regulated Enterprises Need to Know About the World's First AI Management System Standard
- 12 Months to OSFI E-23: A Readiness Checklist for AI and Model Risk Programs
Aeon AI Risk Management helps regulated enterprises design and operationalize multi-layer AI agent governance frameworks. Contact us at info@airiskmanagement.ca or visit /contact.