Two frameworks dominate AI governance conversations right now: NIST AI RMF (the Risk Management Framework, plus the Generative AI Profile) and ISO/IEC 42001 (the AI Management System standard, certifiable). Both were written before agents were a deployment pattern. Both still apply but the controls land in different places than they do for a classic ML model.
For agent workloads, almost every meaningful control lives at the network boundary: the proxy that sees every model call, every tool call, every outbound request. That's where auditing happens because that's where ground truth lives.
The four control families that actually matter
Mapping the spec to operational controls, here's what shows up in real agent audits.
1. Inventory & data provenance
NIST AI RMF MAP-4 and ISO 42001 A.7 both require knowing what data flows into and out of the AI system. For an agent, that means:
- An inventory of every external destination the agent reaches.
- For each destination: data classes sent, retention agreement, DPA status.
- For each model: provider, version, modality, training-data opt-out status.
2. Decision logging & explainability
NIST AI RMF MEASURE-2.8 requires that "decisions made by AI actors are explainable to relevant audiences." For agents, the relevant decisions are tool calls. The audit log needs:
- The triggering user request (or upstream task).
- The model's reasoning trace, including which tools were considered.
- The actual request and response payloads (redacted where appropriate).
- Latency, cost, and outcome.
Bonus points if the log is structured enough to answer "show me every time the agent moved money on behalf of customer X" in a single query.
3. Boundary controls & least privilege
ISO 42001 A.6.1.2 (information security) and A.8.3 (operational controls) both require that the AI system operate with least privilege. For agents this is almost entirely a network problem:
- Default-deny egress with an explicit allowlist.
- Destructive verbs (
DELETE, payment endpoints,DROP) require approval. - PII patterns in outbound payloads trigger redaction or block.
- Per-agent and per-tenant scoping the support agent cannot call the admin API.
4. Incident response & rollback
NIST AI RMF MANAGE-4 requires "documented procedures for incidents." For agents the relevant incidents are: tool-call cascades that did harm, prompt injection success, unintended data egress, and runaway cost. You need:
- A kill switch that disables the agent globally.
- Per-tool circuit breakers.
- Replay capability given a logged decision, reproduce it for forensics.
- Rollback patterns for stateful tool calls where the provider supports it.
What an auditor will ask for
- The list of destinations the agent can reach, and the policy that authorizes each.
- A sample of decision logs for a randomly-chosen 24-hour window.
- Evidence that destructive actions are gated by approval (logs of approve/deny events).
- Evidence that PII patterns are detected and handled (test cases + production telemetry).
- An incident from the last 12 months and the response timeline.
- The change-management process for adding a new destination or tool.
If you can answer all six in one afternoon by querying a single log store, your governance is in shape. If any of them requires "let me ask three engineers and look at four dashboards," you're carrying audit risk.
How to make this auditable by construction
The pattern that works: put a policy-as-code proxy on the agent's outbound path, write the policy in your repo, log decisions to a queryable store, and treat the policy file as the canonical answer to "what is allowed?" An auditor reads one file instead of touring your codebase.
AI governance isn't separate from agent engineering. The controls that make auditors happy are the same ones that keep agents from doing damage in production. The framework just gives you the vocabulary to defend them.