AI governance glossary: 40 terms worth getting right

AI governance has accumulated vocabulary faster than it has accumulated agreement. Several of these terms mean materially different things depending on who is using them, and where that is true, this says so.

Grouped by area rather than alphabetically, because the terms make more sense next to their neighbours.

Regulation and frameworks

AI Act (EU). Regulation (EU) 2024/1689. Binding EU law assigning obligations by risk tier and by role. Prohibitions and AI literacy applied from February 2025, general-purpose AI model obligations from August 2025, transparency and Annex III high-risk requirements from August 2026, and Annex I high-risk from August 2027. See the EU AI Act overview.

Annex III. The list of eight areas in the EU AI Act where a system is presumed high risk: biometrics, critical infrastructure, education, employment, essential services, law enforcement, migration, and justice and democratic processes.

Article 6(3) derogation. The escape hatch from Annex III. A system in a listed area is not high risk if it performs only a narrow procedural task, improves a completed human activity, detects patterns without replacing human assessment, or performs a preparatory task. Profiling natural persons closes the hatch regardless.

Provider vs deployer. Under the EU AI Act, a provider develops a system and places it on the market under its own name. A deployer uses a system under its own authority. Building an internal tool on a foundation model typically makes you both. The distinction determines which obligations attach.

FRIA (Fundamental Rights Impact Assessment). An Article 27 obligation on deployers of certain high-risk systems, required before first use. Triggered for public bodies, providers of public services, credit scoring of natural persons, and life or health insurance pricing.

GPAI (General-Purpose AI model). A model with broad capability usable across many tasks. Carries its own obligations under Chapter V of the EU AI Act, distinct from those on systems built with it.

NIST AI RMF. A voluntary US framework organised as four functions: Govern, Map, Measure, Manage. No certification exists. See the NIST AI RMF overview.

ISO/IEC 42001. The certifiable AI management system standard. Clauses 4 to 10 define the management system. Annex A lists 38 controls. See the ISO 42001 overview.

Trust Services Criteria. The AICPA criteria underlying SOC 2: nine common criteria plus availability, confidentiality, processing integrity and privacy. See SOC 2 for AI systems.

Crosswalk. A mapping between controls in different frameworks, so evidence recorded once satisfies several. This is the practical difference between running four frameworks and running one programme reported four ways.

Control. A specific requirement within a framework that you assess yourself against. Distinct from a policy, which is what you say you do, and from evidence, which is proof you did it.

Conformity assessment. The EU AI Act Article 43 procedure by which a provider demonstrates a high-risk system meets Chapter III requirements before placing it on the market.

Risk and assessment

AI system. The unit governance attaches to. Defined by purpose rather than by model. The same model serving document search and candidate ranking is two systems with two classifications.

Risk tier. Under the EU AI Act, one of prohibited, high risk, limited risk or minimal risk. Derived from the system’s purpose and context rather than chosen.

Inherent vs residual risk. Inherent is risk before controls. Residual is what remains after. Reporting one and labelling it the other is the error to check your register for.

Risk dimension. An axis along which an AI system is scored, commonly bias, security, privacy, fairness, performance and transparency. Multi-dimensional scoring avoids the failure where a single “risk: high” label hides which kind of high.

Impact assessment. A structured evaluation of a system’s effects on individuals and society. Required in various forms by ISO 42001 A.5, EU AI Act Article 27, and GDPR, where it is called a DPIA.

Model card. Structured documentation of a model’s purpose, training data, performance characteristics and limitations. Commonly attached as evidence for transparency and documentation controls.

Evidence artifact. A concrete document supporting a control assessment. A model card, a DPIA, a penetration test, a signed contract. The thing an auditor asks for when you claim a control is met.

Governance exception. A documented, time-bound decision to operate outside policy. Two properties make it governance rather than an excuse: an expiry date, and a named owner.

Coverage. The proportion of applicable controls assessed for a system or estate. Useful as a completeness measure and misleading as a safety measure, since it counts assessments rather than outcomes.

Agents and tooling

AI agent. A system that plans and takes actions toward a goal, typically invoking tools in a loop rather than returning a single response. Governed by what it can invoke.

Autonomy level. How much human oversight applies to an agent’s actions. A common scale runs manual, human-in-the-loop, human-on-the-loop, supervised, fully autonomous.

Human-in-the-loop. A human approves each consequential action before it executes. Distinct from human-on-the-loop, where a human monitors and can intervene but does not gate each step. The terms get used interchangeably, and they describe very different control strengths.

MCP (Model Context Protocol). A standard for connecting AI models to external tools and data sources. Its significance for governance is that it makes tool access a legible, observable interface rather than bespoke integration code.

MCP server. A process exposing a set of tools over MCP. Frequently third-party code holding credentials of its own, which makes it a supply-chain dependency that never went through procurement.

Tool allowlist. The set of servers and tools a given agent is permitted to invoke. Most useful when built from observed behaviour rather than declared intent. See governing MCP tools.

Tool call kind. Whether an invocation was an MCP tool, a provider-hosted server tool, or a client-side tool defined by the application. Different trust boundaries, worth recording separately.

Session trace. A reconstruction of one agent or assistant session as an ordered sequence of turns, model calls and tool invocations. The artefact you need when someone asks what a system did, rather than what it was meant to do.

Shadow AI

Shadow AI. AI tools in use without having gone through registration, review or approval. The AI-specific case of shadow IT, and larger, because the barrier to adoption is a browser tab.

Discovery. The process of finding unregistered AI tools. Practical sources include identity-provider OAuth grants, MDM inventories, endpoint telemetry, DNS and proxy logs, and self-reporting. See five ways unregistered AI tools get discovered.

OAuth scope. The specific permission a user granted a third-party app. The most useful signal in shadow AI discovery, because it describes what the tool can reach rather than merely that it exists.

Match confidence. How certain a discovery system is that an observation corresponds to a known tool. Observations that match nothing are the most interesting findings, not noise to discard.

Deduplication. Collapsing multiple observations of the same tool from different sources into one record. Duplicate counts make every other number in the report suspect.

Blocklist feed. A machine-readable list of domains for tools marked blocked, published for network controls, firewalls or a CASB to consume. This is the mechanism that turns a “blocked” status in a governance tool into an actual block, since the governance tool does not sit in the network path.

Identity-layer enforcement. Disabling a tool’s access at the identity provider by revoking the OAuth application or disabling its service principal. Stronger than domain blocking, because it survives the user switching networks.

Operations and telemetry

AI proxy. A transparent intermediary between your applications and model providers. The highest-value component in AI governance, because it is the one place where policy can be enforced, traffic inspected and usage recorded without changing every application.

Policy-as-code. Governance rules expressed in machine-readable form and evaluated automatically rather than reviewed by a human. Typically runs in one of three modes: off, dry-run, or enforce.

Dry-run mode. Evaluating a policy and recording the verdict without acting on it. The mechanism that makes enforcement adoptable, because it lets you answer “what would this have blocked last month?” before you turn it on.

Policy denial. A recorded instance of a policy blocking a request. Valuable as evidence that controls operate rather than merely exist, which is the distinction auditors care about.

Inline DLP. Inspecting AI traffic in both directions for secrets, credentials, personal data or exfiltration attempts, at the point the traffic passes through. Distinct from scanning data at rest.

Prompt leakage probe. A scheduled test that attempts to extract system prompts, credentials or personal data from your own AI endpoints. Continuous testing of a control rather than a point-in-time assertion.

Audit log. An append-only record of who changed what and when. Underpins SOC 2 CC4 and CC7, ISO 42001 A.6.2.8, and EU AI Act Article 12 record-keeping. It costs nothing when built in, and cannot be retrofitted over history.

---

These definitions reflect how the terms are used in practice as of September 2026. Several are still settling.