AI governance and guardrails: the adult in the room, built into the code

If agentic AI is the loudest keyword of 2026, governance is the one that decides whether any of it stays switched on. The consultancy Wavestone frames the whole year around it: less chasing new capability, more making the AI already deployed governed and enterprise-ready. The regulators agree. The EU AI Act's obligations for high-risk systems became enforceable on 2 August 2026, covering risk management, data governance, logging, transparency, human oversight, and post-market monitoring. Standards bodies have ISO/IEC 42001 for AI management systems and NIST's AI Risk Management Framework. Boards now ask about all of it.

Most of what gets written about this is policy. This post is about the other half, which is engineering. Governance that lives in a document is an intention. Governance that lives in the code is a control. Here is what the second kind looks like, and what to build first. We are engineers, not lawyers, so read the regulatory parts as orientation and get your own advice on what applies to you.

Why agents changed the problem

Governance for the first wave of generative AI was about content. Does the model say something false, biased, or leaked? Filters on inputs and outputs handled a lot of it. Agents break that model, because an agent's output is not a sentence, it is an action. It reads a record, calls an API, writes to a database, sends a message, and does that dozens of times in one run, with errors compounding as it goes. Independent reviews have pointed out that the major frameworks, the EU AI Act included, were written before this and do not mention agents at all.

That gap is the reason guardrails have become an engineering topic. You cannot filter an action after it happens. You have to constrain what actions are possible, and record every one that is taken.

Guardrails, translated into components

Here is what "guardrails" means when it is code rather than a slide.

Permissions at the tool boundary. Every tool an agent can call is scoped to the narrowest access that lets it work, enforced by the tool and the system behind it, never by the prompt. A prompt saying "do not delete records" is a suggestion. A tool that has no delete capability is a guardrail. We wrote about designing that boundary in MCP servers, done safely.

Budgets and stop conditions. Steps, tokens, money, and wall-clock time per run, with the run halting when any is exceeded and a person notified. This is the difference between a bad run costing a dollar and costing a thousand.

Human oversight, as a path in the workflow. The EU AI Act asks for meaningful human oversight of high-risk systems. In practice that means a defined point where the agent hands off: any irreversible action, any decision below a confidence threshold, anything touching a protected category. The hand-off is a queue with a person on it, not a checkbox.

An audit log that would survive a regulator. Every input, every tool call with its arguments, every output, every override, with timestamps and identities, retained for as long as the rules say. This is the logging obligation in the EU AI Act and it is also, not by coincidence, exactly what you need to debug an agent. Build it once for both reasons.

Data governance at the context layer. What goes into the model's context window is where most leaks start. Guardrails here mean retrieval that respects the caller's permissions, redaction before the prompt is assembled, and a record of what the model was shown. The prompt injection and data leakage post covers the attacks; the governance version is proving to an auditor that the controls exist.

Evaluation as a gate. A model or prompt change is a change to the system's behaviour and should be treated like a code change: tested against an evaluation set before it ships, with the results kept. Evals before features is the practice; the governance framing is that it is your evidence the system does what you documented.

A kill switch. Any AI feature should be switchable off in production without a deploy, per tenant if you have tenants. When something goes wrong at 2am, the first question is whether you can stop it.

What the frameworks want, mapped to the components

A rough map, for orientation. ISO/IEC 42001 wants a management system: policies, roles, risk assessment, and continual improvement, which is the paperwork around the components above. NIST's framework wants you to govern, map, measure, and manage risk, where "measure" is your evaluation set and "manage" is your permissions, budgets, and hand-offs. The EU AI Act, for systems it classifies as high-risk, wants documented data governance, logging, human oversight, accuracy and robustness, and registration. If you build the components, the documents write themselves from the code. If you write the documents first, you have a binder and no controls.

Most systems our clients build are not high-risk under the Act. The components are still worth building, because they are what makes an agent debuggable, and because the customer security questionnaire asks for the same evidence the regulator does.

What to build first

1. Inventory. List every place a model runs in your product and what it can touch. Most teams find one or two they had forgotten.
2. Permissions at the tool boundary, tightened to the minimum, before anything else.
3. The audit log, complete and retained, because every other control produces evidence into it.
4. Budgets, stop conditions, and the kill switch, which together take a week and remove the worst-case scenarios.
5. The human hand-off path for irreversible actions, wired into the workflow rather than described in a policy.
6. The evaluation set as a release gate.

Governance is being sold as a brake on AI. Built properly, it is the thing that lets you take the brake off, because you can answer the question every stakeholder is about to ask: what is this system allowed to do, and how do you know that is all it did? If you want that built into an agent you already have, or one you are planning, that is a normal engagement for us.