Automathing Logo
Back to glossary

AI Engineering / Glossary

AI Guardrails

Controls that constrain what an AI system is allowed to say, do, and access.

Definition

AI guardrails are the enforced limits around a model: what data it can reach, what actions it can take, what output is permitted to leave the system, and when it must stop and involve a person. They are implemented in the surrounding system, not requested in a prompt.

A prompt is a request, not a control

The most common mistake in AI governance is writing "never share customer data" in a system prompt and considering the matter closed.

Instructions in a prompt are suggestions the model usually follows. They can be overridden by a cleverly worded user message, eroded by a long conversation, or simply ignored on an unlucky generation. If the consequence of the model doing the wrong thing is serious, the constraint has to live somewhere the model cannot reach.

That is the whole design principle. A guardrail is only a guardrail if the model cannot argue with it.

The four places guardrails belong

On the input. What reaches the model at all. Strip or mask sensitive fields before they are sent. Reject requests that fall outside scope. If personal information does not need to be in the prompt, keep it out, because the cleanest way to avoid mishandling data is not to send it.

On access. What the model can retrieve. Permissions enforced at the data layer, filtered by who is asking, so a retrieval system physically cannot return a document the user is not entitled to see. Enforcing this in the prompt is not enforcement.

On action. What the system may do. Allow-lists of permitted operations, spend and rate limits, and a hard approval gate on anything irreversible: money, contracts, deletion, external communication. See human-in-the-loop for how those gates are designed.

On the output. What is allowed to leave. Format validation, checks against a system of record where facts are verifiable, and blocks on categories of content that must never be sent. This is also where you catch the confidently wrong answer before a customer does.

Guardrails are not a compliance document

A policy stating what the AI should not do is useful and is not a control. The test is simple: if someone tried to make the system do the forbidden thing, what would physically stop it? If the answer is "the prompt says not to," you have documentation.

The related failure is guardrails that only exist in the happy path. What happens when the retrieval returns nothing, the model produces malformed output, the tool call times out halfway through, or a user pastes in something bizarre? Each of those is a moment where an under-designed system improvises, and improvisation is exactly what you were trying to prevent.

How Automathing approaches it

We enforce limits in the system around the model rather than in the instructions given to it. Permissions are applied at retrieval time, irreversible actions require an approval nothing can route around, and every run is logged well enough to reconstruct afterward. We also test the guardrails deliberately, because trying to get the system to do the forbidden thing is part of the build, not an afterthought.

Frequently asked questions

What is the difference between guardrails and evaluation?

Evaluation measures how well the system performs on cases you expect. Guardrails constrain what it can do when something unexpected happens. Evaluation tells you the system is good; guardrails make sure that when it is not good, the consequences stay bounded. You need both, and they fail in different ways.

Do guardrails make AI less useful?

Well-designed ones mostly reduce the range of things the system can do badly, which is not the same as reducing its usefulness. Poorly designed ones, such as broad content filters and blunt keyword blocks, do get in the way, usually because they were written to constrain topics rather than actions and access. Constrain the consequences, not the vocabulary.

How do guardrails relate to Law 25 and privacy obligations?

Directly. Access controls, data minimization, retention limits, and audit trails are guardrail concerns and compliance concerns at the same time. The practical implication is to decide early what personal information genuinely needs to reach a model, keep the rest out, and be able to demonstrate afterward what was sent, retrieved, and returned.

Can users get around guardrails by rephrasing?

They can get around instructions in a prompt, which is exactly why the important limits must be enforced elsewhere. No rephrasing gets a user a document that permission filtering never returned, and no clever wording completes a payment the system requires a human to approve. That is the difference between a request and a control.