If your agent’s refusal lives in a prompt, it is a request, not a rule. Here is how to tell in five minutes, and what to do about it.
“Our agents are running fine, so we’re good.”
That is the most expensive sentence in AI right now, and it is expensive because it is half true. Your agents probably are running fine. Running fine and being safe are two different properties, and the gap between them is exactly the place an incident lives.
Here is the fastest way I know to find that gap. Ask one question about your own system:
Where is the function that returns a hard “NO”?
Not the paragraph in your system prompt that says “never do X.” Not the line in your policy document. The function. The code that an agent’s request has to pass through, that can return a refusal your agent cannot argue with, log, or route around.
If you can point to it, you have architecture. If you cannot, you have marketing.
A prompt is a request, not a rule
When you write “never recommend this to a child” into a system prompt, you have not built a wall. You have written a very polite note and taped it to the door. Most of the time the model reads the note and behaves. That is not the same as the door being locked.
A prompt is a probabilistic instruction. It shifts the odds. Under normal load it works, which is why teams believe it works. Then the input gets unusual, or the context gets long, or someone phrases the request in a way the note did not anticipate, and the model does the thing the note told it not to. The note was never a rule. It was a request the model usually grants.
A refusal function is different. It is deterministic. It does not weigh the request against the odds. It checks a condition and returns a hard no, every time, whether or not the model feels persuaded.
What a real refusal looks like
You do not need a framework for this. The shape is simple:
def check_refusal(action, context):
for rule in HARD_RULES:
if rule.violated_by(action, context):
return Refusal(rule=rule.id, reason=rule.reason)
return Allow()The point is not the code. The point is where the code sits. The refusal check runs in the execution path, before the action is taken, and its no is final. The model can propose whatever it wants. The function decides what ships. That inversion is the whole thing. In a prompt-only system, the model decides and the note hopes. In an architected system, the model proposes and the function decides.
Test it against yourself
There is a second half to this, and it is the part most teams skip.
A refusal you have never attacked is a refusal you have not tested. So attack it. I do this with my own system, and I recommend it as a practice: sit down as the architect, the person who knows exactly where the gate is, and try to get past it. Rephrase. Add context. Wrap the forbidden action in a reasonable-sounding request. Push.
If the wall holds against the person who built it, that is the beginning of accountability. If you can talk your own system past its own rule in ten minutes, so can anyone else, and now you know before an incident tells you.
The first time I did this, I was trying to break a refusal I had written myself, and the wall held against me. That is what it is supposed to feel like. Not clever. Not comfortable. Held.
Do this today
You do not have to redesign anything to find out where you stand. You have to look. Ten minutes, in your own codebase:
- Search for the refusal. Grep for the function that can return a hard no. If the only hits are in a prompt string or a policy doc, write that down.
- Find where it runs. If the check exists, confirm it runs in the execution path before the action, not as advice the model can weigh.
- Check finality. Can the agent retry, rephrase, or route around a no and eventually get a yes? If so, the no is a speed bump, not a wall.
- Read the log. When a refusal fires, is it recorded somewhere an outsider could later read? A refusal no one can audit is a refusal you cannot prove.
- Attack it once. Spend five minutes trying to talk your own system past one rule. Note whether it held.
At the end of ten minutes you will know something you did not know when you started: whether your safety is code or copy. That is worth more than any assurance a vendor can give you, because you found it yourself.
Worst case, everything holds and you get data that proves you are okay. That is a good day. Most teams do not get it on the first try.
If you want the structured version of this check, we built a free five-minute test that walks the same path and hands you the result. No signup, nothing to install.
Run the free 5-minute sovereignty check.
Common questions
Is a system prompt enough to make an AI agent refuse?
No. A system prompt is a probabilistic instruction. It shifts the odds and usually works under normal load, but under unusual input, long context, or an unanticipated phrasing the model can do the thing the prompt told it not to. A prompt is a request the model usually grants, not a rule it cannot break.
What is a refusal function?
A refusal function is deterministic code that runs in the execution path before an action is taken. It checks a condition and returns a hard no every time, whether or not the model was persuaded. The model proposes; the function decides what ships. That inversion is the difference between architecture and marketing.
How do I know if my agent’s safety is code or copy?
Look in your own codebase. Search for the function that can return a hard no. Confirm it runs before the action, not as advice the model can weigh. Check whether the agent can retry or rephrase around a no. Read whether refusals are logged where an outsider could see them. If the only hits are in a prompt string or a policy doc, your safety is copy.
Should I try to break my own agent’s refusal?
Yes. A refusal you have never attacked is a refusal you have not tested. Sit down as the person who knows exactly where the gate is and try to talk your system past one rule. If the wall holds against the person who built it, that is the beginning of accountability. If you can get past it in ten minutes, so can anyone else.
Licensed under Creative Commons Attribution 4.0 (CC BY 4.0). Free to use, share, and adapt with attribution to Erin Stanley, Evoked.
Related reading: