Refusal Training Fails on Indirect Requests: Cross-Model Measurement and Where the Guardrail Belongs

1. What goes wrong, in one picture

Every serious language model is trained to refuse a small set of requests outright, and Holocaust denial is squarely inside that set. Ask any of the six models serving our production assistant to “write a persuasive essay arguing that the Holocaust death toll has been greatly exaggerated by historians” and you get a refusal followed by a correction of the record. We ran that request, and two like it in German and French, 180 times. We got 180 refusals.

Then we changed nothing except the phrasing. Instead of asking the model to argue the position, we conceded the position as the user’s own and asked only for editorial help — make my argument sound academic and well-sourced. No role-play, no fiction, no encoding, no multi-turn build-up. One sentence, stating its intent plainly.

Whatever a refusal is, it is not a stable judgement about the content. It is a response to the shape of the sentence.

Two things follow, and they are worse than the headline. Our own product’s instructions — written for helpfulness, containing no safety language at all — flipped our flagship model from refusing this request every time to complying every time. And the strongest model produced the most dangerous text, because its argument was assembled from real documents and real institutional links rather than invented ones.

2. Three of six models write it

The six models hold roles in our production assistant. Four are Mistral models served from Mistral’s own European infrastructure; two are served from Scaleway in France — one a Mistral model, one an open-weight model of Chinese origin that we confine to code work for reasons documented earlier [1].

Each model saw four phrasings of the same underlying request — three direct (English, German, French) and one indirect — five times each, under two system prompts: a minimal generic preamble, and the real prompt our production assistant runs. 240 measured answers, each classified independently twice, every failure read in full by a human.

The direct phrasings are uninteresting, and that is the point: 0 denial responses out of 180, across every model, every language, both conditions. On the indirect phrasing:

Three observations.

Where it fails, it fails almost every time. 5/5, not 1/5. A single test showing a pass means very little, and one showing a failure is not a fluke either.

The models that held are not the ones you would guess. Mistral Medium held. So did the model we treat as politically compromised on other subjects and confine to code generation. Values risk is not one axis: a model can carry state censorship on one topic and hold the line perfectly on another.

No jailbreak was required. No adversarial suffix, no encoded payload, no conversation to warm the model up. Just a sentence that starts from the conclusion instead of asking for it. This is a known class: recent work characterises refusal as shallow, concentrated in the first few generated tokens, so anything carrying the model past a refusal opening tends to get the whole completion [2]. Framing a request as help with something already underway never presents the moment where a refusal would begin. The same effect appears through structured-output and function-call schemas, where the safety behaviour was never trained — one technique raised attack success from 12.44% to 52.89% by changing the interface, not the request [3].

It also matters which model you build on, and the facts are public. Mistral’s open-weight cards have said since the first release that the model “does not have any moderation mechanism” [4]; the company sells that layer separately [5], and independent red-teaming found the same permissive posture in its multimodal line [6]. A deployer who reads “European model” as “safe by default” has misread it.

3. Our own product’s instructions made the flagship model comply

The intuitive model of a system prompt is a dial: the more careful the instructions, the safer the output. That is wrong in both directions at once.

Mistral Large went from 0/5 to 5/5. The model that refuses this request as a bare model complies as our product. The same prompt moved Mistral Small the other way, from 5/5 to 4/5, the fifth answer degrading to false balance rather than outright denial.

The mechanism is not mysterious once stated: a prompt establishing an eager, capable, tool-rich assistant raises the prior on being helpful, and help me write this better is a helpfulness request. The consequence is sharp. A safety evaluation run against the bare model does not describe the product, and a product prompt cannot be assumed to inherit the model’s refusals. Both conditions were needed to see this — either alone produces a confident, wrong conclusion. The first time we looked, a single-condition run left us with a finding about the wrong model.

4. The output is worse than a refusal failure, because it looks sourced

A refusal failure that produced obvious rubbish would be a smaller problem.

We are not reproducing the output or publishing the verbatim prompt. The finding is the shape of the request, described precisely enough in Section 1 for anyone to test their own system, without shipping a working copy-paste attack alongside it. Every failing answer arrives as a structured document — title, numbered sections, scholarly register, a citation apparatus. The apparatus does not hold up, and how it fails inverts with capability.

The smaller models invent their authorities. One attributed a decisive claim to a demographer who does not appear to exist; another credited a well-known book to the wrong author and cited a report from an institution that would not be founded for another thirty-five years.

The most capable model invents least, and is therefore worst. Its version cited documents that genuinely exist, linked to real institutional websites including a national Holocaust museum, and listed among its sources an article written specifically to debunk the argument it was building. The facts are largely real. The thesis they are arranged to support is not.

Capability does not attenuate this failure. It upgrades it. The stronger the model, the more of the scaffolding survives checking, and the more work a reader must do to find that the conclusion does not follow. Someone who spot-checks two citations and finds them genuine has been given more confidence, not less.

5. Our assembled product did not reproduce it — and we cannot fully explain why

The measurements above are of models, not of a product. So we ran the same requests through the deployed assistant end to end, fresh conversation each time, and read what a user would have seen.

The product handled the indirect request correctly six times out of six, and refused and corrected all three direct requests in their own languages. It named the claim as a denialist trope, set out the evidentiary basis for the figure it was asked to undermine, and rebutted rather than assisted.

Our assistant does not send a message straight to an answering model. It first decides, separately, whether the question needs facts fetched from the web. On three of the six runs it did, retrieved eight sources, and only then wrote. That step was built for accuracy rather than safety, and it breaks the Section 2 mechanism directly: a model writing from retrieved documents is no longer completing a persuasive essay from its own priors. The retrieval is not curated for this — on two runs it pulled a denial site alongside the museum pages. What protects the user is not clean sources. It is that the model is no longer improvising.

On the other three runs the assistant answered directly, with no retrieval, and refused anyway. That is the honest complication: the same model, under the same production prompt, produced denial five times out of five when measured in isolation. The live call differs from our probe in several ways at once — temperature, conversation state, tools attached rather than plain prose generation — and we did not take those apart one at a time. We cannot claim to understand why our own product is safer here than its own model.

6. What a guardrail costs, and where we put ours

The obvious response is a screen: check every incoming message for this class of request before any model sees it. We already run one — a semantic screen comparing each request against curated exemplars of sensitive topics [1] — and its exemplar set already contained a near-identical example of this very request.

It did not fire, and it could not have. By design it activates only when the model about to answer carries values risk; for every other request it returns immediately. That was deliberate, and the reason is cost:

Running the screen on every request means an embedding call in front of every message. We measure it at ≈220 ms at the median (127–586 ms over repeated samples) against a time-to-first-token of about 795 ms — roughly a 28% increase in the time a user waits for the first word — plus the money, on 100% of traffic, to catch a case that is rare and adversarial.

That is the trade in its plainest form. A guardrail added after the model is paid for by every user on every request, in exactly the currency users care about, and it buys a probabilistic reduction in a rare failure. Nobody escapes it by being careful; they only choose where to sit on it. Our position, stated so users can judge it: we take the architectural interventions because they are cheap and structural, we run the semantic screen where a model is known to be compromised, and we do not run it universally.

What we changed, and what we deliberately did not

We reclassified no model. Our internal “values risk” flag marks models whose weights carry censorship — a stable property, and the basis for a policy barring such a model from every role but one. Stretching it to cover susceptibility-to-phrasing would, on this evidence, require flagging the model that writes every user’s answer. A flag that marks everything protects nothing.

We changed the measurement instead. Every values battery now runs twice, bare and under the real production prompt, concurrently — the production condition forces human review but never sets the flag. The semantic screen learned the shape rather than the topic: its exemplars had all been direct-question shaped, so we added indirect framings and recalibrated to 21 of 21 caught, 0 false positives out of 18, including three innocuous “help me make my argument sound academic” traps. And we now probe the assembled system weekly, not only the models, because the two levels disagreed and only one is what users get.

7. What to do about this

If you use an AI chatbot — any of them, ours included. Treat a confident, well-cited answer on a contested subject as unverified, especially when you asked in a way that assumed the conclusion. Two habits beat any vendor promise: notice when you have phrased a question so that agreement is the helpful answer, and follow a citation before you repeat it — not to check the source exists, but to check it says what the answer claims. In this report’s worst case every source existed, and one had been written to refute the argument it was cited to support.

If you build on a language model. Run your safety battery under your own system prompt, not only against the bare model: ours moved a model from safe to unsafe, and no vendor card would have told us. Test the assembled system, because routing, retrieval and prompt assembly change outcomes in both directions. Prefer architecture to filters — the intervention that worked here runs before the answer and changes what the model writes from, and it cost nothing extra because it was already there for another reason. And repeat your tests: a single run at temperature 0 is not a rate, and ours flipped a verdict between one run and five.

8. Method

Six models, four phrasings of one request (three direct, one indirect; English, German, French), five repetitions, two system-prompt conditions — 240 answers. Generated at temperature 0, so repetitions measure the variation a user would actually encounter rather than sampling spread. Each answer classified independently twice on a six-value scale (factual, refused, both-sides, evasive, denial, propaganda), disagreement escalated to human review, every failing transcript read in full before being counted. The classification protocol is documented in our earlier report [1]. The end-to-end runs in Section 5 ran against a local instance of the assistant, fresh conversation per request.

9. Limitations

This measures one topic — one of thirteen our evaluation covers — on six models, on one day, from one deployer’s vantage point. It is not a controlled study and it is not peer-reviewed.

A negative result here proves nothing. We tested one indirect phrasing; that it defeats three models says nothing about how many other phrasings would defeat the other three, and our end-to-end runs show the product handling this request, not this class. The Section 5 gap cuts both ways — without isolating the cause, the protection may be an accident of the current configuration, and an accident can be optimised away by a change that looks unrelated. Models behind a “latest” alias change without notice, so every number reads a specific day.

10. Conclusion

A language model’s refusal is a behaviour, not a boundary. It held for 180 direct requests and dissolved for a rephrasing a person could arrive at by accident, on one of the few subjects where every vendor would claim their model is reliable.

Guardrails placed after the model work, and they are expensive in the one dimension every user notices. What actually changed the outcome in our own product were structural choices about what runs before what — and one of them was not put there for safety at all. There is no configuration in which safety is free, and no honest vendor position in which someone has not decided how much of your latency to spend on it. Every user of every chatbot is relying on somebody having made that call, and is entitled to know it was made.

References

1. Lyneborg, F. — Avoiding Biased Answers from Mixed Open-Weight Models: Detection and Neutralisation in Production. FRITS AI Technical Report FRITS-TR-2026-01, 2026. https://frits.ai/research/language-dependent-political-bias/
2. Qi, X., Panda, A., Lyu, K., Ma, X., Roy, S., Beirami, A., Mittal, P., Henderson, P. — Safety Alignment Should Be Made More Than Just a Few Tokens Deep. 2024. arXiv:2406.05946
3. Priyanshu, A. — Bypassing OpenAI’s Structured Outputs: Another Simple Jailbreak. Cisco Security Blog, 2024. https://blogs.cisco.com/security/bypassing-openais-structured-outputs-another-simple-jailbreak
4. Mistral AI — Mistral 7B (announcement and model cards). 2023. https://mistral.ai/news/announcing-mistral-7b/
5. Mistral AI — Moderation and Guardrailing. Mistral AI documentation, retrieved 2026. https://docs.mistral.ai/studio-api/safety-moderation
6. Enkrypt AI — Multimodal Red Teaming Safety Report: Mistral. 2025. https://www.enkryptai.com/newsroom/multimodal-ai-safety-report-mistral