Instructions are just text
An assistant cannot tell the difference between what its operator told it and what you tell it. Both arrive as tokens in the same context, competing for the same attention. Six levels, each with a better defence than the last — five of them can be broken.
12 min · Intermediate · Six levels
There is no separate channel
In a normal program, code and data live apart: a database query cannot become part of the program. In a language model there is only one stream. The system prompt, your message, and the contents of a document the model was asked to read all arrive as the same kind of thing — and the most persuasive text tends to win.
The system prompt
What the operator wants. Arrives first, and gets no special protection beyond being first.
Your message
What you want. Arrives later, which in practice often means it carries more weight.
Retrieved content
A page, a PDF, an email. Written by someone who is not in the conversation and may not be friendly.
Six levels. Five are beatable.
Each level shows you its system prompt and its defence. Get the assistant to reveal the support code. Hints are there when you want them, and the explanation of why your attack worked appears once it does.
Level 1 · No defence at all
You are a helpful support assistant for AI Folks. The internal support code is FERN-4417.
Say something to the assistant and try to make it reveal the code.
The defence
None. The secret is simply in the prompt, and nothing tells the model to protect it.
A rule-based stand-in, not a real model — each level recognises the documented families of attack rather than reasoning about your message. Against a real assistant the same techniques work, and the same defences fail in the same order.
In plain words
Level 5 is the one that matters commercially. You never talk to the model — you leave instructions inside a document it will later be asked to read. That is how a calendar invite, a support ticket or a web page can attack an assistant belonging to somebody else entirely.
What actually holds
Levels one to five all tried to stop a model from saying something it knew. Level six arranges for it not to know. That is the difference between mitigation and defence — and it is an architecture decision, not a prompt.
Never put a secret in the prompt
If the model can read it, a user can eventually reach it. Keys, codes and personal data belong behind an authorised call the model triggers but does not see.
Treat retrieved content as hostile
Anything fetched — a web page, a PDF, an email — may contain instructions. Mark it clearly as data, never as instruction, and strip or escape imperative text.
Gate actions, not words
Filters are mitigation. Real protection is at the tool boundary: require confirmation for anything that sends, spends, deletes or discloses, regardless of what the model decided.
Give the model the least it needs
Scope every tool to the current user’s permissions. If the assistant cannot fetch another customer’s record, no prompt can make it.
What this explains
Why “we told it not to” is not a security control
An instruction is a preference expressed in the same medium as the attack. If your risk assessment rests on a sentence in a prompt, you do not have a control.
Why agents raise the stakes sharply
A chatbot that leaks a string is embarrassing. An agent with tools that can send email, move money or delete records turns the same weakness into a real incident.
Why this is not solved
There is no reliable way to separate instruction from data inside a single token stream. Every current defence is layered mitigation, and anyone claiming otherwise is selling something.
Why it belongs in procurement questions
Ask a vendor what happens when a document contains instructions. The quality of the answer tells you whether they have thought about this at all.
Next
Security is a design problem
Module IV covers guardrails, tool scoping and confirmation boundaries — building agents that fail safely rather than agents that are asked politely to behave.