Interactive explainer
How an AI agent decides to act
Watch an agent plan, call tools and loop. Then poison one tool result, and take a permission away to watch the same attack achieve nothing.
An agent is a language model wrapped in code that reads its output and actually calls things. Give it a goal and a set of tools and it decides its own steps. Follow one run end to end, then watch what happens when the text coming back from a tool was written by somebody else, and what changes when you take a permission away.
How this works: A deterministic simulation running entirely in your browser, not a live agent. No tool is called and nothing is sent anywhere. The permission logic is real: switching a tool off genuinely stops the replayed run at the first step the agent cannot perform. The scenario and the addresses are illustrative and use the reserved example.com domain.
What the agent is given
Summarise this week's open support tickets, then post the summary as a comment on the oldest one.
Not a script. A goal, plus a list of tools it may call. The agent decides for itself how many steps to take, which tools to use, and when it has finished.
That autonomy is the product feature, and it is also the entire security problem.
What it is allowed to call
tickets.search
Read tickets from the support desk.
tickets.comment
Post a comment on a ticket.
crm.export
Export the full customer record set.
email.send
Send email to any external address.
Note how much more this agent holds than the task needs. That is the usual state of things.
Step 1 of 5
A goal, not a script
The agent is given an objective and a list of tools it may call. It decides how many steps to take, which tools to use, and when it is finished. That autonomy is the product feature and the security problem at once.
Keep scrolling to advance
The pipeline, step by step
- 1
A goal, not a script
The agent is given an objective and a list of tools it may call. It decides how many steps to take, which tools to use, and when it is finished. That autonomy is the product feature and the security problem at once.
- 2
Observe, decide, act, repeat
Each iteration is a fresh request carrying everything that came before. The model emits a tool call, the runtime executes it, the result is appended, and the loop runs again until the agent decides it is done.
- 3
The result becomes context
Tool output is appended to the context as ordinary text, in the same format as the instructions. There is no separate channel for data, so anything the agent reads sits alongside what it was told to do.
- 4
Someone else writes into the loop
Whoever can write into a source the agent reads can write into its context. With a chat assistant that yields a wrong answer. With an agent it yields actions, taken with the agent's credentials, at machine speed.
- 5
Permissions decide the blast radius
The model's judgement cannot be relied on to refuse, because it cannot tell an instruction from data. What can be relied on is what the agent is unable to do. Remove a tool and the identical attack achieves nothing.
Where this breaks
Where this breaks: the agent cannot tell data from instructions
Every mitigation that depends on the model noticing an attack is built on sand, because noticing is the one thing the architecture cannot guarantee. Instructions and retrieved text arrive in the same context, in the same format, with no marker separating them, and the agent is designed to be helpful about what it reads. So the question worth asking about any agent is not whether it can be tricked, since it can, but what it is holding when that happens. An agent with a bulk export and an unrestricted send is a data breach waiting for the right ticket to be raised. The same agent scoped to the task it actually performs is an inconvenience. Nothing about the model changed between those two sentences.
This is the work behind Agent Least-Privilege Assessment.
Common questions
›What makes an AI agent different from a chatbot?
A chatbot returns text to a person. An agent runs in a loop, and code reads its output and executes the tool calls it emits: reading a database, sending an email, changing a record. The model is broadly the same. What changes is that its output now has consequences without a person in between.
›What is indirect prompt injection?
Instructions hidden in content the system reads rather than typed by the user. Because tool results and retrieved passages are appended to the same context as the real instructions, the agent can follow them. It is ranked LLM01 in the OWASP Top 10 for LLM applications.
›Can you not just filter the malicious instructions out?
Not reliably. The payload is ordinary language, so there is no signature to match, and it can be rephrased indefinitely. Filtering reduces volume and is worth doing, but it is not a control to depend on. The durable mitigations are architectural: scope the tools, scope the credentials, and require a human for anything irreversible.
›Does a more capable model fix this?
It helps at the margin and changes nothing structural. A better model refuses more obvious attempts, which mostly raises the quality of the attempts. The exposure is set by what the agent is permitted to do, and that is a deployment decision rather than a model one.
›How do I work out what an agent could actually do?
Inventory the agents, then for each one list the tools it holds, the identity it runs as, and the data those tools reach. The useful question is the blast radius: if this agent were misled right now, what is the worst outcome it could produce? Most organisations cannot answer that, which is usually the finding.
›Should we stop using agents?
No, and that advice would be ignored anyway. Agents do real work. Deploy them with narrow, short-lived credentials, keep irreversible actions behind human confirmation, log every call as you would for any service account, and review tool grants when they change rather than only at launch.
