Injected instruction (prompt injection): what it is and how it is attacked
Text written to be read by an automated system rather than by a person, telling it to do something its owner did not ask for.
Also known as
- prompt injection
- indirect prompt injection
- hostile prompt
- poisoned content
- instruction injection
Typing any of them into the editor finds this object.
Why it matters on a security diagram
It is not a flaw in any product and there is no patch for it. A model reading a page, a document or a message has no reliable way to tell material it was given from instructions it was told to follow, so hostile text placed anywhere it will read becomes an action taken with the organisation's own credentials.
How it gets attacked, and what reduces it
How it gets attacked
- Placed in a page, a document or an email the assistant will read while doing its job
- Hidden from the person and visible to the machine, in white text, a comment, an image caption or metadata
- Written to make the resulting action look ordinary in the record, because it is performed by an authorised account
What reduces it
- Treat everything the model reads as untrusted input, in the same way a form field is treated, and never as instruction
- Put the boundary on the action rather than the text: require a person to approve anything that sends, pays, deletes or grants
- Give the assistant its own account with the least access that lets it work, so an injected instruction inherits little
- Log what the assistant read as well as what it did, because the record otherwise shows only an authorised account behaving normally
- Accept that filtering the text is a partial control at best; the input is natural language and the ways to say the same thing are unbounded
Where it sits
- Group
- Threats · The adversary and the things they bring.
- Whose side, by default
- Attacker · Working against the organisation.
- Catalogue identifier
- injected-instruction
Reviewed . CC BY 4.0.
Others in threats
- Extortion siteleak site · data leak site · DLS · shame site · negotiation portal · victim blogThe page an extortion group uses to name victims and publish what they took.
- Look-alike domaintyposquat · cousin domain · homoglyph domain · impersonating domain · spoofed domainA name registered to be mistaken for a real one at a glance.
The threats group carries every object in it, including the ones with no page of their own.
Injected instruction on your own diagram
Open the editor, press N, and type prompt injection. The object is placed and connected to whatever was selected, and Tab adds the next one already joined to it. Nothing is uploaded: the page is served with a Content Security Policy that forbids the browser from making any outbound request at all.
Open the diagram maker