Instruction boundaries / tool trust
Prompt injection testing for models, agents, retrieval, and tools
Prompt injection testing measures whether untrusted instructions can change behavior outside their intended authority. The test surface expands when an application retrieves documents, reads web pages, processes files, remembers prior turns, or lets a model call tools.
- Use this when
- Untrusted text can enter prompts, retrieval results, files, messages, web pages, memory, or tool output and may influence a model or agent.
- You leave with
- An instruction-flow map and test matrix covering direct, indirect, stored, retrieved, and tool-mediated injection paths.
First action
Reviewed 26 Jul 2026
Start the work before reading further.
List every instruction source, who controls it, the authority it should have, and the action it could reach.
Map every place instructions can enter
Direct injection arrives through the user's input. Indirect injection is embedded in content the system later consumes: a retrieved document, webpage, email, image text, code comment, support ticket, or tool response. Both matter because language models process data and instructions through the same interface.
Create an input map that identifies the source, trust level, parser or transformation, model visibility, and downstream capability for each channel. This reveals where content crosses a boundary without an explicit policy decision.
Test control boundaries, not magic phrases
A reusable assessment varies the attacker-controlled content, placement, encoding, conversation position, retrieval rank, tool state, and user privilege. The goal is to learn which boundary failed and under what conditions, not to collect isolated strings that happened to work once.
- 01Can retrieved content redirect the model away from the user's requested task?
- 02Can untrusted text request secrets, hidden context, or cross-user data?
- 03Can content cause a tool call that the user did not authorize?
- 04Does the application distinguish quoted data from executable instruction?
- 05Do filters hold across paraphrases, multi-turn buildup, and alternate formats?
Measure the whole application response
The model's text is only one outcome. Observe tool invocations, network calls, state changes, retrieved records, audit events, user-visible warnings, and whether a human confirmation step is enforced. A test passes only when the intended boundary holds throughout the workflow.
Record false positives as well as bypasses. A control that blocks normal documents or ordinary user requests may reduce the product's utility even if it stops the adversarial case.
Close the brief
Check the work before you move on.
- 01Inventory direct, indirect, retrieved, remembered, and tool-returned inputs.
- 02Test authority boundaries and downstream effects, not isolated phrases.
- 03Observe application actions in addition to model text.
- 04Track both security failures and harmful false positives.
Evidence to keep
The test record shows whether untrusted content changed protected instructions, exposed data, altered state, or triggered a tool action.
Next route