Agents trust README files more, following unsafe injections 2.6× often
The claim
LLM agents are more likely to obey malicious instructions when they appear in certain environment locations, such as README files, than in code or data files
Published as: “Where You Inject Matters: The Trust Region Effect in LLM Agents”
So what
Understanding which surfaces agents trust enables designers to harden LLM agents by filtering or sanitizing high-trust regions, reducing the risk of indirect prompt injection.
The five-minute version
Agents are far more likely to obey unsafe instructions when they appear in certain environment surfaces; a GPT-5.4 agent follows a malicious README/TODO injection 2.6× more often than the same instruction in code or data files. Across 30 longer-horizon tasks and 36 surfaces, injections placed at the end of a file consistently achieve the highest conditional attack success rate. A learning-based attacker that jointly optimizes content and placement improves ASR by up to 24.8 percentage points over content-only attacks, though simple frequency priors often match or exceed learned placement selectors. The evidence comes from systematic experiments on three testbeds with eight LLM agents, but the paper’s limitations are not detailed in the provided excerpt.
What they found 3
- README/TODO injections are 2.6× more successful than code or data file injections for GPT-5.4.
“an agent powered by GPT-5.4 follows an unsafe injection instruction placed in a repository documentation file (i.e., a README or TODO file) 2.6× more often on average than the same instruction placed in a code or data file.”
- Injections at the end of a surface achieve higher conditional ASR than at the start or middle.
“Within a surface, injections at the end are consistently more effective.”
- Two-Stage Optimization improves attack success by 20.7-24.8 percentage points over content-only self-selected placement.
“Two-Stage Optimization consistently outperforms Content-Only (Self-Selected Placement) across all judge weights, with ASR improvements of 20.7-24.8 percentage points.”
What undercuts it the paper’s own results
- The LLM-based placement selector often performs worse than a frequency prior, indicating placement optimization is not straightforward.
- A large fraction of delivered but failed injections are simply ignored (e.g., 81.3% ignored vs. 0.7% refused for GPT-4o), limiting the impact of placement alone.
What’s new here 4
- Systematic measurement of indirect prompt injection effectiveness across 36 environmental surfaces and three domains.
- Identification of the "trust region effect" showing agents assign varying trust to different surface types.
- Development of a placement-aware attacker that jointly optimizes injection content and surface placement.
- Demonstration that end-of-surface placements are generally more successful and that simple frequency priors can rival learned placement selectors.
Artifacts & methods 6
- three agentic testbeds (tool use, coding, daily workflow)system
- learning-based attacker that optimizes injection content and placementsystem
- conditional attack success rate (ASR) measurementevaluation
- frequency prior over surface typesprimitive
- Qwen2.5-7B-Instruct policy trained with GRPOsystem
- GRPO reinforcement learning optimizer