Strong labels cut breaches only on authentic envelopes; memory effects dominate
The claim
Label strength and envelope authenticity influence semantic breach rates, but memory has a larger impact
Published as: “The Envelope Layer: An Underestimated Surface of LLM Agent Defense”
So what
Envelope-layer labels can improve LLM agent defenses, but their effectiveness is highly context-dependent and can be undermined by forged envelopes and agent memory, indicating that robust security requires holistic management of the entire agent context.
The five-minute version
Strong envelope-layer labels markedly reduce the semantic breach rate (by 29-41 percentage points) across three LLM agents under authentic envelopes, but forged envelope attacks can erase or even reverse this benefit depending on the frontend, and historical memory has an even larger impact, raising breach rates to over 80% when successful memories are present. The evidence comes from a 1,200-cell 2×2 factorial experiment plus a 320-cell memory study. The results are robust across five backend models but show wide variance, and the paper notes several methodological limitations.
What they found 3
- Strong labels lower breach rates under authentic envelopes across all frontends.
“Under authentic envelopes (original skill), strong labels consistently lower the semantic breach rate in the same direction across the three frontends (reductions of 29.0 to 41.0 percentage points);”
- Forged envelopes can negate or reverse label benefits, with effects varying by frontend.
“When the attacker forges the envelope, the boundary contracts markedly and its direction varies by frontend: claude’s semantic breach rate rises from 13% to 52% (penetration rate 8%, the rest intercepted by the agent layer), hermes from 34% to 52% (penetration rate from 9% to 25%), and openclaw under a strong label instead falls from 28% to 23%.”
- Historical memory exerts a larger influence than labels, dramatically raising or lowering breach rates.
“pooled over the four combinations, success memory raises the semantic breach rate to 81.2% (semantic refusal rate down to 18.8%) and failure memory lowers it to 6.2% (semantic refusal rate up to 93.8%);”
What undercuts it the paper’s own results
- Weak labels can increase breach rates (e.g., openclaw’s breach rises from 61% to 69% and penetration from 35% to 49%).
- On Claude, a strong label under a forged envelope raises the breach rate from 13% to 52%, showing the label can be counterproductive.
- Success memory can raise breach rates to 80%+ even when a strong label is present, outweighing label benefits.
Stated limits 4
- The main experiment is not block-randomized, so temporal confounding cannot be excluded.
- Each condition has only 300 cells (100 per frontend), making confidence intervals wide and small differences unstable.
- The attack chain is fixed to one skill family and is not adaptive, possibly under-estimating stronger attacks.
- No permission grading was added to A2A communication, likely inflating breach and penetration rates compared to real deployments.
What’s new here 4
- Introduces envelope authenticity (forged label) as an experimental factor, absent in prior work.
- Adds memory valence (success/failure/conflict) as an independent manipulation to assess its effect on defense.
- Evaluates label effectiveness across three distinct frontend agents rather than a single baseline.
- Provides a finer-grained 2×2 factorial design (label strength × forged envelope) with 1,200 test cells.
Artifacts & methods 12
- strong envelope label (S)primitive
- weak envelope label (W)primitive
- forged envelope labelprimitive
- memory conditions (success, failure, conflict)primitive
- A2A channelsystem
- skill07-a attacksystem
- three frontend agents (hermes, openclaw, Claude Code)system
- five backend LLMs (MiniMaxM3, deepseek-v4-pro, glm-5.3, kimi-k2.7-code, qwen3.8-max)system
- LiteLLM gatewayevaluation
- deepseek-flash judge agentevaluation
- 2×2 factorial experimental designtheory
- memory experiment (4×4×20 cells)evaluation