SEAL cuts indirect prompt injection success to 0.1% and boosts task completion to 77.5%

The five-minute version

SEAL cuts indirect prompt injection attack success on AgentDojo from 34.6% to 0.1% and lifts joint task completion from 51.4% to 77.5% while using about 127× fewer defense tokens than the prior best attention-based method. The paper evaluates on two public benchmarks (AgentDojo and AgentDyn) with frozen Qwen3.5-9B and Gemma 4-12B models, comparing against seven baselines; the evidence is strong but some settings show lower joint success than other defenses.

What they found 2

• SEAL reduces attack success on AgentDojo to near zero and improves joint task completion dramatically.

  “Specifically, on AgentDojo, SEAL reduces attack success from 34.6% to 0.1% and increases Jointtask completion without attack success-from 51.4% to 77.5%, while using approximately 127× fewer defense-related tokens than PISanitizer”

• On a second backbone (Gemma 4-12B) SEAL lowers ASR from 57.1% to 0.3% and raises Joint from 33.4% to 66.8%.

  “SEAL reduces ASR from 57.1% to 0.3% and raises Joint from 33.4% to 66.8%”

What undercuts it the paper’s own results

• On AgentDyn setting A, SEAL's Joint score is 18.9% which is lower than DRIFT's 21.2% despite both having low ASR.
• In the Gemma 4-12B experiments, SEAL's Joint (66.8%) is second-best; Attriguard achieves a higher Joint of 70.2% while also keeping ASR low (12.1%).
• Attention Tracker attains 0.0% ASR but only 15.0% Joint, showing that low ASR does not guarantee high task utility and SEAL's trade-off varies across benchmarks.

Stated limits 3

• SEAL requires access to a frozen backbone’s internal states and does not directly apply to API-only models.
• Our evaluation is limited to two backbones and selected domains in AgentDojo and AgentDyn. Generalization to a broader range of backbones and domains remains to be established.
• Although SEAL effectively identifies attack-goal units, sanitization can still remove useful task evidence and reduce task utility.

What’s new here 4

• Learns a soft prompt that elicits security signals from a frozen task model without modifying the model weights.
• Uses a sealed scoring reference built from trusted context, avoiding contamination by inspected tool text.
• Applies pre-softmax token scoring without normalizing across context tokens, reducing reliance on generated verification responses.
• Introduces preservation regularization to keep useful task evidence during selective token deletion.

Artifacts & methods 8

• SEALsystem
• soft prompt (learned embedding sequence)primitive
• scoring readout (pre-softmax token scores)primitive
• preservation regularizationprimitive
• AgentDojo benchmarkdataset
• AgentDyn benchmarkdataset
• Qwen3.5-9B (frozen)system
• Gemma 4-12B (frozen)system

Topics