Surprise-driven continual learning absorbs most lies unless filtered by a verifier
The claim
Surprise-based continual learning for LLMs learns nearly every false claim in a stream, matching random selection performance
Published as: “Surprise Alone Learns Lies: Evidence-Gated Continual Learning for LLMs from Streaming Claims”
So what
Deployments that need parametric updates must combine a verifier with a prior-conflict filter and trusted-source evidence; otherwise surprise-driven continual learning will readily ingest falsehoods.
The five-minute version
Surprise-driven continual learning absorbs almost all false claims: the surprise-only learner asserts 99-100% of known-fact lies and 84-93% of all lies. A verifier key can reduce this to the verifier’s error rate, but a prior-conflict key cuts known-fact lies by 3-5× while saving 43% of judge calls. Regression testing throttles learning but does not filter poison, and an evidence route can recover legitimate updates without re-introducing lies only when sources are trustworthy. The evidence shows that surprise alone offers no safety benefit over random selection, and an attacker who screens lies can bypass the prior key, leaving the verifier as the sole protection. The study is limited to 1.5-3 B parameter models and fixed-in-advance poison.
What they found 2
- Surprise-only learning absorbs nearly all known-fact lies.
“Surprise-only learning absorbed 99-100% of them at every poison rate, against 68-86% of lies about invented people and 84-93% of all lies (the measure in Figure 2).”
- The prior-conflict key reduces known-fact lies by several times.
“Added to the imperfect judge, it cut the known-fact lies the student ends up asserting by 3-5× at every accuracy”
What undercuts it the paper’s own results
- Selecting by surprise neither raised nor lowered the share of lies in the training input. On known-fact lies it did worse than random selection.
Stated limits 3
- Legitimate updates of known facts. True news that changes something the model already knew (a new office holder, say) contradicts its prior just as a lie does; the prior key held back about three quarters of such updates in Section 5.6.
- Adaptive attackers. Our poison is fixed in advance. Section 5.9 shows that an attacker who screens lies with the open model passes the prior key at will, so against such an attacker only the judge protects the model.
- Small models only. Main results use Qwen2.5-1.5B; the key comparisons are repeated on Qwen2.5-3B and Llama-3.2-3B (Section 5.7), and neither strengthened the prior key. Models larger than 3B parameters were not tested.
What’s new here 4
- Shows that surprise-driven selection does not reduce factual poisoning compared to random sampling.
- Introduces a prior-conflict key that blocks known-fact lies without a judge, saving 43% of verification calls.
- Evaluates an evidence-routing mechanism that recovers legitimate updates while preventing lies, contingent on trusted sources.
- Provides a systematic benchmark combining invented entities, CounterFact counterfactuals, and legitimate updates for continual-learning poisoning studies.
Artifacts & methods 7
- Qwen2.5-1.5B LoRA-adapted studentsystem
- Surprise key (negative log-likelihood)primitive
- Verifier key (LLM judge or oracle)primitive
- Prior-conflict key (gap threshold g≥6)primitive
- Regression test (statistical rollback)primitive
- Evidence route (trusted source check)primitive
- Claim stream benchmark (invented people, CounterFact, legitimate updates)dataset