Caught in the Act: Probes Effectively Detect Sabotage and Catch Unverbalized Deception
arXiv cs.AI · 9 Oct 2026, 03:58 AEST
Recent research demonstrates that probes can effectively detect deception in language models. The study introduces a novel architecture that achieves high accuracy in identifying unverbalized deception.
Key facts
- We show that white-box deception detection via probes can be scaled up to frontier monitoring settings by collecting the largest deception dataset to date for training probes and introducing a novel probe architecture which can aggregate information across many layers and tokens.
- Our probes achieve 98.8% AUC in SHADE-Arena, surpassing an Opus 5.5 text-monitoring baseline, and show improved efficacy as the underlying model is scaled up.
- In one such evaluation, we show that probes can distinguish transcripts containing a model's true hidden goal from other goals with an AUC of up to 99.7%.
- Our probes also readily detect deception on prominent open-weight models which lie about politically sensitive topics, and about their beliefs when put under pressure.
Read aloud
Recent research demonstrates that probes can effectively detect deception in language models. The study introduces a novel architecture that achieves high accuracy in identifying unverbalized deception. We show that white-box deception detection via probes can be scaled up to frontier monitoring settings by collecting the largest deception dataset to date for training probes and introducing a novel probe architecture which can aggregate information across many layers and tokens. Our probes achieve ninety eight point eight percent AUC in SHADE-Arena, surpassing an Opus five point five text-monitoring baseline, and show improved efficacy as the underlying model is scaled up. In one such evaluation, we show that probes can distinguish transcripts containing a model's true hidden goal from other goals with an AUC of up to ninety nine point seven percent. Our probes also readily detect deception on prominent open-weight models which lie about politically sensitive topics, and about their beliefs when put under pressure.
Sources in this cluster
- Caught in the Act: Probes Effectively Detect Sabotage and Catch Unverbalized Deception
arXiv cs.AI · 9 Oct 2026, 03:58 AEST