Every Ablation Is a Dose: Counterweights and the Semblance of Self-Repair
arXiv cs.CL · 2 Oct 2026, 03:57 AEST
Ablate a component of a language model, and other components often appear to adjust and compensate. This phenomenon, termed self-repair, has been observed repeatedly, but its mechanism remains unclear.
Key facts
- Ablate a component of a language model, and other components often appear to adjust and compensate.
- This phenomenon, termed self-repair, has been observed repeatedly, but its mechanism remains unclear.
- The most systematic study to date concluded that self-repair is noisy and unlikely to have a single explanation.
- We argue that it has one: a gain already present before any ablation.
- Any intervention on a causally important component can be viewed as a point on a coordinate axis $λ$, the signed strength of a counterfactual contrast.
Read aloud
Ablate a component of a language model, and other components often appear to adjust and compensate. This phenomenon, termed self-repair, has been observed repeatedly, but its mechanism remains unclear. Ablate a component of a language model, and other components often appear to adjust and compensate. This phenomenon, termed self-repair, has been observed repeatedly, but its mechanism remains unclear. The most systematic study to date concluded that self-repair is noisy and unlikely to have a single explanation. We argue that it has one: a gain already present before any ablation. Any intervention on a causally important component can be viewed as a point on a coordinate axis $λ$, the signed strength of a counterfactual contrast.
Sources in this cluster
- Every Ablation Is a Dose: Counterweights and the Semblance of Self-Repair
arXiv cs.CL · 2 Oct 2026, 03:57 AEST