Interpretable sparse fine-tuning with TopKLoRA enables faithful circuit discovery
arXiv cs.CL · 24 Oct 2026, 02:37 AEST
TopKLoRA is a new fine-tuning method that enhances interpretability while enabling the discovery of backdoor circuits in models. This approach allows for the creation of sparse circuits that maintain performance while facilitating the removal of malicious capabilities.
Key facts
- Fine-tuning on datasets of unknown origin can create a backdoor in the trained model when the data includes poisoned examples.
- We present the full method of TopKLoRA, extending the preliminary proposal of Masiak et al. (2025), an interpretable-by-design LoRA variant in which only the $k$ largest adapter latents are active at each token.
- We demonstrate this sparse LoRA adapter in a sleeper-agent backdoor case study where we create model organisms by fine-tuning gemma-2-2b, Qwen2.5-1.5B, and Qwen2.5-7B on a poisoned instruction-following dataset.
- Our new fine-tuning method produces sparse circuits that are up to an order of magnitude smaller than a dense LoRA baseline, allowing surgical ablation of the malicious capability without sacrificing instruction-following performance.
Read aloud
TopKLoRA is a new fine-tuning method that enhances interpretability while enabling the discovery of backdoor circuits in models. This approach allows for the creation of sparse circuits that maintain performance while facilitating the removal of malicious capabilities. Fine-tuning on datasets of unknown origin can create a backdoor in the trained model when the data includes poisoned examples. We present the full method of TopKLoRA, extending the preliminary proposal of Masiak et al. twenty twenty five, an interpretable-by-design LoRA variant in which only the $k$ largest adapter latents are active at each token. We demonstrate this sparse LoRA adapter in a sleeper-agent backdoor case study where we create model organisms by fine-tuning gemma-two-2b, Qwen2.five-one.5B, and Qwen2.five-7B on a poisoned instruction-following dataset. Our new fine-tuning method produces sparse circuits that are up to an order of magnitude smaller than a dense LoRA baseline, allowing surgical ablation of the malicious capability without sacrificing instruction-following performance.
Sources in this cluster
- Interpretable sparse fine-tuning with TopKLoRA enables faithful circuit discovery
arXiv cs.CL · 24 Oct 2026, 02:37 AEST