ROSE: Risk-Aware Orthogonal Subspace Navigation for Lifelong Knowledge Editing in Multimodal Large Language Models
arXiv cs.CL · 7 Nov 2026, 03:32 AEST
ROSE is a new framework designed for lifelong knowledge editing in Multimodal Large Language Models. It aims to mitigate issues like Locality Erosion and Catastrophic Forgetting by restricting updates to specific parameter subspaces.
Key facts
- Multimodal Large Language Models (MLLMs) require continuous factual updates to stay current, yet lifelong knowledge editing remains a daunting challenge.
- The primary obstacle is the severe Locality Erosion and Catastrophic Forgetting, where sequential edits inevitably interfere with pre-trained capabilities and prior updates.
- ROSE addresses forgetting by aiming to restrict parameter updates to the orthogonal complement of previously edited subspaces.
- Crucially, it introduces a risk-guided mechanism that utilizes a dynamically computed risk mask to safeguard specific factual parameter subspaces essential for model stability.
- Comprehensive experiments on two prominent multimodal benchmarks show that ROSE is competitive with state-of-the-art approaches across five key metrics, while maintaining outstanding stability over $1,000$ sequential updates.
Read aloud
ROSE is a new framework designed for lifelong knowledge editing in Multimodal Large Language Models. It aims to mitigate issues like Locality Erosion and Catastrophic Forgetting by restricting updates to specific parameter subspaces. Multimodal Large Language Models MLLMs require continuous factual updates to stay current, yet lifelong knowledge editing remains a daunting challenge. The primary obstacle is the severe Locality Erosion and Catastrophic Forgetting, where sequential edits inevitably interfere with pre-trained capabilities and prior updates. ROSE addresses forgetting by aiming to restrict parameter updates to the orthogonal complement of previously edited subspaces. Crucially, it introduces a risk-guided mechanism that utilizes a dynamically computed risk mask to safeguard specific factual parameter subspaces essential for model stability. Comprehensive experiments on two prominent multimodal benchmarks show that ROSE is competitive with state-of-the-art approaches across five key metrics, while maintaining outstanding stability over one thousand dollars$ sequential updates.
Sources in this cluster
- ROSE: Risk-Aware Orthogonal Subspace Navigation for Lifelong Knowledge Editing in Multimodal Large Language Models
arXiv cs.CL · 7 Nov 2026, 03:32 AEST