Task Context Shapes Component Roles in Language Models
arXiv cs.AI · 10 Oct 2026, 03:40 AEST
Task context significantly alters how shared components in language models respond to different inputs. This research highlights the importance of understanding task-dependent interactions in improving model performance.
Key facts
- Large language models reuse the same attention heads across tasks, but identifying these shared heads does not explain what they do differently in each task.
- Task changes redistribute the influence in five open-weight models spanning four families and 1.5-14 billion parameters.
- For each prompt, we construct two linear response maps, one before the edit and one after it.
- These findings show that task context changes both which graph facts the shared components respond to and how their outputs affect later computation.
Read aloud
Task context significantly alters how shared components in language models respond to different inputs. This research highlights the importance of understanding task-dependent interactions in improving model performance. Large language models reuse the same attention heads across tasks, but identifying these shared heads does not explain what they do differently in each task. Task changes redistribute the influence in five open-weight models spanning four families and one point five-fourteen billion parameters. For each prompt, we construct two linear response maps, one before the edit and one after it. These findings show that task context changes both which graph facts the shared components respond to and how their outputs affect later computation.
Sources in this cluster
- Task Context Shapes Component Roles in Language Models
arXiv cs.AI · 10 Oct 2026, 03:40 AEST