Steering beyond Language: Improving Multilingual Training-Free Text Embeddings via Paired-Residual Subspace Removal
arXiv cs.CL · 17 Oct 2026, 03:30 AEST
A new study explores multilingual training-free text embeddings in large language models (LLMs). The proposed method, Paired-Residual Subspace Removal (PRSR), aims to enhance embedding quality by addressing language-specific variations.
Key facts
- While deriving text embeddings directly from frozen LLMs has emerged as a promising direction, existing methods focus almost exclusively on English, leaving multilingual settings largely unexplored.
- Our analysis reveals a language-conditioned embedding geometry in which semantically equivalent sentences remain misaligned across languages.
- To this end, we propose Paired-Residual Subspace Removal (PRSR), a training-free inference-time intervention that steers LLM representations toward a more language-invariant semantic space.
- Empirical results demonstrate that PRSR consistently improves the performance of existing training-free baselines in both monolingual and cross-lingual settings across different LLMs.
Read aloud
A new study explores multilingual training-free text embeddings in large language models L L M s. The proposed method, Paired-Residual Subspace Removal PRSR, aims to enhance embedding quality by addressing language-specific variations. While deriving text embeddings directly from frozen L L M s has emerged as a promising direction, existing methods focus almost exclusively on English, leaving multilingual settings largely unexplored. Our analysis reveals a language-conditioned embedding geometry in which semantically equivalent sentences remain misaligned across languages. To this end, we propose Paired-Residual Subspace Removal PRSR, a training-free inference-time intervention that steers L L M representations toward a more language-invariant semantic space. Empirical results demonstrate that PRSR consistently improves the performance of existing training-free baselines in both monolingual and cross-lingual settings across different L L M s.
Sources in this cluster
- Steering beyond Language: Improving Multilingual Training-Free Text Embeddings via Paired-Residual Subspace Removal
arXiv cs.CL · 17 Oct 2026, 03:30 AEST