Are These Programmers Ripe Enough to Eat? Evaluating How LLMs Handle Lexical Knowledge Conflicts in Translation
arXiv cs.CL · 7 Nov 2026, 03:44 AEST
A study evaluates how large language models (LLMs) handle conflicts between external lexical resources and their prior knowledge during translation. The research highlights the varying reliance on contextual knowledge based on word frequency and sentence context across different models.
Key facts
- External lexical resources can help large language models (LLMs) translate texts in low-resource languages and specialized domains, but these resources may conflict with models' prior lexical knowledge.
- We treat these prompt-supplied meanings as contextual knowledge (CK) and systematically evaluate the effects of word frequency, sentence context, and prompt framing on the translated outputs.
- Across nine models, CK reliance is greater for less frequent source words and in sentence contexts that support the supplied meanings.
- Prompt framing can reduce CK reliance, but seven of nine models favor CK even when explicitly instructed to ignore the supplied meanings.
- Overall, our controlled study reveals how CK reliance varies across conditions and models, motivating new methods to detect and evaluate lexical conflicts and guide models' lexical choices in LLM-based translation.
Read aloud
A study evaluates how large language models L L M s handle conflicts between external lexical resources and their prior knowledge during translation. The research highlights the varying reliance on contextual knowledge based on word frequency and sentence context across different models. External lexical resources can help large language models L L M s translate texts in low-resource languages and specialized domains, but these resources may conflict with models' prior lexical knowledge. We treat these prompt-supplied meanings as contextual knowledge CK and systematically evaluate the effects of word frequency, sentence context, and prompt framing on the translated outputs. Across nine models, CK reliance is greater for less frequent source words and in sentence contexts that support the supplied meanings. Prompt framing can reduce CK reliance, but seven of nine models favor CK even when explicitly instructed to ignore the supplied meanings. Overall, our controlled study reveals how CK reliance varies across conditions and models, motivating new methods to detect and evaluate lexical conflicts and guide models' lexical choices in L L M-based translation.
Sources in this cluster
- Are These Programmers Ripe Enough to Eat? Evaluating How LLMs Handle Lexical Knowledge Conflicts in Translation
arXiv cs.CL · 7 Nov 2026, 03:44 AEST