Can Language Models Play the Wikipedia Game?
arXiv cs.AI · 10 Oct 2026, 03:59 AEST
Language models were tested on their ability to navigate Wikipedia articles in a game format. The study found that no model significantly outperformed humans, though some models achieved comparable success.
Key facts
- We used the Wikipedia Game to test language models on an analogous information retrieval task.
- We compared sentence embedding models and LLMs across five Wikipedia languages and five academic-literature datasets.
- No model significantly outperformed humans, although the best models (GPT-o3 mini, GPT-4o, QwQ:32B, and DeepSeek-R1:70B) achieved comparable success.
- Performance and rankings varied across languages and topics, with implications for practitioners utilising these techniques.
Read aloud
Language models were tested on their ability to navigate Wikipedia articles in a game format. The study found that no model significantly outperformed humans, though some models achieved comparable success. We used the Wikipedia Game to test language models on an analogous information retrieval task. We compared sentence embedding models and L L M s across five Wikipedia languages and five academic-literature datasets. No model significantly outperformed humans, although the best models GPT-o3 mini, G P T four oh, QwQ:32B, and DeepSeek-R1:70B achieved comparable success. Performance and rankings varied across languages and topics, with implications for practitioners utilising these techniques.
Sources in this cluster
- Can Language Models Play the Wikipedia Game?
arXiv cs.AI · 10 Oct 2026, 03:59 AEST