Benchmarking Arabic Dialects for Math Problems
arXiv cs.AI · 21 Oct 2026, 03:50 AEST
Mathematical reasoning in low-resource languages, particularly Dialectal Arabic, is underexplored. The DAR-MGSM Benchmark introduces a new tool for evaluating mathematical reasoning across various Arabic dialects.
Key facts
- To bridge this gap, we introduce the DAR-MGSM (Dialectal Arabic Multilingual Grade School Math) Benchmark, the first manually curated benchmark for evaluating mathematical reasoning in Dialectal Arabic.
- DAR-MGSM consists of math word problems (MWPs) requiring multi-step reasoning and covers five major Arabic dialects alongside Modern Standard Arabic (MSA).
- The benchmark comprises two parallel versions: translated-only and translated & culturally localized.
- The results show that the performance generally declines on the localized version, highlighting the challenges introduced by culturally grounded adaptations.
- Overall, our results show that mathematical reasoning performance depends on both linguistic variety and cultural context.
Read aloud
Mathematical reasoning in low-resource languages, particularly Dialectal Arabic, is underexplored. The DAR-MGSM Benchmark introduces a new tool for evaluating mathematical reasoning across various Arabic dialects. To bridge this gap, we introduce the DAR-MGSM Dialectal Arabic Multilingual Grade School Math Benchmark, the first manually curated benchmark for evaluating mathematical reasoning in Dialectal Arabic. DAR-MGSM consists of math word problems MWPs requiring multi-step reasoning and covers five major Arabic dialects alongside Modern Standard Arabic MSA. The benchmark comprises two parallel versions: translated-only and translated & culturally localized. The results show that the performance generally declines on the localized version, highlighting the challenges introduced by culturally grounded adaptations. Overall, our results show that mathematical reasoning performance depends on both linguistic variety and cultural context.
Sources in this cluster
- Benchmarking Arabic Dialects for Math Problems
arXiv cs.AI · 21 Oct 2026, 03:50 AEST