Pterodactyl: Pushing the Limits of Efficient Automated Translation Evaluation
arXiv cs.CL · 11 Nov 2026, 04:41 AEST
Pterodactyl is a new system designed for efficient automated translation evaluation. It focuses on segment-level quality score prediction and error-free segment detection.
Key facts
- Our system, Pterodactyl, was engineered specifically for subtasks 2 and 3, that is, segment-level quality score prediction and the detection of error-free segments.
- Pterodactyl is a highly customizable, ultra-low cost system optimized for binary classification in translation triage and data curation.
- It is based on an LLM-as-a-judge approach, with prompt engineering and model selection guided by ROC analysis.
- We also developed several variants of Pterodactyl that were specifically adapted for subtasks 3.
Read aloud
Pterodactyl is a new system designed for efficient automated translation evaluation. It focuses on segment-level quality score prediction and error-free segment detection. Our system, Pterodactyl, was engineered specifically for subtasks two and three, that is, segment-level quality score prediction and the detection of error-free segments. Pterodactyl is a highly customizable, ultra-low cost system optimized for binary classification in translation triage and data curation. It is based on an L L M-as-a-judge approach, with prompt engineering and model selection guided by ROC analysis. We also developed several variants of Pterodactyl that were specifically adapted for subtasks three.
Sources in this cluster
- Pterodactyl: Pushing the Limits of Efficient Automated Translation Evaluation
arXiv cs.CL · 11 Nov 2026, 04:41 AEST