Maya a1: Do the Options Read the Model, or Does Their Position? Order Sensitivity in a Non-Autoregressive Decision Model
arXiv cs.CL · 7 Nov 2026, 03:54 AEST
The study investigates how the position of options affects the answers generated by a non-autoregressive decision model. Results show that the model's accuracy varies significantly based on the order of options presented.
Key facts
- Laya answers typed multiple-choice questions in one forward pass by scoring a hidden state at one [MASK] marker per option.
- Presenting 951 Qwen-verified questions under every cyclic rotation of their options, the released checkpoint changes its answer for 25.8% of questions.
- Accuracy with the gold option first versus last differs by 10.1 points, and the flips concentrate where the model's own confidence is low.
- Fine-tuning with Laya's trainer on 2,818 teacher-verified rows with option shuffling cuts the flip rate to 7.2%; the identical rows without shuffling give 11.4%.
- A confidence-weighted vote over orderings gives 69.7% against 69.1% for a single base pass at 4.1x the compute.
Read aloud
The study investigates how the position of options affects the answers generated by a non-autoregressive decision model. Results show that the model's accuracy varies significantly based on the order of options presented. Laya answers typed multiple-choice questions in one forward pass by scoring a hidden state at one MASK marker per option. Presenting nine hundred fifty one Qwen-verified questions under every cyclic rotation of their options, the released checkpoint changes its answer for twenty five point eight percent of questions. Accuracy with the gold option first versus last differs by ten point one points, and the flips concentrate where the model's own confidence is low. Fine-tuning with Laya's trainer on two thousand eight hundred eighteen teacher-verified rows with option shuffling cuts the flip rate to seven point two percent; the identical rows without shuffling give eleven point four percent. A confidence-weighted vote over orderings gives sixty nine point seven percent against sixty nine point one percent for a single base pass at four.1x the compute.
Sources in this cluster
- Maya a1: Do the Options Read the Model, or Does Their Position? Order Sensitivity in a Non-Autoregressive Decision Model
arXiv cs.CL · 7 Nov 2026, 03:54 AEST