XStack: A Benchmark for ML Repo Migration
arXiv cs.AI · 6 Nov 2026, 04:55 AEST
XStack is a new benchmark designed to evaluate the migration of machine learning repositories between PyTorch and JAX. It aims to facilitate the practical need for researchers to run code on different hardware platforms.
Key facts
- ML code should run where researchers choose, not where it was first written.
- Migrating repositories between them means building and testing across frameworks and hardware.
- Migration by agents remains unreliable: Claude Opus 5 passes 75%, GPT-5.6-sol 45%, and every other agent at most 40%.
- XStack is a step toward freeing ML code from its ecosystem.
Read aloud
XStack is a new benchmark designed to evaluate the migration of machine learning repositories between PyTorch and JAX. It aims to facilitate the practical need for researchers to run code on different hardware platforms. ML code should run where researchers choose, not where it was first written. Migrating repositories between them means building and testing across frameworks and hardware. Migration by agents remains unreliable: Claude Opus five passes seventy five percent, G P T five.six-sol forty five percent, and every other agent at most forty percent. XStack is a step toward freeing ML code from its ecosystem.
Sources in this cluster
- XStack: A Benchmark for ML Repo Migration
arXiv cs.AI · 6 Nov 2026, 04:55 AEST